2024-09-13T17:06:21.314 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T17:06:21.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:06:21.446 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881 branch: wip-rishabh-testing-20240910.100730-reef description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7903881' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACGxdgvOZty75kNqkgJiTJctKW1jYMaGavGxrke1pIpLv/f7Izan8YYo9UlnVaQ/OUtoIpkb57bGs0Sdz10k6Y= smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmlYPvJsyvY5kxUEoRz3hK6CjpNivM3FpU1YbaClL2xI0livb8GQBhdrtYOSQWE2f6XkXP9ffYTaY7VcEDPDOc= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEMQPAP7GtLkN0IbnPOXkvDy+N6Lanid53zR7mcB+2O8thgRVyYyrMI0BjccoEWzMTQ39UUFfmGyOx+RCs0QRkQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T17:06:21.447 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T17:06:21.447 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T17:06:21.448 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T17:06:21.449 INFO:teuthology.task.internal:Checking packages... 2024-09-13T17:06:21.470 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T17:06:21.470 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:06:21.470 INFO:teuthology.packaging:ref: None 2024-09-13T17:06:21.470 INFO:teuthology.packaging:tag: None 2024-09-13T17:06:21.470 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:06:21.470 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:06:21.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:06:21.625 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860.g6c6b083f 2024-09-13T17:06:21.626 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T17:06:21.635 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T17:06:21.635 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T17:06:21.672 INFO:teuthology.task.internal:Saving configuration 2024-09-13T17:06:21.692 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T17:06:21.731 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T17:06:21.765 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:01:22.213535', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACGxdgvOZty75kNqkgJiTJctKW1jYMaGavGxrke1pIpLv/f7Izan8YYo9UlnVaQ/OUtoIpkb57bGs0Sdz10k6Y='} 2024-09-13T17:06:21.797 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:01:22.212447', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmlYPvJsyvY5kxUEoRz3hK6CjpNivM3FpU1YbaClL2xI0livb8GQBhdrtYOSQWE2f6XkXP9ffYTaY7VcEDPDOc='} 2024-09-13T17:06:21.827 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:01:22.211113', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEMQPAP7GtLkN0IbnPOXkvDy+N6Lanid53zR7mcB+2O8thgRVyYyrMI0BjccoEWzMTQ39UUFfmGyOx+RCs0QRkQ='} 2024-09-13T17:06:21.827 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T17:06:21.834 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-13T17:06:21.835 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-13T17:06:21.835 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-13T17:06:21.835 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T17:06:21.994 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4e0dd8bbe0>, signals=[15]) 2024-09-13T17:06:21.994 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T17:06:22.001 INFO:teuthology.task.internal:Opening connections... 2024-09-13T17:06:22.001 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:06:22.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:06:22.088 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T17:06:22.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:06:22.174 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-13T17:06:22.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:06:22.257 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T17:06:22.265 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-09-13T17:06:22.283 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-09-13T17:06:22.283 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-09-13T17:06:22.341 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-09-13T17:06:22.341 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-09-13T17:06:22.341 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-09-13T17:06:22.341 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:06:22.342 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:06:22.343 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-09-13T17:06:22.374 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-13T17:06:22.392 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-13T17:06:22.393 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-13T17:06:22.450 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-13T17:06:22.450 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-13T17:06:22.450 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-13T17:06:22.450 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:06:22.451 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:06:22.451 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-13T17:06:22.482 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-13T17:06:22.501 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-13T17:06:22.501 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:06:22.559 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:06:22.560 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:06:22.560 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:06:22.560 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-13T17:06:22.599 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T17:06:22.609 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T17:06:22.654 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T17:06:22.654 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:06:22.656 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:06:22.659 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:06:22.676 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T17:06:22.687 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T17:06:22.688 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:06:22.713 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:06:22.716 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:06:22.822 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T17:06:22.926 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T17:06:22.942 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T17:06:22.942 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T17:06:22.942 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:06:22.943 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:06:22.943 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:06:22.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:06:22.961 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-09-13T17:06:22.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:06:22.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:06:22.963 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-13T17:06:22.963 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-13T17:06:23.019 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:06:23.019 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-503.el9.x86_64 2024-09-13T17:06:23.019 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-09-13T17:06:23.020 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:06:23.020 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-503.el9.x86_64 2024-09-13T17:06:23.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-09-13T17:06:23.021 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:06:23.022 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-503.el9.x86_64 2024-09-13T17:06:23.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-09-13T17:06:27.039 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-13T17:06:27.311 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.2 MB 00:02 2024-09-13T17:06:28.558 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-13T17:06:29.265 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-13T17:06:32.559 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 20 MB 00:02 2024-09-13T17:06:33.591 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-13T17:06:35.925 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-13T17:06:36.834 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-13T17:06:37.037 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-13T17:06:38.002 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 20 MB 00:09 2024-09-13T17:06:38.335 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-13T17:06:39.339 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-13T17:06:42.244 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-13T17:06:42.693 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-13T17:06:43.745 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-13T17:06:44.674 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-13T17:06:44.744 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T17:06:44.749 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:06:44.749 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:06:44.840 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T17:06:44.850 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T17:06:44.850 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-09-13T17:06:44.851 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:06:44.852 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 79 M 2024-09-13T17:06:44.852 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 126 M 2024-09-13T17:06:44.852 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T17:06:45.019 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 188 kB/s | 23 kB 00:00 2024-09-13T17:06:45.628 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 47 MB/s | 29 MB 00:00 2024-09-13T17:06:46.212 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-13T17:06:46.612 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-13T17:06:46.612 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:06:46.612 INFO:teuthology.orchestra.run.smithi044.stdout:Total 45 MB/s | 79 MB 00:01 2024-09-13T17:06:46.709 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:06:46.709 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:06:46.806 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:06:46.823 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T17:06:46.824 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T17:06:46.824 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-13T17:06:46.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:06:46.824 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 79 M 2024-09-13T17:06:46.824 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 126 M 2024-09-13T17:06:46.824 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T17:06:47.137 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 78 kB/s | 23 kB 00:00 2024-09-13T17:06:47.321 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T17:06:47.726 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T17:06:47.727 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T17:06:48.694 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T17:06:48.695 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T17:06:50.148 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-13T17:06:51.283 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T17:06:51.947 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:06:52.046 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:06:52.161 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:06:52.162 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:06:52.266 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-13T17:06:52.273 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-13T17:06:52.273 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-09-13T17:06:52.273 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-13T17:06:52.273 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-13T17:06:52.273 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-09-13T17:06:52.274 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:06:52.275 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 79 M 2024-09-13T17:06:52.275 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 126 M 2024-09-13T17:06:52.275 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-13T17:06:52.523 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 198 kB/s | 23 kB 00:00 2024-09-13T17:06:54.854 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:06:54.993 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:06:59.506 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-09-13T17:07:00.118 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:07:00.160 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:01.901 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-13T17:07:04.256 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.4 MB/s | 29 MB 00:11 2024-09-13T17:07:05.298 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:08.750 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 1.6 MB/s | 35 MB 00:21 2024-09-13T17:07:08.789 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.1 MB/s | 35 MB 00:16 2024-09-13T17:07:08.789 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:07:08.789 INFO:teuthology.orchestra.run.smithi100.stdout:Total 4.8 MB/s | 79 MB 00:16 2024-09-13T17:07:09.117 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 1.3 MB/s | 29 MB 00:21 2024-09-13T17:07:09.117 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:07:09.118 INFO:teuthology.orchestra.run.smithi070.stdout:Total 3.5 MB/s | 79 MB 00:22 2024-09-13T17:07:09.472 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-13T17:07:09.800 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T17:07:09.805 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-13T17:07:09.805 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-13T17:07:10.148 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T17:07:10.148 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T17:07:10.747 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-13T17:07:10.748 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-13T17:07:11.110 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T17:07:11.110 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T17:07:13.169 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-13T17:07:13.555 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T17:07:13.881 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:07:13.965 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:07:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:07:14.367 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:07:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:07:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:07:17.276 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:07:17.421 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:07:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:07:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:22.239 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:07:22.337 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:26.671 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:27.182 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:44.060 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:44.101 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:44.451 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:44.451 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:07:44.451 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:07:44.451 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:07:45.325 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:07:45.325 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:07:45.325 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T17:07:45.325 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:07:45.326 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:07:45.326 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:07:45.326 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:07:45.326 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:07:45.326 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T17:07:45.706 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:07:45.798 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:07:45.798 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:07:45.798 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:07:45.798 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:07:45.798 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:07:45.798 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T17:07:45.799 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi044.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:07:45.799 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:07:45.858 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:07:45.858 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:07:45.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -qi grub2-tools 2024-09-13T17:07:45.889 INFO:teuthology.orchestra.run.smithi044.stdout:Name : grub2-tools 2024-09-13T17:07:45.889 INFO:teuthology.orchestra.run.smithi044.stdout:Epoch : 1 2024-09-13T17:07:45.889 INFO:teuthology.orchestra.run.smithi044.stdout:Version : 2.06 2024-09-13T17:07:45.889 INFO:teuthology.orchestra.run.smithi044.stdout:Release : 82.el9 2024-09-13T17:07:45.889 INFO:teuthology.orchestra.run.smithi044.stdout:Architecture: x86_64 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Group : Unspecified 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Size : 8274670 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:License : GPLv3+ 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Packager : builder@centos.org 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Vendor : CentOS 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Summary : Support tools for GRUB. 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout:Description : 2024-09-13T17:07:45.890 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:07:45.891 INFO:teuthology.orchestra.run.smithi044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:07:45.891 INFO:teuthology.orchestra.run.smithi044.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:07:45.891 INFO:teuthology.orchestra.run.smithi044.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:07:45.891 INFO:teuthology.orchestra.run.smithi044.stdout:hardware devices. 2024-09-13T17:07:45.891 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:07:45.891 INFO:teuthology.orchestra.run.smithi044.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:07:45.892 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:07:45.892 INFO:teuthology.task.kernel:Updating grub on smithi044 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:07:45.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:07:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-09-13T17:07:47.858 INFO:teuthology.orchestra.run.smithi044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:07:47.887 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-09-13T17:07:47.889 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-09-13T17:07:47.904 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.9ic9v3JHsB 2024-09-13T17:07:47.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9ic9v3JHsB 2024-09-13T17:07:47.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.9ic9v3JHsB 2024-09-13T17:07:48.123 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.9ic9v3JHsB is 6KB 2024-09-13T17:07:48.171 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.9ic9v3JHsB 2024-09-13T17:07:48.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:07:48.248 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:07:48.249 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:07:48.249 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:07:48.249 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:07:48.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:07:48.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo shutdown -r now 2024-09-13T17:08:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:08:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:08:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:08:05.473 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:05.782 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:05.782 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:08:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-13T17:08:05.782 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:08:05.783 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:08:05.783 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:08:05.783 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:08:05.783 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:08:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-13T17:08:05.861 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:05.861 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:08:05.861 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:08:05.861 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:08:06.148 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:08:06.275 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:08:06.276 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:08:06.276 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:08:06.276 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:08:06.276 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:08:06.276 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-13T17:08:06.276 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:08:06.276 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:08:06.334 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:08:06.334 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:08:06.334 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-09-13T17:08:06.369 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-09-13T17:08:06.369 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-09-13T17:08:06.369 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-09-13T17:08:06.369 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 82.el9 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8274670 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-09-13T17:08:06.370 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:08:06.371 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:08:06.372 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:08:06.372 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:08:06.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:08:06.678 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T17:08:07.030 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:08:07.033 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-13T17:08:07.120 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:08:07.121 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:08:07.121 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:08:07.121 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:08:07.121 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:08:07.121 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T17:08:07.121 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:08:07.121 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:08:07.180 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:08:07.180 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:08:07.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-13T17:08:07.217 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-13T17:08:07.217 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-13T17:08:07.218 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:08:07.219 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:08:07.220 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:08:07.220 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:08:07.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:08:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-13T17:08:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:08:08.447 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-13T17:08:08.449 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-13T17:08:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.5JCMFINrKL 2024-09-13T17:08:08.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5JCMFINrKL 2024-09-13T17:08:08.531 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.5JCMFINrKL 2024-09-13T17:08:08.673 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.5JCMFINrKL is 6KB 2024-09-13T17:08:08.721 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.5JCMFINrKL 2024-09-13T17:08:08.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:08:08.800 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:08:08.800 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:08:08.800 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:08:08.800 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:08:08.801 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:08:08.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-13T17:08:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:08:09.216 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-13T17:08:09.219 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-13T17:08:09.234 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.GlBuHgs5rG 2024-09-13T17:08:09.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GlBuHgs5rG 2024-09-13T17:08:09.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.GlBuHgs5rG 2024-09-13T17:08:09.439 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.GlBuHgs5rG is 6KB 2024-09-13T17:08:09.489 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.GlBuHgs5rG 2024-09-13T17:08:09.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:08:09.568 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:08:09.569 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:08:09.569 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:08:09.569 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:08:09.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:08:09.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-13T17:08:18.417 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T17:08:18.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T17:08:18.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:08:36.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-09-13T17:08:38.964 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-13T17:08:38.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:08:38.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:08:39.728 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T17:08:39.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T17:08:39.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:08:45.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T17:08:45.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:08:57.537 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-13T17:08:58.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-13T17:09:06.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:09:06.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:09:07.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T17:09:07.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:09:09.633 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-13T17:09:10.365 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-13T17:09:21.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:09:21.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:09:22.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T17:09:22.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:09:40.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-13T17:09:45.817 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T17:09:54.116 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-13T17:09:54.581 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T17:09:54.581 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:09:54.581 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-13T17:09:54.595 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:09:54.596 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:09:54.596 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:09:54.596 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:09:55.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:09:55.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:09:55.357 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-13T17:09:55.597 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T17:09:55.597 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:09:55.684 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-09-13T17:09:55.705 DEBUG:teuthology.parallel:result is None 2024-09-13T17:09:55.732 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:09:55.732 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:09:55.733 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-13T17:09:55.747 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:09:55.747 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:09:55.747 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:09:55.747 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:09:56.747 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-13T17:09:56.748 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:09:56.853 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-09-13T17:09:56.875 DEBUG:teuthology.parallel:result is None 2024-09-13T17:09:57.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T17:09:57.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:09:58.197 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-09-13T17:09:58.554 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T17:09:58.554 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:09:58.554 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-09-13T17:09:58.571 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:09:58.571 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:09:58.572 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:09:58.572 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:09:59.573 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T17:09:59.573 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:09:59.646 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-09-13T17:09:59.671 DEBUG:teuthology.parallel:result is None 2024-09-13T17:09:59.671 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T17:09:59.678 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T17:09:59.678 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:09:59.680 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:09:59.683 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:09:59.702 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T17:09:59.719 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T17:09:59.769 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T17:09:59.770 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:09:59.772 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:09:59.774 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:09:59.868 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T17:09:59.874 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T17:09:59.875 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:09:59.877 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:09:59.880 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:09:59.916 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:09:59.916 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:09:59.918 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:09:59.927 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:09:59.927 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:09:59.930 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:09:59.932 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T17:09:59.944 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T17:09:59.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:09:59.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:09:59.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:10:00.001 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T17:10:00.041 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T17:10:00.041 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:10:00.044 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:10:00.046 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:10:00.064 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:10:00.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:10:00.195 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:10:00.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:10:00.323 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:10:00.323 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:10:00.390 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:10:00.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:10:00.494 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:10:00.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:10:00.622 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:10:00.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:10:00.689 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:10:00.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:10:00.787 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:10:00.881 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:10:00.918 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:10:00.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:10:00.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-09-13T17:10:00.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-13T17:10:00.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-13T17:10:01.033 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:10:01.040 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:10:01.081 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:10:01.565 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T17:10:01.572 INFO:teuthology.task.internal:Starting timer... 2024-09-13T17:10:01.572 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T17:10:01.623 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T17:10:01.672 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-13T17:10:01.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-09-13T17:10:01.744 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-09-13T17:10:01.746 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-13T17:10:01.810 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-13T17:10:01.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-09-13T17:10:01.867 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-09-13T17:10:01.869 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T17:10:01.869 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-09-13T17:10:01.898 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-09-13T17:10:01.898 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-13T17:10:01.930 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-13T17:10:01.931 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-09-13T17:10:01.954 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-09-13T17:10:01.954 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2024-09-13T17:10:01.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T17:10:01.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:10:01.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T17:10:02.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:10:02.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T17:10:02.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:10:02.041 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T17:10:02.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:10:02.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:10:02.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:10:02.145 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T17:10:02.155 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T17:10:02.166 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T17:10:02.167 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi070.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T17:13:48.830 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2024-09-13T17:13:48.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T17:13:48.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:13:48.937 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-09-13T17:13:49.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T17:13:49.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T17:13:49.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:13:49.105 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-13T17:13:49.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T17:13:49.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:13:49.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:13:49.267 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-13T17:13:49.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-13T17:13:49.350 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T17:13:49.360 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T17:13:49.360 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:13:49.360 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:13:49.363 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:13:49.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:13:49.365 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:13:49.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:13:49.394 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:13:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:13:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:13:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:13:49.420 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:13:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-13T17:13:49.435 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:13:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-09-13T17:13:49.444 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-13T17:13:49.453 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-09-13T17:13:49.458 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:13:49.460 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-09-13T17:13:49.468 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:13:49.471 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-09-13T17:13:49.473 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:13:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:13:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:13:49.500 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:13:49.529 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:13:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:13:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-13T17:13:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.536 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:13:49.540 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:13:49.540 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-09-13T17:13:49.541 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.541 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.541 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.541 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:13:49.556 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:13:49.556 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-13T17:13:49.557 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.557 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.557 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.557 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:13:49.558 INFO:teuthology.run_tasks:Running task install... 2024-09-13T17:13:49.565 DEBUG:teuthology.task.install:project ceph 2024-09-13T17:13:49.565 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T17:13:49.566 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:13:49.566 INFO:teuthology.task.install:Using flavor: default 2024-09-13T17:13:49.571 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T17:13:49.571 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T17:13:49.571 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:13:49.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:13:49.573 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:13:49.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:13:49.575 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:13:49.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:13:49.724 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T17:13:49.724 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T17:13:49.736 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T17:13:49.737 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T17:13:49.749 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T17:13:49.749 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T17:13:49.845 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:13:49.845 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:13:49.846 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:13:49.855 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:13:49.855 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:13:49.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:13:49.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:13:49.868 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:13:49.868 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:13:49.875 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T17:13:49.875 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:13:49.881 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T17:13:49.881 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:13:49.894 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T17:13:49.894 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:13:49.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:13:49.946 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:13:49.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:13:50.021 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:13:50.022 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:13:50.034 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:13:50.085 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-13T17:13:50.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-13T17:13:50.088 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-09-13T17:13:50.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-09-13T17:13:50.097 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-09-13T17:13:50.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-09-13T17:13:50.308 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-13T17:13:50.327 INFO:teuthology.orchestra.run.smithi044.stdout:45 files removed 2024-09-13T17:13:50.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T17:13:50.341 INFO:teuthology.orchestra.run.smithi100.stdout:45 files removed 2024-09-13T17:13:50.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T17:13:50.367 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T17:13:50.906 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 271 kB/s | 78 kB 00:00 2024-09-13T17:13:50.913 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 301 kB/s | 78 kB 00:00 2024-09-13T17:13:50.925 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 284 kB/s | 78 kB 00:00 2024-09-13T17:13:51.190 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T17:13:51.197 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T17:13:51.209 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T17:13:51.458 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-13T17:13:51.473 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-13T17:13:51.485 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-13T17:13:51.991 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-13T17:13:52.340 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.2 MB 00:00 2024-09-13T17:13:52.910 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.2 MB 00:01 2024-09-13T17:13:53.892 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-13T17:13:54.974 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-13T17:13:56.377 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 20 MB 00:02 2024-09-13T17:13:59.300 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-13T17:13:59.309 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-13T17:14:00.868 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-13T17:14:00.886 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-13T17:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-13T17:14:02.572 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-13T17:14:02.689 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-13T17:14:03.222 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-13T17:14:04.176 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-13T17:14:07.994 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T17:14:08.378 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-09-13T17:14:09.615 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T17:14:10.356 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:14:10.357 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:14:10.370 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:14:10.371 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:14:10.372 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:14:10.373 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T17:14:10.375 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T17:14:10.378 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T17:14:10.379 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T17:14:10.440 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T17:14:10.459 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-13T17:14:10.459 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-13T17:14:10.459 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-13T17:14:10.459 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T17:14:10.460 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T17:14:10.461 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-13T17:14:10.462 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T17:14:10.463 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:14:10.464 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T17:14:10.465 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T17:14:10.466 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:14:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:14:10.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:14:10.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T17:14:10.470 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout:Install 130 Packages 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 8 Packages 2024-09-13T17:14:10.471 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:14:10.472 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 203 M 2024-09-13T17:14:10.472 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T17:14:10.799 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:14:10.800 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:14:10.812 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:14:10.814 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:14:10.814 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:14:10.815 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T17:14:10.817 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T17:14:10.820 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T17:14:10.821 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T17:14:10.888 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T17:14:10.904 INFO:teuthology.orchestra.run.smithi044.stdout:====================================================================================== 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout:====================================================================================== 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T17:14:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T17:14:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T17:14:10.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T17:14:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T17:14:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T17:14:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:14:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:14:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:14:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:14:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:14:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T17:14:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout:====================================================================================== 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout:Install 130 Packages 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 8 Packages 2024-09-13T17:14:10.917 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:14:10.918 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 203 M 2024-09-13T17:14:10.918 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T17:14:11.312 INFO:teuthology.orchestra.run.smithi070.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-13T17:14:11.413 INFO:teuthology.orchestra.run.smithi044.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-13T17:14:11.505 INFO:teuthology.orchestra.run.smithi070.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 4.3 MB/s | 852 kB 00:00 2024-09-13T17:14:11.587 INFO:teuthology.orchestra.run.smithi070.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 1.7 MB/s | 142 kB 00:00 2024-09-13T17:14:11.712 INFO:teuthology.orchestra.run.smithi070.stdout:(4/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 9.7 MB/s | 5.1 MB 00:00 2024-09-13T17:14:11.805 INFO:teuthology.orchestra.run.smithi070.stdout:(5/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 16 MB/s | 1.5 MB 00:00 2024-09-13T17:14:11.872 INFO:teuthology.orchestra.run.smithi070.stdout:(6/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 7.4 MB/s | 2.1 MB 00:00 2024-09-13T17:14:11.959 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:14:11.960 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:14:11.970 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:14:11.971 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:14:11.972 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:14:11.973 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T17:14:11.974 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T17:14:11.976 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T17:14:11.977 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T17:14:12.038 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-13T17:14:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-09-13T17:14:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-09-13T17:14:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-09-13T17:14:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-13T17:14:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T17:14:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T17:14:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:14:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T17:14:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:14:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T17:14:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:14:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:14:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:14:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T17:14:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-09-13T17:14:12.064 INFO:teuthology.orchestra.run.smithi100.stdout:Install 130 Packages 2024-09-13T17:14:12.065 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 8 Packages 2024-09-13T17:14:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:14:12.065 INFO:teuthology.orchestra.run.smithi070.stdout:(7/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-13T17:14:12.065 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 203 M 2024-09-13T17:14:12.066 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-13T17:14:12.079 INFO:teuthology.orchestra.run.smithi044.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 1.2 MB/s | 852 kB 00:00 2024-09-13T17:14:12.180 INFO:teuthology.orchestra.run.smithi044.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 1.4 MB/s | 142 kB 00:00 2024-09-13T17:14:12.255 INFO:teuthology.orchestra.run.smithi070.stdout:(8/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 17 MB/s | 18 MB 00:01 2024-09-13T17:14:12.305 INFO:teuthology.orchestra.run.smithi070.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 507 kB/s | 25 kB 00:00 2024-09-13T17:14:12.500 INFO:teuthology.orchestra.run.smithi070.stdout:(10/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9 18 MB/s | 7.7 MB 00:00 2024-09-13T17:14:12.575 INFO:teuthology.orchestra.run.smithi070.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 2.9 MB/s | 224 kB 00:00 2024-09-13T17:14:12.633 INFO:teuthology.orchestra.run.smithi070.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 534 kB/s | 31 kB 00:00 2024-09-13T17:14:12.725 INFO:teuthology.orchestra.run.smithi070.stdout:(13/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 7.6 MB/s | 712 kB 00:00 2024-09-13T17:14:12.730 INFO:teuthology.orchestra.run.smithi044.stdout:(4/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 3.8 MB/s | 2.1 MB 00:00 2024-09-13T17:14:12.930 INFO:teuthology.orchestra.run.smithi044.stdout:(5/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 7.3 MB/s | 1.5 MB 00:00 2024-09-13T17:14:12.934 INFO:teuthology.orchestra.run.smithi070.stdout:(14/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 16 MB/s | 17 MB 00:01 2024-09-13T17:14:12.959 INFO:teuthology.orchestra.run.smithi070.stdout:(15/138): libcephsqlite-18.2.4-860.g6c6b083f.el 711 kB/s | 166 kB 00:00 2024-09-13T17:14:12.985 INFO:teuthology.orchestra.run.smithi070.stdout:(16/138): librados-devel-18.2.4-860.g6c6b083f.e 2.5 MB/s | 127 kB 00:00 2024-09-13T17:14:13.018 INFO:teuthology.orchestra.run.smithi070.stdout:(17/138): libradosstriper1-18.2.4-860.g6c6b083f 8.0 MB/s | 478 kB 00:00 2024-09-13T17:14:13.085 INFO:teuthology.orchestra.run.smithi070.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 674 kB/s | 45 kB 00:00 2024-09-13T17:14:13.194 INFO:teuthology.orchestra.run.smithi070.stdout:(19/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 21 MB/s | 4.5 MB 00:00 2024-09-13T17:14:13.219 INFO:teuthology.orchestra.run.smithi070.stdout:(20/138): python3-ceph-common-18.2.4-860.g6c6b0 968 kB/s | 129 kB 00:00 2024-09-13T17:14:13.253 INFO:teuthology.orchestra.run.smithi070.stdout:(21/138): python3-cephfs-18.2.4-860.g6c6b083f.e 2.7 MB/s | 162 kB 00:00 2024-09-13T17:14:13.289 INFO:teuthology.orchestra.run.smithi044.stdout:(6/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 2.5 MB/s | 5.1 MB 00:02 2024-09-13T17:14:13.294 INFO:teuthology.orchestra.run.smithi070.stdout:(22/138): python3-rados-18.2.4-860.g6c6b083f.el 4.2 MB/s | 321 kB 00:00 2024-09-13T17:14:13.320 INFO:teuthology.orchestra.run.smithi070.stdout:(23/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 4.4 MB/s | 301 kB 00:00 2024-09-13T17:14:13.354 INFO:teuthology.orchestra.run.smithi070.stdout:(24/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 1.7 MB/s | 99 kB 00:00 2024-09-13T17:14:13.379 INFO:teuthology.orchestra.run.smithi070.stdout:(25/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.4 MB/s | 87 kB 00:00 2024-09-13T17:14:13.400 INFO:teuthology.orchestra.run.smithi100.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 17 kB/s | 6.5 kB 00:00 2024-09-13T17:14:13.414 INFO:teuthology.orchestra.run.smithi044.stdout:(7/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 9.7 MB/s | 4.7 MB 00:00 2024-09-13T17:14:13.471 INFO:teuthology.orchestra.run.smithi070.stdout:(26/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 1.8 MB/s | 172 kB 00:00 2024-09-13T17:14:13.529 INFO:teuthology.orchestra.run.smithi070.stdout:(27/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 17 MB/s | 3.0 MB 00:00 2024-09-13T17:14:13.555 INFO:teuthology.orchestra.run.smithi070.stdout:(28/138): ceph-grafana-dashboards-18.2.4-860.g6 293 kB/s | 24 kB 00:00 2024-09-13T17:14:13.597 INFO:teuthology.orchestra.run.smithi070.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 2.0 MB/s | 139 kB 00:00 2024-09-13T17:14:13.847 INFO:teuthology.orchestra.run.smithi070.stdout:(30/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 12 MB/s | 3.5 MB 00:00 2024-09-13T17:14:13.875 INFO:teuthology.orchestra.run.smithi100.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 1.8 MB/s | 852 kB 00:00 2024-09-13T17:14:13.923 INFO:teuthology.orchestra.run.smithi070.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 3.2 MB/s | 246 kB 00:00 2024-09-13T17:14:14.006 INFO:teuthology.orchestra.run.smithi070.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 593 kB/s | 49 kB 00:00 2024-09-13T17:14:14.025 INFO:teuthology.orchestra.run.smithi100.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 951 kB/s | 142 kB 00:00 2024-09-13T17:14:14.115 INFO:teuthology.orchestra.run.smithi070.stdout:(33/138): ceph-mgr-diskprediction-local-18.2.4- 14 MB/s | 7.4 MB 00:00 2024-09-13T17:14:14.140 INFO:teuthology.orchestra.run.smithi070.stdout:(34/138): ceph-prometheus-alerts-18.2.4-860.g6c 123 kB/s | 16 kB 00:00 2024-09-13T17:14:14.174 INFO:teuthology.orchestra.run.smithi070.stdout:(35/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 4.4 MB/s | 264 kB 00:00 2024-09-13T17:14:14.189 INFO:teuthology.orchestra.run.smithi044.stdout:(8/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9. 10 MB/s | 7.7 MB 00:00 2024-09-13T17:14:14.207 INFO:teuthology.orchestra.run.smithi070.stdout:(36/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 3.3 MB/s | 226 kB 00:00 2024-09-13T17:14:14.233 INFO:teuthology.orchestra.run.smithi070.stdout:(37/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 311 kB/s | 18 kB 00:00 2024-09-13T17:14:14.239 INFO:teuthology.orchestra.run.smithi044.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 507 kB/s | 25 kB 00:00 2024-09-13T17:14:14.492 INFO:teuthology.orchestra.run.smithi100.stdout:(4/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 4.5 MB/s | 2.1 MB 00:00 2024-09-13T17:14:14.759 INFO:teuthology.orchestra.run.smithi100.stdout:(5/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 5.5 MB/s | 1.5 MB 00:00 2024-09-13T17:14:14.767 INFO:teuthology.orchestra.run.smithi070.stdout:(38/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 16 MB/s | 40 MB 00:02 2024-09-13T17:14:15.068 INFO:teuthology.orchestra.run.smithi100.stdout:(6/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 2.5 MB/s | 5.1 MB 00:02 2024-09-13T17:14:15.233 INFO:teuthology.orchestra.run.smithi070.stdout:(39/138): fuse-2.9.9-16.el9.x86_64.rpm 78 kB/s | 80 kB 00:01 2024-09-13T17:14:15.368 INFO:teuthology.orchestra.run.smithi100.stdout:(7/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 7.8 MB/s | 18 MB 00:02 2024-09-13T17:14:15.492 INFO:teuthology.orchestra.run.smithi070.stdout:(40/138): libgfortran-11.5.0-2.el9.x86_64.rpm 636 kB/s | 801 kB 00:01 2024-09-13T17:14:15.526 INFO:teuthology.orchestra.run.smithi070.stdout:(41/138): libquadmath-11.5.0-2.el9.x86_64.rpm 257 kB/s | 191 kB 00:00 2024-09-13T17:14:15.735 INFO:teuthology.orchestra.run.smithi100.stdout:(8/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9. 21 MB/s | 7.7 MB 00:00 2024-09-13T17:14:15.785 INFO:teuthology.orchestra.run.smithi100.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 506 kB/s | 25 kB 00:00 2024-09-13T17:14:15.877 INFO:teuthology.orchestra.run.smithi100.stdout:(10/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86 4.2 MB/s | 4.7 MB 00:01 2024-09-13T17:14:15.969 INFO:teuthology.orchestra.run.smithi100.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 2.4 MB/s | 224 kB 00:00 2024-09-13T17:14:16.001 INFO:teuthology.orchestra.run.smithi070.stdout:(42/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 330 kB/s | 253 kB 00:00 2024-09-13T17:14:16.036 INFO:teuthology.orchestra.run.smithi100.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 466 kB/s | 31 kB 00:00 2024-09-13T17:14:16.039 INFO:teuthology.orchestra.run.smithi044.stdout:(10/138): ceph-common-18.2.4-860.g6c6b083f.el9. 3.9 MB/s | 18 MB 00:04 2024-09-13T17:14:16.173 INFO:teuthology.orchestra.run.smithi044.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 1.6 MB/s | 224 kB 00:00 2024-09-13T17:14:16.193 INFO:teuthology.orchestra.run.smithi070.stdout:(43/138): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-13T17:14:16.257 INFO:teuthology.orchestra.run.smithi044.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 374 kB/s | 31 kB 00:00 2024-09-13T17:14:16.328 INFO:teuthology.orchestra.run.smithi100.stdout:(13/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 2.4 MB/s | 712 kB 00:00 2024-09-13T17:14:16.411 INFO:teuthology.orchestra.run.smithi100.stdout:(14/138): libcephsqlite-18.2.4-860.g6c6b083f.el 2.0 MB/s | 166 kB 00:00 2024-09-13T17:14:16.426 INFO:teuthology.orchestra.run.smithi070.stdout:(44/138): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-13T17:14:16.512 INFO:teuthology.orchestra.run.smithi100.stdout:(15/138): librados-devel-18.2.4-860.g6c6b083f.e 1.2 MB/s | 127 kB 00:00 2024-09-13T17:14:16.687 INFO:teuthology.orchestra.run.smithi100.stdout:(16/138): libradosstriper1-18.2.4-860.g6c6b083f 2.7 MB/s | 478 kB 00:00 2024-09-13T17:14:16.782 INFO:teuthology.orchestra.run.smithi044.stdout:(13/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 16 MB/s | 40 MB 00:02 2024-09-13T17:14:16.824 INFO:teuthology.orchestra.run.smithi044.stdout:(14/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 1.2 MB/s | 712 kB 00:00 2024-09-13T17:14:16.827 INFO:teuthology.orchestra.run.smithi070.stdout:(45/138): python3-pycparser-2.20-6.el9.noarch.r 164 kB/s | 135 kB 00:00 2024-09-13T17:14:16.849 INFO:teuthology.orchestra.run.smithi044.stdout:(15/138): libcephsqlite-18.2.4-860.g6c6b083f.el 2.4 MB/s | 166 kB 00:00 2024-09-13T17:14:16.874 INFO:teuthology.orchestra.run.smithi044.stdout:(16/138): librados-devel-18.2.4-860.g6c6b083f.e 2.5 MB/s | 127 kB 00:00 2024-09-13T17:14:16.925 INFO:teuthology.orchestra.run.smithi044.stdout:(17/138): libradosstriper1-18.2.4-860.g6c6b083f 6.2 MB/s | 478 kB 00:00 2024-09-13T17:14:16.926 INFO:teuthology.orchestra.run.smithi070.stdout:(46/138): python3-requests-2.25.1-8.el9.noarch. 170 kB/s | 125 kB 00:00 2024-09-13T17:14:16.967 INFO:teuthology.orchestra.run.smithi044.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 1.1 MB/s | 45 kB 00:00 2024-09-13T17:14:17.017 INFO:teuthology.orchestra.run.smithi044.stdout:(19/138): python3-ceph-common-18.2.4-860.g6c6b0 2.5 MB/s | 129 kB 00:00 2024-09-13T17:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout:(20/138): python3-cephfs-18.2.4-860.g6c6b083f.e 3.2 MB/s | 162 kB 00:00 2024-09-13T17:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout:(21/138): python3-rados-18.2.4-860.g6c6b083f.el 6.3 MB/s | 321 kB 00:00 2024-09-13T17:14:17.121 INFO:teuthology.orchestra.run.smithi100.stdout:(17/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 30 MB/s | 40 MB 00:01 2024-09-13T17:14:17.168 INFO:teuthology.orchestra.run.smithi044.stdout:(22/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 5.9 MB/s | 301 kB 00:00 2024-09-13T17:14:17.171 INFO:teuthology.orchestra.run.smithi100.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 908 kB/s | 45 kB 00:00 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi044.stdout:(23/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 2.3 MB/s | 99 kB 00:00 2024-09-13T17:14:17.221 INFO:teuthology.orchestra.run.smithi100.stdout:(19/138): python3-ceph-common-18.2.4-860.g6c6b0 2.5 MB/s | 129 kB 00:00 2024-09-13T17:14:17.260 INFO:teuthology.orchestra.run.smithi044.stdout:(24/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.7 MB/s | 87 kB 00:00 2024-09-13T17:14:17.271 INFO:teuthology.orchestra.run.smithi100.stdout:(20/138): python3-cephfs-18.2.4-860.g6c6b083f.e 3.2 MB/s | 162 kB 00:00 2024-09-13T17:14:17.322 INFO:teuthology.orchestra.run.smithi100.stdout:(21/138): python3-rados-18.2.4-860.g6c6b083f.el 6.3 MB/s | 321 kB 00:00 2024-09-13T17:14:17.369 INFO:teuthology.orchestra.run.smithi070.stdout:(47/138): python3-urllib3-1.26.5-6.el9.noarch.r 230 kB/s | 215 kB 00:00 2024-09-13T17:14:17.381 INFO:teuthology.orchestra.run.smithi100.stdout:(22/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 5.0 MB/s | 301 kB 00:00 2024-09-13T17:14:17.422 INFO:teuthology.orchestra.run.smithi100.stdout:(23/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 2.3 MB/s | 99 kB 00:00 2024-09-13T17:14:17.444 INFO:teuthology.orchestra.run.smithi044.stdout:(25/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 16 MB/s | 3.0 MB 00:00 2024-09-13T17:14:17.473 INFO:teuthology.orchestra.run.smithi100.stdout:(24/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.7 MB/s | 87 kB 00:00 2024-09-13T17:14:17.494 INFO:teuthology.orchestra.run.smithi044.stdout:(26/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 3.4 MB/s | 172 kB 00:00 2024-09-13T17:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout:(27/138): ceph-grafana-dashboards-18.2.4-860.g6 420 kB/s | 24 kB 00:00 2024-09-13T17:14:17.606 INFO:teuthology.orchestra.run.smithi100.stdout:(25/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 22 MB/s | 3.0 MB 00:00 2024-09-13T17:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout:(28/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 5.8 MB/s | 4.5 MB 00:00 2024-09-13T17:14:17.669 INFO:teuthology.orchestra.run.smithi070.stdout:(48/138): automake-1.16.2-8.el9.noarch.rpm 819 kB/s | 689 kB 00:00 2024-09-13T17:14:17.670 INFO:teuthology.orchestra.run.smithi044.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 1.2 MB/s | 139 kB 00:00 2024-09-13T17:14:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:(26/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 4.5 MB/s | 4.5 MB 00:00 2024-09-13T17:14:17.707 INFO:teuthology.orchestra.run.smithi100.stdout:(27/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 1.7 MB/s | 172 kB 00:00 2024-09-13T17:14:17.732 INFO:teuthology.orchestra.run.smithi100.stdout:(28/138): ceph-grafana-dashboards-18.2.4-860.g6 487 kB/s | 24 kB 00:00 2024-09-13T17:14:17.744 INFO:teuthology.orchestra.run.smithi070.stdout:(49/138): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-13T17:14:17.757 INFO:teuthology.orchestra.run.smithi100.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 2.7 MB/s | 139 kB 00:00 2024-09-13T17:14:17.778 INFO:teuthology.orchestra.run.smithi070.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-13T17:14:17.828 INFO:teuthology.orchestra.run.smithi070.stdout:(51/138): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-13T17:14:17.870 INFO:teuthology.orchestra.run.smithi070.stdout:(52/138): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-13T17:14:17.895 INFO:teuthology.orchestra.run.smithi070.stdout:(53/138): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-13T17:14:17.895 INFO:teuthology.orchestra.run.smithi044.stdout:(30/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 3.6 MB/s | 17 MB 00:04 2024-09-13T17:14:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-13T17:14:18.004 INFO:teuthology.orchestra.run.smithi070.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-13T17:14:18.012 INFO:teuthology.orchestra.run.smithi044.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 2.1 MB/s | 246 kB 00:00 2024-09-13T17:14:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:(30/138): ceph-mgr-diskprediction-local-18.2.4- 27 MB/s | 7.4 MB 00:00 2024-09-13T17:14:18.062 INFO:teuthology.orchestra.run.smithi044.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 990 kB/s | 49 kB 00:00 2024-09-13T17:14:18.091 INFO:teuthology.orchestra.run.smithi100.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 4.1 MB/s | 246 kB 00:00 2024-09-13T17:14:18.096 INFO:teuthology.orchestra.run.smithi070.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-13T17:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout:(33/138): ceph-prometheus-alerts-18.2.4-860.g6c 330 kB/s | 16 kB 00:00 2024-09-13T17:14:18.121 INFO:teuthology.orchestra.run.smithi070.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-13T17:14:18.141 INFO:teuthology.orchestra.run.smithi100.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 991 kB/s | 49 kB 00:00 2024-09-13T17:14:18.192 INFO:teuthology.orchestra.run.smithi100.stdout:(33/138): ceph-prometheus-alerts-18.2.4-860.g6c 328 kB/s | 16 kB 00:00 2024-09-13T17:14:18.221 INFO:teuthology.orchestra.run.smithi070.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-13T17:14:18.229 INFO:teuthology.orchestra.run.smithi044.stdout:(34/138): ceph-mgr-diskprediction-local-18.2.4- 13 MB/s | 7.4 MB 00:00 2024-09-13T17:14:18.242 INFO:teuthology.orchestra.run.smithi100.stdout:(34/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 5.2 MB/s | 264 kB 00:00 2024-09-13T17:14:18.255 INFO:teuthology.orchestra.run.smithi044.stdout:(35/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 1.8 MB/s | 264 kB 00:00 2024-09-13T17:14:18.280 INFO:teuthology.orchestra.run.smithi044.stdout:(36/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 4.4 MB/s | 226 kB 00:00 2024-09-13T17:14:18.292 INFO:teuthology.orchestra.run.smithi100.stdout:(35/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 4.4 MB/s | 226 kB 00:00 2024-09-13T17:14:18.305 INFO:teuthology.orchestra.run.smithi044.stdout:(37/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 363 kB/s | 18 kB 00:00 2024-09-13T17:14:18.338 INFO:teuthology.orchestra.run.smithi070.stdout:(59/138): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-13T17:14:18.364 INFO:teuthology.orchestra.run.smithi070.stdout:(60/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 173 kB/s | 24 kB 00:00 2024-09-13T17:14:18.368 INFO:teuthology.orchestra.run.smithi100.stdout:(36/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 5.6 MB/s | 3.5 MB 00:00 2024-09-13T17:14:18.389 INFO:teuthology.orchestra.run.smithi044.stdout:(38/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 4.7 MB/s | 3.5 MB 00:00 2024-09-13T17:14:18.393 INFO:teuthology.orchestra.run.smithi100.stdout:(37/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 182 kB/s | 18 kB 00:00 2024-09-13T17:14:18.447 INFO:teuthology.orchestra.run.smithi070.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-13T17:14:18.497 INFO:teuthology.orchestra.run.smithi070.stdout:(62/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-13T17:14:18.498 INFO:teuthology.orchestra.run.smithi044.stdout:(39/138): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-13T17:14:18.535 INFO:teuthology.orchestra.run.smithi100.stdout:(38/138): fuse-2.9.9-16.el9.x86_64.rpm 564 kB/s | 80 kB 00:00 2024-09-13T17:14:18.565 INFO:teuthology.orchestra.run.smithi070.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-13T17:14:18.585 INFO:teuthology.orchestra.run.smithi100.stdout:(39/138): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-13T17:14:18.589 INFO:teuthology.orchestra.run.smithi044.stdout:(40/138): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-13T17:14:18.615 INFO:teuthology.orchestra.run.smithi070.stdout:(64/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-13T17:14:18.615 INFO:teuthology.orchestra.run.smithi044.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-13T17:14:18.627 INFO:teuthology.orchestra.run.smithi100.stdout:(40/138): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-13T17:14:18.648 INFO:teuthology.orchestra.run.smithi044.stdout:(42/138): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-13T17:14:18.652 INFO:teuthology.orchestra.run.smithi100.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-13T17:14:18.673 INFO:teuthology.orchestra.run.smithi044.stdout:(43/138): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-13T17:14:18.707 INFO:teuthology.orchestra.run.smithi044.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-13T17:14:18.732 INFO:teuthology.orchestra.run.smithi044.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-13T17:14:18.753 INFO:teuthology.orchestra.run.smithi100.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-13T17:14:18.766 INFO:teuthology.orchestra.run.smithi044.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-13T17:14:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:(43/138): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-09-13T17:14:18.799 INFO:teuthology.orchestra.run.smithi044.stdout:(47/138): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-13T17:14:18.837 INFO:teuthology.orchestra.run.smithi100.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-13T17:14:18.849 INFO:teuthology.orchestra.run.smithi070.stdout:(65/138): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-13T17:14:18.862 INFO:teuthology.orchestra.run.smithi100.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-13T17:14:18.913 INFO:teuthology.orchestra.run.smithi100.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-13T17:14:18.916 INFO:teuthology.orchestra.run.smithi044.stdout:(48/138): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-13T17:14:18.942 INFO:teuthology.orchestra.run.smithi044.stdout:(49/138): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-09-13T17:14:18.967 INFO:teuthology.orchestra.run.smithi044.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-13T17:14:18.982 INFO:teuthology.orchestra.run.smithi070.stdout:(66/138): postgresql-contrib-13.16-1.el9.x86_64 6.7 MB/s | 910 kB 00:00 2024-09-13T17:14:19.009 INFO:teuthology.orchestra.run.smithi044.stdout:(51/138): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-13T17:14:19.034 INFO:teuthology.orchestra.run.smithi044.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-13T17:14:19.059 INFO:teuthology.orchestra.run.smithi044.stdout:(53/138): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-13T17:14:19.085 INFO:teuthology.orchestra.run.smithi044.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-13T17:14:19.099 INFO:teuthology.orchestra.run.smithi070.stdout:(67/138): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-13T17:14:19.110 INFO:teuthology.orchestra.run.smithi044.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-13T17:14:19.135 INFO:teuthology.orchestra.run.smithi044.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-13T17:14:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:(47/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 4.1 MB/s | 17 MB 00:04 2024-09-13T17:14:19.152 INFO:teuthology.orchestra.run.smithi044.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-13T17:14:19.196 INFO:teuthology.orchestra.run.smithi044.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-13T17:14:19.244 INFO:teuthology.orchestra.run.smithi044.stdout:(59/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-13T17:14:19.261 INFO:teuthology.orchestra.run.smithi044.stdout:(60/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-13T17:14:19.295 INFO:teuthology.orchestra.run.smithi044.stdout:(61/138): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-13T17:14:19.320 INFO:teuthology.orchestra.run.smithi044.stdout:(62/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-13T17:14:19.338 INFO:teuthology.orchestra.run.smithi044.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 922 kB/s | 38 kB 00:00 2024-09-13T17:14:19.363 INFO:teuthology.orchestra.run.smithi044.stdout:(64/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-13T17:14:19.446 INFO:teuthology.orchestra.run.smithi044.stdout:(65/138): openblas-openmp-0.3.26-2.el9.x86_64.r 39 MB/s | 4.9 MB 00:00 2024-09-13T17:14:19.488 INFO:teuthology.orchestra.run.smithi044.stdout:(66/138): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-13T17:14:19.613 INFO:teuthology.orchestra.run.smithi044.stdout:(67/138): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-13T17:14:19.639 INFO:teuthology.orchestra.run.smithi044.stdout:(68/138): postgresql-private-libs-13.16-1.el9.x 915 kB/s | 137 kB 00:00 2024-09-13T17:14:19.722 INFO:teuthology.orchestra.run.smithi044.stdout:(69/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T17:14:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:(68/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-13T17:14:19.764 INFO:teuthology.orchestra.run.smithi044.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-09-13T17:14:19.821 INFO:teuthology.orchestra.run.smithi100.stdout:(48/138): boost-program-options-1.75.0-8.el9.x8 158 kB/s | 107 kB 00:00 2024-09-13T17:14:19.839 INFO:teuthology.orchestra.run.smithi044.stdout:(71/138): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-13T17:14:19.915 INFO:teuthology.orchestra.run.smithi044.stdout:(72/138): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-13T17:14:19.940 INFO:teuthology.orchestra.run.smithi044.stdout:(73/138): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-13T17:14:19.965 INFO:teuthology.orchestra.run.smithi044.stdout:(74/138): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-09-13T17:14:19.990 INFO:teuthology.orchestra.run.smithi044.stdout:(75/138): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-13T17:14:20.016 INFO:teuthology.orchestra.run.smithi044.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-13T17:14:20.041 INFO:teuthology.orchestra.run.smithi044.stdout:(77/138): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-13T17:14:20.066 INFO:teuthology.orchestra.run.smithi044.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-13T17:14:20.092 INFO:teuthology.orchestra.run.smithi044.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-13T17:14:20.117 INFO:teuthology.orchestra.run.smithi044.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-13T17:14:20.142 INFO:teuthology.orchestra.run.smithi044.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-13T17:14:20.167 INFO:teuthology.orchestra.run.smithi044.stdout:(82/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-13T17:14:20.192 INFO:teuthology.orchestra.run.smithi044.stdout:(83/138): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-13T17:14:20.218 INFO:teuthology.orchestra.run.smithi044.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-13T17:14:20.243 INFO:teuthology.orchestra.run.smithi044.stdout:(85/138): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-13T17:14:20.285 INFO:teuthology.orchestra.run.smithi044.stdout:(86/138): python3-toml-0.10.2-6.el9.noarch.rpm 997 kB/s | 42 kB 00:00 2024-09-13T17:14:20.330 INFO:teuthology.orchestra.run.smithi100.stdout:(49/138): automake-1.16.2-8.el9.noarch.rpm 470 kB/s | 689 kB 00:01 2024-09-13T17:14:20.347 INFO:teuthology.orchestra.run.smithi100.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-13T17:14:20.352 INFO:teuthology.orchestra.run.smithi044.stdout:(87/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-13T17:14:20.377 INFO:teuthology.orchestra.run.smithi044.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-13T17:14:20.403 INFO:teuthology.orchestra.run.smithi044.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-13T17:14:20.428 INFO:teuthology.orchestra.run.smithi044.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-13T17:14:20.453 INFO:teuthology.orchestra.run.smithi044.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T17:14:20.487 INFO:teuthology.orchestra.run.smithi044.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 762 kB/s | 45 kB 00:00 2024-09-13T17:14:20.546 INFO:teuthology.orchestra.run.smithi044.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.2 MB/s | 111 kB 00:00 2024-09-13T17:14:20.588 INFO:teuthology.orchestra.run.smithi044.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 3.0 MB/s | 308 kB 00:00 2024-09-13T17:14:20.613 INFO:teuthology.orchestra.run.smithi044.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-13T17:14:20.638 INFO:teuthology.orchestra.run.smithi044.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T17:14:20.664 INFO:teuthology.orchestra.run.smithi044.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-13T17:14:20.739 INFO:teuthology.orchestra.run.smithi100.stdout:(51/138): bison-3.7.4-5.el9.x86_64.rpm 516 kB/s | 943 kB 00:01 2024-09-13T17:14:20.839 INFO:teuthology.orchestra.run.smithi044.stdout:(98/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-13T17:14:20.881 INFO:teuthology.orchestra.run.smithi044.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-09-13T17:14:20.906 INFO:teuthology.orchestra.run.smithi044.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.9 MB/s | 43 kB 00:00 2024-09-13T17:14:20.923 INFO:teuthology.orchestra.run.smithi044.stdout:(101/138): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-13T17:14:20.939 INFO:teuthology.orchestra.run.smithi100.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 55 kB/s | 33 kB 00:00 2024-09-13T17:14:20.939 INFO:teuthology.orchestra.run.smithi044.stdout:(102/138): python3-certifi-2023.05.07-4.el9.noa 854 kB/s | 14 kB 00:00 2024-09-13T17:14:20.973 INFO:teuthology.orchestra.run.smithi044.stdout:(103/138): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-13T17:14:20.999 INFO:teuthology.orchestra.run.smithi044.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-13T17:14:21.032 INFO:teuthology.orchestra.run.smithi044.stdout:(105/138): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-13T17:14:21.057 INFO:teuthology.orchestra.run.smithi044.stdout:(106/138): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-13T17:14:21.083 INFO:teuthology.orchestra.run.smithi044.stdout:(107/138): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-13T17:14:21.108 INFO:teuthology.orchestra.run.smithi044.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-13T17:14:21.133 INFO:teuthology.orchestra.run.smithi044.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-13T17:14:21.161 INFO:teuthology.orchestra.run.smithi044.stdout:(110/138): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-13T17:14:21.184 INFO:teuthology.orchestra.run.smithi044.stdout:(111/138): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-13T17:14:21.209 INFO:teuthology.orchestra.run.smithi044.stdout:(112/138): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-13T17:14:21.236 INFO:teuthology.orchestra.run.smithi044.stdout:(113/138): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-13T17:14:21.281 INFO:teuthology.orchestra.run.smithi100.stdout:(53/138): flex-2.6.4-9.el9.x86_64.rpm 330 kB/s | 313 kB 00:00 2024-09-13T17:14:21.335 INFO:teuthology.orchestra.run.smithi044.stdout:(114/138): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-09-13T17:14:21.360 INFO:teuthology.orchestra.run.smithi044.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-09-13T17:14:21.385 INFO:teuthology.orchestra.run.smithi044.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-13T17:14:21.410 INFO:teuthology.orchestra.run.smithi044.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T17:14:21.435 INFO:teuthology.orchestra.run.smithi044.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-13T17:14:21.456 INFO:teuthology.orchestra.run.smithi100.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-13T17:14:21.461 INFO:teuthology.orchestra.run.smithi044.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-13T17:14:21.486 INFO:teuthology.orchestra.run.smithi044.stdout:(120/138): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-13T17:14:21.520 INFO:teuthology.orchestra.run.smithi044.stdout:(121/138): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-09-13T17:14:21.533 INFO:teuthology.orchestra.run.smithi070.stdout:(69/138): openblas-openmp-0.3.26-2.el9.x86_64.r 1.6 MB/s | 4.9 MB 00:03 2024-09-13T17:14:21.545 INFO:teuthology.orchestra.run.smithi044.stdout:(122/138): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-09-13T17:14:21.570 INFO:teuthology.orchestra.run.smithi044.stdout:(123/138): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-09-13T17:14:21.596 INFO:teuthology.orchestra.run.smithi044.stdout:(124/138): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-13T17:14:21.621 INFO:teuthology.orchestra.run.smithi044.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-13T17:14:21.646 INFO:teuthology.orchestra.run.smithi044.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-13T17:14:21.672 INFO:teuthology.orchestra.run.smithi044.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-13T17:14:21.697 INFO:teuthology.orchestra.run.smithi044.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T17:14:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-13T17:14:21.789 INFO:teuthology.orchestra.run.smithi100.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-13T17:14:21.814 INFO:teuthology.orchestra.run.smithi044.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T17:14:21.908 INFO:teuthology.orchestra.run.smithi070.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-13T17:14:21.981 INFO:teuthology.orchestra.run.smithi044.stdout:(131/138): librados2-18.2.4-860.g6c6b083f.el9.x 12 MB/s | 3.3 MB 00:00 2024-09-13T17:14:22.173 INFO:teuthology.orchestra.run.smithi100.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 224 kB/s | 160 kB 00:00 2024-09-13T17:14:22.198 INFO:teuthology.orchestra.run.smithi044.stdout:(132/138): NetworkManager-1.49.90-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-09-13T17:14:22.265 INFO:teuthology.orchestra.run.smithi044.stdout:(133/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 5.6 MB/s | 3.0 MB 00:00 2024-09-13T17:14:22.290 INFO:teuthology.orchestra.run.smithi044.stdout:(134/138): NetworkManager-team-1.49.90-1.el9.x8 445 kB/s | 41 kB 00:00 2024-09-13T17:14:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:(71/138): python3-jinja2-2.11.3-6.el9.noarch.rp 634 kB/s | 248 kB 00:00 2024-09-13T17:14:22.381 INFO:teuthology.orchestra.run.smithi100.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-09-13T17:14:22.382 INFO:teuthology.orchestra.run.smithi044.stdout:(135/138): NetworkManager-libnm-1.49.90-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-09-13T17:14:22.400 INFO:teuthology.orchestra.run.smithi070.stdout:(72/138): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-09-13T17:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout:(136/138): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-13T17:14:22.465 INFO:teuthology.orchestra.run.smithi100.stdout:(58/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.7 MB/s | 3.0 MB 00:01 2024-09-13T17:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout:(137/138): NetworkManager-tui-1.49.90-1.el9.x86 1.1 MB/s | 249 kB 00:00 2024-09-13T17:14:22.500 INFO:teuthology.orchestra.run.smithi070.stdout:(73/138): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-13T17:14:22.632 INFO:teuthology.orchestra.run.smithi044.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-09-13T17:14:22.639 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:14:22.639 INFO:teuthology.orchestra.run.smithi044.stdout:Total 17 MB/s | 203 MB 00:11 2024-09-13T17:14:22.792 INFO:teuthology.orchestra.run.smithi070.stdout:(74/138): python3-mako-1.1.4-6.el9.noarch.rpm 590 kB/s | 172 kB 00:00 2024-09-13T17:14:22.909 INFO:teuthology.orchestra.run.smithi070.stdout:(75/138): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-09-13T17:14:22.993 INFO:teuthology.orchestra.run.smithi070.stdout:(76/138): postgresql-server-13.16-1.el9.x86_64. 1.5 MB/s | 5.8 MB 00:03 2024-09-13T17:14:23.024 INFO:teuthology.orchestra.run.smithi100.stdout:(59/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-13T17:14:23.368 INFO:teuthology.orchestra.run.smithi070.stdout:(77/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 668 kB/s | 250 kB 00:00 2024-09-13T17:14:23.432 INFO:teuthology.orchestra.run.smithi100.stdout:(60/138): librdkafka-1.6.1-102.el9.x86_64.rpm 526 kB/s | 662 kB 00:01 2024-09-13T17:14:23.466 INFO:teuthology.orchestra.run.smithi100.stdout:(61/138): libtool-2.4.6-46.el9.x86_64.rpm 536 kB/s | 581 kB 00:01 2024-09-13T17:14:23.666 INFO:teuthology.orchestra.run.smithi100.stdout:(62/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-09-13T17:14:23.760 INFO:teuthology.orchestra.run.smithi070.stdout:(78/138): python3-oauthlib-3.1.1-5.el9.noarch.r 567 kB/s | 222 kB 00:00 2024-09-13T17:14:23.779 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T17:14:23.941 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T17:14:23.941 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T17:14:23.968 INFO:teuthology.orchestra.run.smithi070.stdout:(79/138): python3-packaging-20.9-5.el9.noarch.r 370 kB/s | 77 kB 00:00 2024-09-13T17:14:24.016 INFO:teuthology.orchestra.run.smithi100.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 70 kB/s | 38 kB 00:00 2024-09-13T17:14:24.094 INFO:teuthology.orchestra.run.smithi070.stdout:(80/138): python3-prettytable-0.7.2-27.el9.noar 338 kB/s | 42 kB 00:00 2024-09-13T17:14:24.392 INFO:teuthology.orchestra.run.smithi100.stdout:(64/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 305 kB/s | 292 kB 00:00 2024-09-13T17:14:24.394 INFO:teuthology.orchestra.run.smithi070.stdout:(81/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 529 kB/s | 159 kB 00:00 2024-09-13T17:14:24.877 INFO:teuthology.orchestra.run.smithi070.stdout:(82/138): python3-pyasn1-modules-0.4.8-6.el9.no 577 kB/s | 279 kB 00:00 2024-09-13T17:14:25.011 INFO:teuthology.orchestra.run.smithi070.stdout:(83/138): python3-pytz-2021.1-5.el9.noarch.rpm 382 kB/s | 51 kB 00:00 2024-09-13T17:14:25.195 INFO:teuthology.orchestra.run.smithi070.stdout:(84/138): python3-requests-oauthlib-1.3.0-12.el 293 kB/s | 54 kB 00:00 2024-09-13T17:14:25.475 INFO:teuthology.orchestra.run.smithi100.stdout:(65/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-09-13T17:14:26.104 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T17:14:26.105 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T17:14:26.834 INFO:teuthology.orchestra.run.smithi100.stdout:(66/138): postgresql-13.16-1.el9.x86_64.rpm 665 kB/s | 1.6 MB 00:02 2024-09-13T17:14:27.117 INFO:teuthology.orchestra.run.smithi100.stdout:(67/138): postgresql-contrib-13.16-1.el9.x86_64 554 kB/s | 910 kB 00:01 2024-09-13T17:14:27.567 INFO:teuthology.orchestra.run.smithi100.stdout:(68/138): openblas-openmp-0.3.26-2.el9.x86_64.r 1.3 MB/s | 4.9 MB 00:03 2024-09-13T17:14:27.593 INFO:teuthology.orchestra.run.smithi100.stdout:(69/138): postgresql-private-libs-13.16-1.el9.x 181 kB/s | 137 kB 00:00 2024-09-13T17:14:28.526 INFO:teuthology.orchestra.run.smithi100.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 262 kB/s | 245 kB 00:00 2024-09-13T17:14:28.845 INFO:teuthology.orchestra.run.smithi070.stdout:(85/138): python3-babel-2.9.1-2.el9.noarch.rpm 671 kB/s | 6.0 MB 00:09 2024-09-13T17:14:29.003 INFO:teuthology.orchestra.run.smithi070.stdout:(86/138): python3-toml-0.10.2-6.el9.noarch.rpm 263 kB/s | 42 kB 00:00 2024-09-13T17:14:29.170 INFO:teuthology.orchestra.run.smithi070.stdout:(87/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 834 kB/s | 5.1 MB 00:06 2024-09-13T17:14:29.334 INFO:teuthology.orchestra.run.smithi100.stdout:(71/138): python3-jinja2-2.11.3-6.el9.noarch.rp 307 kB/s | 248 kB 00:00 2024-09-13T17:14:29.420 INFO:teuthology.orchestra.run.smithi070.stdout:(88/138): uuid-1.6.2-55.el9.x86_64.rpm 231 kB/s | 58 kB 00:00 2024-09-13T17:14:29.537 INFO:teuthology.orchestra.run.smithi070.stdout:(89/138): socat-1.7.4.1-6.el9.x86_64.rpm 570 kB/s | 304 kB 00:00 2024-09-13T17:14:29.649 INFO:teuthology.orchestra.run.smithi070.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 425 kB/s | 96 kB 00:00 2024-09-13T17:14:29.738 INFO:teuthology.orchestra.run.smithi070.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-13T17:14:29.763 INFO:teuthology.orchestra.run.smithi070.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 383 kB/s | 45 kB 00:00 2024-09-13T17:14:29.785 INFO:teuthology.orchestra.run.smithi100.stdout:(72/138): postgresql-server-13.16-1.el9.x86_64. 2.2 MB/s | 5.8 MB 00:02 2024-09-13T17:14:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.4 MB/s | 111 kB 00:00 2024-09-13T17:14:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 3.0 MB/s | 308 kB 00:00 2024-09-13T17:14:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-13T17:14:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T17:14:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-13T17:14:29.960 INFO:teuthology.orchestra.run.smithi100.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 26 kB 00:00 2024-09-13T17:14:30.031 INFO:teuthology.orchestra.run.smithi070.stdout:(98/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-13T17:14:30.090 INFO:teuthology.orchestra.run.smithi070.stdout:(99/138): python3-asyncssh-2.13.2-5.el9.noarch. 9.2 MB/s | 548 kB 00:00 2024-09-13T17:14:30.107 INFO:teuthology.orchestra.run.smithi070.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-13T17:14:30.124 INFO:teuthology.orchestra.run.smithi070.stdout:(101/138): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-13T17:14:30.141 INFO:teuthology.orchestra.run.smithi070.stdout:(102/138): python3-certifi-2023.05.07-4.el9.noa 849 kB/s | 14 kB 00:00 2024-09-13T17:14:30.216 INFO:teuthology.orchestra.run.smithi070.stdout:(103/138): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-13T17:14:30.242 INFO:teuthology.orchestra.run.smithi070.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-09-13T17:14:30.252 INFO:teuthology.orchestra.run.smithi100.stdout:(74/138): python3-babel-2.9.1-2.el9.noarch.rpm 2.2 MB/s | 6.0 MB 00:02 2024-09-13T17:14:30.267 INFO:teuthology.orchestra.run.smithi070.stdout:(105/138): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-13T17:14:30.285 INFO:teuthology.orchestra.run.smithi100.stdout:(75/138): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-13T17:14:30.292 INFO:teuthology.orchestra.run.smithi070.stdout:(106/138): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-13T17:14:30.317 INFO:teuthology.orchestra.run.smithi070.stdout:(107/138): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-13T17:14:30.343 INFO:teuthology.orchestra.run.smithi070.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-09-13T17:14:30.368 INFO:teuthology.orchestra.run.smithi070.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-13T17:14:30.393 INFO:teuthology.orchestra.run.smithi070.stdout:(110/138): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-13T17:14:30.419 INFO:teuthology.orchestra.run.smithi070.stdout:(111/138): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-13T17:14:30.444 INFO:teuthology.orchestra.run.smithi070.stdout:(112/138): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-13T17:14:30.473 INFO:teuthology.orchestra.run.smithi070.stdout:(113/138): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-13T17:14:30.494 INFO:teuthology.orchestra.run.smithi070.stdout:(114/138): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-13T17:14:30.536 INFO:teuthology.orchestra.run.smithi070.stdout:(115/138): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-13T17:14:30.562 INFO:teuthology.orchestra.run.smithi070.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-13T17:14:30.595 INFO:teuthology.orchestra.run.smithi070.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-13T17:14:30.620 INFO:teuthology.orchestra.run.smithi070.stdout:(118/138): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-13T17:14:30.646 INFO:teuthology.orchestra.run.smithi070.stdout:(119/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-13T17:14:30.671 INFO:teuthology.orchestra.run.smithi070.stdout:(120/138): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T17:14:30.697 INFO:teuthology.orchestra.run.smithi070.stdout:(121/138): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-13T17:14:30.722 INFO:teuthology.orchestra.run.smithi070.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-13T17:14:30.747 INFO:teuthology.orchestra.run.smithi070.stdout:(123/138): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-13T17:14:30.769 INFO:teuthology.orchestra.run.smithi100.stdout:(76/138): python3-mako-1.1.4-6.el9.noarch.rpm 213 kB/s | 172 kB 00:00 2024-09-13T17:14:30.781 INFO:teuthology.orchestra.run.smithi070.stdout:(124/138): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-13T17:14:30.828 INFO:teuthology.orchestra.run.smithi100.stdout:(77/138): python3-markupsafe-1.1.1-12.el9.x86_6 61 kB/s | 35 kB 00:00 2024-09-13T17:14:30.964 INFO:teuthology.orchestra.run.smithi070.stdout:(125/138): python3-webob-1.8.8-2.el9.noarch.rpm 952 kB/s | 230 kB 00:00 2024-09-13T17:14:31.040 INFO:teuthology.orchestra.run.smithi070.stdout:(126/138): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-09-13T17:14:31.065 INFO:teuthology.orchestra.run.smithi070.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-09-13T17:14:31.099 INFO:teuthology.orchestra.run.smithi070.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-13T17:14:31.207 INFO:teuthology.orchestra.run.smithi070.stdout:(129/138): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T17:14:31.374 INFO:teuthology.orchestra.run.smithi070.stdout:(130/138): librados2-18.2.4-860.g6c6b083f.el9.x 12 MB/s | 3.3 MB 00:00 2024-09-13T17:14:31.491 INFO:teuthology.orchestra.run.smithi070.stdout:(131/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 11 MB/s | 3.0 MB 00:00 2024-09-13T17:14:31.561 INFO:teuthology.orchestra.run.smithi100.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 317 kB/s | 250 kB 00:00 2024-09-13T17:14:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:(79/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-09-13T17:14:31.653 INFO:teuthology.orchestra.run.smithi100.stdout:(80/138): python3-oauthlib-3.1.1-5.el9.noarch.r 269 kB/s | 222 kB 00:00 2024-09-13T17:14:31.815 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T17:14:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T17:14:31.984 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T17:14:32.038 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T17:14:32.079 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T17:14:32.112 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T17:14:32.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T17:14:32.195 INFO:teuthology.orchestra.run.smithi100.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 75 kB/s | 42 kB 00:00 2024-09-13T17:14:32.221 INFO:teuthology.orchestra.run.smithi100.stdout:(82/138): python3-packaging-20.9-5.el9.noarch.r 117 kB/s | 77 kB 00:00 2024-09-13T17:14:32.425 INFO:teuthology.orchestra.run.smithi070.stdout:(132/138): NetworkManager-1.49.90-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-13T17:14:32.438 INFO:teuthology.orchestra.run.smithi100.stdout:(83/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 202 kB/s | 159 kB 00:00 2024-09-13T17:14:32.467 INFO:teuthology.orchestra.run.smithi070.stdout:(133/138): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-13T17:14:32.510 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T17:14:32.722 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:14:32.829 INFO:teuthology.orchestra.run.smithi100.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-13T17:14:32.963 INFO:teuthology.orchestra.run.smithi100.stdout:(85/138): python3-pyasn1-modules-0.4.8-6.el9.no 363 kB/s | 279 kB 00:00 2024-09-13T17:14:32.970 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:14:32.984 INFO:teuthology.orchestra.run.smithi070.stdout:(134/138): NetworkManager-team-1.49.90-1.el9.x8 73 kB/s | 41 kB 00:00 2024-09-13T17:14:32.998 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:14:33.105 INFO:teuthology.orchestra.run.smithi100.stdout:(86/138): python3-requests-oauthlib-1.3.0-12.el 81 kB/s | 54 kB 00:00 2024-09-13T17:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:14:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:(135/138): NetworkManager-tui-1.49.90-1.el9.x86 296 kB/s | 249 kB 00:00 2024-09-13T17:14:33.337 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T17:14:33.393 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T17:14:33.419 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:14:33.523 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:14:33.545 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:14:33.572 INFO:teuthology.orchestra.run.smithi100.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-09-13T17:14:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:14:33.881 INFO:teuthology.orchestra.run.smithi100.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 393 kB/s | 304 kB 00:00 2024-09-13T17:14:33.892 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T17:14:33.917 INFO:teuthology.orchestra.run.smithi070.stdout:(136/138): openssl-3.2.2-6.el9.x86_64.rpm 1.5 MB/s | 1.4 MB 00:00 2024-09-13T17:14:34.008 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:34.085 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:34.264 INFO:teuthology.orchestra.run.smithi100.stdout:(89/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-13T17:14:34.289 INFO:teuthology.orchestra.run.smithi100.stdout:(90/138): uuid-1.6.2-55.el9.x86_64.rpm 81 kB/s | 58 kB 00:00 2024-09-13T17:14:34.334 INFO:teuthology.orchestra.run.smithi070.stdout:(137/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-13T17:14:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T17:14:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T17:14:34.526 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T17:14:34.566 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T17:14:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T17:14:34.623 INFO:teuthology.orchestra.run.smithi100.stdout:(91/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 129 kB/s | 96 kB 00:00 2024-09-13T17:14:34.665 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T17:14:34.673 INFO:teuthology.orchestra.run.smithi100.stdout:(92/138): fmt-8.1.1-5.el9.x86_64.rpm 2.2 MB/s | 111 kB 00:00 2024-09-13T17:14:34.706 INFO:teuthology.orchestra.run.smithi100.stdout:(93/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 9.0 MB/s | 308 kB 00:00 2024-09-13T17:14:34.725 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T17:14:34.755 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T17:14:34.798 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T17:14:34.899 INFO:teuthology.orchestra.run.smithi100.stdout:(94/138): libarrow-9.0.0-11.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-13T17:14:34.913 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T17:14:34.924 INFO:teuthology.orchestra.run.smithi100.stdout:(95/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-09-13T17:14:34.949 INFO:teuthology.orchestra.run.smithi100.stdout:(96/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-09-13T17:14:34.964 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T17:14:34.974 INFO:teuthology.orchestra.run.smithi100.stdout:(97/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 358 kB/s | 26 kB 00:00 2024-09-13T17:14:35.022 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T17:14:35.025 INFO:teuthology.orchestra.run.smithi100.stdout:(98/138): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-09-13T17:14:35.066 INFO:teuthology.orchestra.run.smithi100.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-13T17:14:35.092 INFO:teuthology.orchestra.run.smithi100.stdout:(100/138): libunwind-1.6.2-1.el9.x86_64.rpm 474 kB/s | 67 kB 00:00 2024-09-13T17:14:35.117 INFO:teuthology.orchestra.run.smithi100.stdout:(101/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-09-13T17:14:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T17:14:35.142 INFO:teuthology.orchestra.run.smithi100.stdout:(102/138): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-13T17:14:35.167 INFO:teuthology.orchestra.run.smithi100.stdout:(103/138): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-13T17:14:35.201 INFO:teuthology.orchestra.run.smithi100.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-13T17:14:35.227 INFO:teuthology.orchestra.run.smithi100.stdout:(105/138): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-09-13T17:14:35.252 INFO:teuthology.orchestra.run.smithi100.stdout:(106/138): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-13T17:14:35.277 INFO:teuthology.orchestra.run.smithi100.stdout:(107/138): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-09-13T17:14:35.291 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T17:14:35.302 INFO:teuthology.orchestra.run.smithi100.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-13T17:14:35.328 INFO:teuthology.orchestra.run.smithi100.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-13T17:14:35.353 INFO:teuthology.orchestra.run.smithi100.stdout:(110/138): python3-google-auth-2.34.0-1.el9.noa 1.4 MB/s | 220 kB 00:00 2024-09-13T17:14:35.355 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T17:14:35.370 INFO:teuthology.orchestra.run.smithi100.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-09-13T17:14:35.387 INFO:teuthology.orchestra.run.smithi100.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-09-13T17:14:35.404 INFO:teuthology.orchestra.run.smithi100.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-13T17:14:35.429 INFO:teuthology.orchestra.run.smithi100.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-13T17:14:35.454 INFO:teuthology.orchestra.run.smithi100.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-13T17:14:35.480 INFO:teuthology.orchestra.run.smithi100.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-13T17:14:35.513 INFO:teuthology.orchestra.run.smithi100.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-13T17:14:35.538 INFO:teuthology.orchestra.run.smithi100.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-13T17:14:35.564 INFO:teuthology.orchestra.run.smithi100.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-13T17:14:35.589 INFO:teuthology.orchestra.run.smithi100.stdout:(120/138): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T17:14:35.614 INFO:teuthology.orchestra.run.smithi100.stdout:(121/138): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-13T17:14:35.639 INFO:teuthology.orchestra.run.smithi100.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-13T17:14:35.673 INFO:teuthology.orchestra.run.smithi100.stdout:(123/138): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-13T17:14:35.698 INFO:teuthology.orchestra.run.smithi100.stdout:(124/138): python3-typing-extensions-4.12.2-2.e 890 kB/s | 74 kB 00:00 2024-09-13T17:14:35.732 INFO:teuthology.orchestra.run.smithi100.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-13T17:14:35.757 INFO:teuthology.orchestra.run.smithi100.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-13T17:14:35.782 INFO:teuthology.orchestra.run.smithi100.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-13T17:14:35.808 INFO:teuthology.orchestra.run.smithi100.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T17:14:35.833 INFO:teuthology.orchestra.run.smithi100.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-13T17:14:35.900 INFO:teuthology.orchestra.run.smithi100.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-13T17:14:35.915 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T17:14:36.019 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T17:14:36.063 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T17:14:36.101 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T17:14:36.143 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T17:14:36.225 INFO:teuthology.orchestra.run.smithi100.stdout:(131/138): NetworkManager-1.49.90-1.el9.x86_64. 7.1 MB/s | 2.3 MB 00:00 2024-09-13T17:14:36.284 INFO:teuthology.orchestra.run.smithi100.stdout:(132/138): librados2-18.2.4-860.g6c6b083f.el9.x 6.9 MB/s | 3.3 MB 00:00 2024-09-13T17:14:36.310 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T17:14:36.334 INFO:teuthology.orchestra.run.smithi100.stdout:(133/138): NetworkManager-libnm-1.49.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-09-13T17:14:36.359 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T17:14:36.384 INFO:teuthology.orchestra.run.smithi100.stdout:(134/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 5.5 MB/s | 3.0 MB 00:00 2024-09-13T17:14:36.410 INFO:teuthology.orchestra.run.smithi100.stdout:(135/138): NetworkManager-team-1.49.90-1.el9.x8 324 kB/s | 41 kB 00:00 2024-09-13T17:14:36.435 INFO:teuthology.orchestra.run.smithi100.stdout:(136/138): NetworkManager-tui-1.49.90-1.el9.x86 2.4 MB/s | 249 kB 00:00 2024-09-13T17:14:36.482 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T17:14:36.530 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T17:14:36.535 INFO:teuthology.orchestra.run.smithi100.stdout:(137/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-13T17:14:36.587 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T17:14:36.619 INFO:teuthology.orchestra.run.smithi100.stdout:(138/138): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-13T17:14:36.624 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:14:36.624 INFO:teuthology.orchestra.run.smithi100.stdout:Total 8.2 MB/s | 203 MB 00:24 2024-09-13T17:14:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T17:14:36.698 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T17:14:36.734 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T17:14:36.781 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T17:14:36.826 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T17:14:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T17:14:37.020 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T17:14:37.061 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T17:14:37.103 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T17:14:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T17:14:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T17:14:37.285 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T17:14:37.326 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T17:14:37.353 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T17:14:37.387 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T17:14:37.416 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T17:14:37.450 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T17:14:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T17:14:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T17:14:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T17:14:37.583 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T17:14:37.626 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T17:14:37.660 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T17:14:37.695 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T17:14:37.697 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-13T17:14:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T17:14:37.841 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-13T17:14:37.841 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-13T17:14:37.862 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T17:14:37.914 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T17:14:38.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T17:14:38.819 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T17:14:38.861 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T17:14:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T17:14:39.002 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T17:14:39.194 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T17:14:39.225 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T17:14:39.305 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T17:14:39.886 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-13T17:14:39.886 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-13T17:14:39.999 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T17:14:40.131 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:14:41.464 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:14:41.557 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T17:14:41.585 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T17:14:41.618 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T17:14:41.934 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T17:14:41.981 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:14:42.056 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:14:42.086 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T17:14:42.123 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T17:14:42.549 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T17:14:42.585 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:14:42.656 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:14:42.686 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T17:14:44.483 INFO:teuthology.orchestra.run.smithi070.stdout:(138/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 863 kB/s | 16 MB 00:19 2024-09-13T17:14:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:14:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:Total 6.0 MB/s | 203 MB 00:34 2024-09-13T17:14:44.858 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:14:44.890 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:14:45.447 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:14:45.485 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:14:45.565 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T17:14:45.587 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:14:45.588 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:14:45.588 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:14:45.588 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:14:45.651 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:14:45.706 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-13T17:14:45.710 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T17:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T17:14:45.806 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T17:14:45.844 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T17:14:45.898 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T17:14:45.931 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T17:14:45.957 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T17:14:46.011 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T17:14:46.327 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T17:14:46.472 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:14:46.711 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:14:46.734 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:14:46.826 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:14:46.855 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:14:47.023 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T17:14:47.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T17:14:47.105 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:14:47.211 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:14:47.241 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:14:47.452 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:14:47.571 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T17:14:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:47.747 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:47.807 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T17:14:47.808 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T17:14:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:48.069 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T17:14:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T17:14:48.177 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T17:14:48.211 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T17:14:48.238 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T17:14:48.307 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T17:14:48.361 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T17:14:48.392 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T17:14:48.435 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T17:14:48.512 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T17:14:48.546 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T17:14:48.589 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T17:14:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T17:14:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T17:14:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T17:14:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T17:14:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T17:14:49.517 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T17:14:49.554 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T17:14:49.596 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T17:14:49.743 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T17:14:49.797 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T17:14:49.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T17:14:49.933 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T17:14:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T17:14:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T17:14:50.093 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T17:14:50.129 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T17:14:50.175 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T17:14:50.220 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T17:14:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T17:14:50.396 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T17:14:50.431 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T17:14:50.472 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T17:14:50.573 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T17:14:50.607 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T17:14:50.648 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T17:14:50.681 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T17:14:50.708 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T17:14:50.741 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T17:14:50.770 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T17:14:50.804 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T17:14:50.840 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T17:14:50.870 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T17:14:50.905 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T17:14:50.938 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T17:14:50.979 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T17:14:51.031 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T17:14:51.066 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T17:14:51.113 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T17:14:51.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T17:14:51.259 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T17:14:51.963 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T17:14:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T17:14:52.138 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T17:14:52.424 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T17:14:52.457 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T17:14:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T17:14:52.672 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T17:14:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T17:14:53.398 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T17:14:53.453 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T17:14:53.518 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:14:53.587 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T17:14:53.633 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T17:14:53.695 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T17:14:53.729 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T17:14:53.754 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T17:14:53.809 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T17:14:54.221 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T17:14:54.308 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:14:54.437 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:14:54.459 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:14:54.551 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:14:54.581 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:14:54.740 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:14:54.751 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T17:14:54.811 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T17:14:54.829 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T17:14:54.838 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:14:54.857 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T17:14:54.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T17:14:54.961 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:14:54.981 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:14:55.190 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T17:14:55.191 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:14:55.228 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:14:55.318 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:14:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T17:14:55.349 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T17:14:55.386 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T17:14:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:55.790 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T17:14:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:14:55.814 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:14:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T17:14:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T17:14:55.885 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:14:55.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T17:14:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T17:14:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T17:14:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T17:14:56.072 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T17:14:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T17:14:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T17:14:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T17:14:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T17:14:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T17:14:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T17:14:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T17:14:56.575 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T17:14:56.631 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T17:14:57.163 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T17:14:57.252 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T17:14:57.291 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T17:14:57.328 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T17:14:57.370 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T17:14:57.534 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T17:14:57.595 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T17:14:57.702 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T17:14:57.741 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T17:14:57.795 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T17:14:57.868 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T17:14:57.909 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T17:14:57.953 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T17:14:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:14:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-09-13T17:14:58.007 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:14:58.008 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T17:14:58.053 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T17:14:58.104 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T17:14:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:14:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:14:58.190 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:14:58.190 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T17:14:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:14:58.239 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T17:14:58.280 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T17:14:58.322 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T17:14:58.424 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T17:14:58.456 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T17:14:58.513 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T17:14:58.535 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:14:58.555 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T17:14:58.576 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:14:58.582 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T17:14:58.615 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T17:14:58.644 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T17:14:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:14:58.678 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:14:58.678 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:14:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:14:58.678 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T17:14:58.714 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T17:14:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:14:58.744 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T17:14:58.779 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T17:14:58.812 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T17:14:58.854 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T17:14:58.888 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T17:14:58.923 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T17:14:58.971 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T17:14:59.093 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T17:14:59.150 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T17:14:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:14:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:14:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:14:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:14:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:14:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:14:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:14:59.870 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T17:14:59.914 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T17:14:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:14:59.958 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T17:14:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:14:59.977 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:14:59.977 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:14:59.977 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:14:59.977 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:14:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T17:15:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T17:15:00.290 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T17:15:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T17:15:00.420 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T17:15:00.445 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:15:00.469 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:15:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:15:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:15:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:15:00.470 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:00.584 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T17:15:00.642 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T17:15:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T17:15:00.899 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T17:15:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T17:15:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T17:15:01.070 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T17:15:01.141 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T17:15:01.189 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:15:01.292 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T17:15:01.338 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T17:15:01.370 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T17:15:01.397 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T17:15:01.847 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:15:01.879 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:15:01.909 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T17:15:01.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T17:15:02.002 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:15:02.451 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:15:02.529 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T17:15:02.556 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T17:15:02.593 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T17:15:02.890 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T17:15:02.931 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:15:03.030 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:15:03.060 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T17:15:03.101 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:15:03.105 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T17:15:03.145 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:15:03.516 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T17:15:03.550 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:15:03.621 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:15:03.652 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T17:15:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:15:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:15:04.294 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:15:04.473 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T17:15:04.498 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:15:04.523 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:15:04.523 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:04.523 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:15:04.524 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:15:04.524 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:15:04.524 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:04.576 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:15:04.588 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:15:04.691 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T17:15:04.747 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T17:15:05.515 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T17:15:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:15:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:15:05.568 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:05.568 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:15:05.568 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:15:05.568 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:15:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:05.610 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:15:05.634 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:15:05.635 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:05.635 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T17:15:05.635 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:05.657 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:15:05.694 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:15:05.699 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:15:05.719 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:15:05.719 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:05.719 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T17:15:05.719 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:15:05.719 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:15:05.720 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:15:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:15:06.053 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:06.053 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:15:06.053 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:15:06.053 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:15:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:06.227 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:15:06.277 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:15:06.363 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:15:06.363 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:15:06.363 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:15:06.363 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:15:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T17:15:10.007 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T17:15:10.048 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T17:15:10.079 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T17:15:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:15:10.252 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-09-13T17:15:10.252 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-09-13T17:15:10.252 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-09-13T17:15:10.252 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-09-13T17:15:10.253 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-09-13T17:15:10.253 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-09-13T17:15:10.253 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-09-13T17:15:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:10.274 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T17:15:10.319 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T17:15:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T17:15:10.379 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T17:15:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:15:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:15:10.431 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:10.431 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T17:15:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:10.433 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T17:15:11.065 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:11.106 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:11.127 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:15:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:15:11.920 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:11.920 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:15:11.920 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:15:11.920 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:15:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T17:15:12.128 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T17:15:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:15:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:15:12.158 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:12.158 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:15:12.159 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:15:12.159 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:15:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T17:15:12.272 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T17:15:12.306 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T17:15:12.340 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T17:15:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T17:15:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:15:12.450 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:15:12.450 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:15:12.542 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:15:12.567 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T17:15:12.574 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:12.574 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:12.594 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:15:12.617 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:15:12.617 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:12.617 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:15:12.618 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:15:12.618 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:15:12.618 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:12.724 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T17:15:12.761 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:12.761 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:15:12.772 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T17:15:12.827 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T17:15:12.853 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:15:12.921 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:13.011 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T17:15:13.061 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T17:15:13.129 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T17:15:13.220 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T17:15:13.364 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T17:15:13.403 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T17:15:13.435 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T17:15:13.463 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T17:15:13.877 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:15:13.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:15:13.948 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T17:15:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T17:15:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:15:15.066 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:15:15.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:15:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:15:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:15:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:15:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T17:15:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:15:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:15:16.438 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:16.438 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:15:16.438 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:15:16.438 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:15:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:15:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:15:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T17:15:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T17:15:17.359 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T17:15:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:15:17.407 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:15:17.407 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:17.407 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:15:17.407 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:15:17.407 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:15:17.407 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:17.451 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:15:17.473 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:15:17.474 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:17.474 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T17:15:17.474 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:15:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:15:17.542 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:17.542 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T17:15:17.542 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:15:17.542 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:15:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:17.835 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:15:17.859 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:15:17.859 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:17.859 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:15:17.859 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:15:17.859 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:15:17.859 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:17.891 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-13T17:15:17.892 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:18.058 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:15:18.073 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:15:18.073 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:18.073 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T17:15:18.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T17:15:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T17:15:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T17:15:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T17:15:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T17:15:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T17:15:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T17:15:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T17:15:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T17:15:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T17:15:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T17:15:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T17:15:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T17:15:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T17:15:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T17:15:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T17:15:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T17:15:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T17:15:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T17:15:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T17:15:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T17:15:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T17:15:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T17:15:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T17:15:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T17:15:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T17:15:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T17:15:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T17:15:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T17:15:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T17:15:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T17:15:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T17:15:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T17:15:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T17:15:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T17:15:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T17:15:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T17:15:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T17:15:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T17:15:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T17:15:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T17:15:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T17:15:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T17:15:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T17:15:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T17:15:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T17:15:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T17:15:19.519 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:15:19.545 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:15:19.545 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:19.545 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:15:19.545 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:15:19.545 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:15:19.546 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:19.745 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:15:19.768 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:15:19.768 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:19.768 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:15:19.768 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:15:19.768 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:15:19.768 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:20.204 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:15:20.226 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:15:20.227 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:20.227 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:15:20.227 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:15:20.227 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:15:20.227 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:20.334 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T17:15:20.382 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T17:15:20.438 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T17:15:20.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T17:15:20.704 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T17:15:20.781 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T17:15:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T17:15:20.942 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:20.942 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:20.942 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-09-13T17:15:20.942 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:20.942 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:20.942 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.943 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.944 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:15:20.945 INFO:teuthology.orchestra.run.smithi044.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:15:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:20.947 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:15:20.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:15:20.949 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:15:20.950 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.951 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:15:20.952 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:15:20.953 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T17:15:21.017 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T17:15:21.063 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T17:15:21.095 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T17:15:21.123 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T17:15:21.466 DEBUG:teuthology.parallel:result is None 2024-09-13T17:15:21.564 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T17:15:21.570 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:15:21.596 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:15:21.605 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T17:15:21.625 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T17:15:21.646 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T17:15:21.678 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T17:15:21.688 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T17:15:21.711 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:15:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T17:15:21.910 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T17:15:21.941 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T17:15:21.970 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T17:15:22.008 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T17:15:22.635 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:22.681 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:22.701 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:22.743 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:15:22.780 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:15:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T17:15:23.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T17:15:23.650 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T17:15:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T17:15:23.744 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:15:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:15:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T17:15:23.798 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T17:15:23.824 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T17:15:23.824 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:15:23.873 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:15:23.908 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:15:23.909 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:15:23.992 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:15:24.017 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T17:15:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T17:15:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:15:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:15:24.097 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:24.097 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:15:24.097 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:15:24.097 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:15:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:15:24.153 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:15:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:15:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T17:15:24.295 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:15:24.306 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T17:15:24.359 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:25.022 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T17:15:25.052 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:15:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:15:25.075 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:25.075 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:15:25.075 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:15:25.076 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:15:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:15:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:15:25.142 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:25.142 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T17:15:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:15:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:15:25.210 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:25.210 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T17:15:25.210 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:15:25.210 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:15:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:25.513 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:15:25.536 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:15:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:15:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:15:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:15:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:15:25.536 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:29.233 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T17:15:29.274 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T17:15:29.314 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T17:15:29.347 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T17:15:29.547 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T17:15:29.595 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T17:15:29.626 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T17:15:29.655 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:15:29.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T17:15:29.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T17:15:29.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T17:15:29.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T17:15:29.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T17:15:29.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T17:15:29.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T17:15:29.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T17:15:29.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T17:15:29.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T17:15:29.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T17:15:29.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T17:15:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T17:15:29.688 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T17:15:29.689 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T17:15:29.690 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T17:15:29.691 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T17:15:29.692 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T17:15:29.693 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T17:15:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T17:15:30.358 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:30.391 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:30.411 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:15:31.233 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T17:15:31.331 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T17:15:31.391 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T17:15:31.480 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T17:15:31.508 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T17:15:31.534 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T17:15:31.560 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T17:15:31.560 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:15:31.669 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:15:31.670 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:15:31.761 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:15:31.787 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T17:15:31.793 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:31.793 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:31.972 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:15:31.972 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:15:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:15:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:32.407 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:32.407 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:32.407 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-09-13T17:15:32.407 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:32.407 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.408 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.409 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:15:32.410 INFO:teuthology.orchestra.run.smithi100.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:15:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:32.412 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:15:32.413 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:15:32.414 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:15:32.415 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:15:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:32.417 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:15:32.418 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-13T17:15:32.778 DEBUG:teuthology.parallel:result is None 2024-09-13T17:15:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T17:15:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T17:15:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T17:15:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T17:15:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T17:15:37.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T17:15:37.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T17:15:37.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T17:15:37.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T17:15:37.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T17:15:37.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T17:15:37.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T17:15:37.322 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T17:15:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T17:15:37.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T17:15:37.325 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:15:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:15:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T17:15:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:15:39.945 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:15:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:15:39.947 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:15:39.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:15:39.949 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T17:15:39.950 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T17:15:39.950 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:15:39.950 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:15:39.950 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T17:15:39.950 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T17:15:40.294 DEBUG:teuthology.parallel:result is None 2024-09-13T17:15:40.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:15:40.449 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:15:40.477 INFO:teuthology.orchestra.run.smithi044.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T17:15:40.478 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T17:15:40.478 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T17:15:40.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:15:40.634 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:15:40.656 INFO:teuthology.orchestra.run.smithi070.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T17:15:40.657 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T17:15:40.657 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T17:15:40.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:15:40.809 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:15:40.836 INFO:teuthology.orchestra.run.smithi100.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T17:15:40.836 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T17:15:40.836 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T17:15:40.838 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T17:15:40.838 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:15:40.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:15:40.872 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:15:40.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:15:40.900 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:15:40.901 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:15:40.932 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T17:15:40.933 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:15:40.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:15:40.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:15:41.034 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:15:41.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:15:41.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:15:41.125 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:15:41.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:15:41.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:15:41.225 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T17:15:41.225 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:15:41.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:15:41.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:15:41.327 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:15:41.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:15:41.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:15:41.418 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:15:41.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:15:41.448 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:15:41.518 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T17:15:41.518 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:15:41.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:15:41.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:15:41.610 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:15:41.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:15:41.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:15:41.699 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:15:41.700 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:15:41.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:15:41.799 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-13T17:15:41.896 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'subvols': {'create': 2}} 2024-09-13T17:15:41.900 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:15:41.900 INFO:tasks.cephadm:Cluster fsid is ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:15:41.900 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-13T17:15:41.901 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.44', 'mon.b': '172.21.15.70', 'mon.c': '172.21.15.100'} 2024-09-13T17:15:41.901 INFO:tasks.cephadm:First mon is mon.a on smithi044 2024-09-13T17:15:41.901 INFO:tasks.cephadm:First mgr is x 2024-09-13T17:15:41.901 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-13T17:15:41.901 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-09-13T17:15:41.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2024-09-13T17:15:41.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-09-13T17:15:41.983 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-13T17:15:41.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:15:42.136 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82954/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-860-g6c6b083f', 'node_name': '172.21.4.58+omani008', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-860.g6c6b083f'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 17:57:22.919627', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240910.100730-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'archs': ['arm64', 'x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82954/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-860-g6c6b083f', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-860.g6c6b083f'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 13:55:29.568955', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240910.100730-reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-13T17:15:42.289 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240910.100730-reef, sha1 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:15:42.291 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm 2024-09-13T17:15:42.291 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:42.640 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 13 17:15 /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:42.640 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:42.972 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 13 17:15 /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:42.972 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:43.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 13 17:15 /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:43.319 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:43.340 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:43.360 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T17:15:43.394 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 on all hosts... 2024-09-13T17:15:43.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 pull 2024-09-13T17:15:43.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 pull 2024-09-13T17:15:43.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 pull 2024-09-13T17:15:44.925 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T17:15:45.008 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T17:15:45.028 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T17:16:11.255 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T17:16:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:16:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61", 2024-09-13T17:16:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2024-09-13T17:16:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256" 2024-09-13T17:16:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T17:16:11.256 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T17:16:28.613 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:16:28.614 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:16:28.614 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61", 2024-09-13T17:16:28.614 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-09-13T17:16:28.614 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256" 2024-09-13T17:16:28.614 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:16:28.614 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:16:36.228 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-13T17:16:36.228 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:16:36.228 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61", 2024-09-13T17:16:36.228 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-09-13T17:16:36.228 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256" 2024-09-13T17:16:36.229 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-13T17:16:36.229 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-13T17:16:36.246 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-09-13T17:16:36.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2024-09-13T17:16:36.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-09-13T17:16:36.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-09-13T17:16:36.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2024-09-13T17:16:36.404 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-09-13T17:16:36.428 INFO:tasks.cephadm:Writing seed config... 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-13T17:16:36.430 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-13T17:16:36.431 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-13T17:16:36.432 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-13T17:16:36.433 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-13T17:16:36.433 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-13T17:16:36.433 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-13T17:16:36.433 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-13T17:16:36.433 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-13T17:16:36.433 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-13T17:16:36.434 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:16:36.434 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-13T17:16:36.453 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ce020790-71f3-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-13T17:16:36.453 DEBUG:teuthology.orchestra.run.smithi044:mon.a> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a.service 2024-09-13T17:16:36.495 DEBUG:teuthology.orchestra.run.smithi044:mgr.x> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x.service 2024-09-13T17:16:36.539 INFO:tasks.cephadm:Bootstrapping... 2024-09-13T17:16:36.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 -v bootstrap --fsid ce020790-71f3-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.44 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:16:36.704 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:16:36.705 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', '-v', 'bootstrap', '--fsid', 'ce020790-71f3-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.44', '--skip-admin-label'] 2024-09-13T17:16:36.729 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 5.2.2 2024-09-13T17:16:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-13T17:16:36.730 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying podman|docker is present... 2024-09-13T17:16:36.752 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 5.2.2 2024-09-13T17:16:36.752 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying lvm2 is present... 2024-09-13T17:16:36.752 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying time synchronization is in place... 2024-09-13T17:16:36.760 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T17:16:36.760 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T17:16:36.767 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T17:16:36.767 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-09-13T17:16:36.774 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2024-09-13T17:16:36.780 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2024-09-13T17:16:36.780 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2024-09-13T17:16:36.780 INFO:teuthology.orchestra.run.smithi044.stdout:Repeating the final host check... 2024-09-13T17:16:36.803 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 5.2.2 2024-09-13T17:16:36.803 INFO:teuthology.orchestra.run.smithi044.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-13T17:16:36.803 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl is present 2024-09-13T17:16:36.803 INFO:teuthology.orchestra.run.smithi044.stdout:lvcreate is present 2024-09-13T17:16:36.810 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T17:16:36.810 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T17:16:36.816 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T17:16:36.816 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-09-13T17:16:36.823 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2024-09-13T17:16:36.832 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Host looks OK 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Cluster fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Acquiring lock 139633249087056 on /run/cephadm/ce020790-71f3-11ef-bceb-c7b262605968.lock 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Lock 139633249087056 acquired on /run/cephadm/ce020790-71f3-11ef-bceb-c7b262605968.lock 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 3300 ... 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 6789 ... 2024-09-13T17:16:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:Base mon IP(s) is [172.21.15.44:3300, 172.21.15.44:6789], mon addrv is [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-09-13T17:16:36.835 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.44 metric 100 2024-09-13T17:16:36.835 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.44 metric 100 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1782sec hoplimit 64 pref medium 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi044.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi044.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-13T17:16:36.842 INFO:teuthology.orchestra.run.smithi044.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Getting image source signatures 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:cdfdf789b01393b98d7733e02a8d993977e2aa05d922747269d27c845cd4920c 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying config sha256:8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 2024-09-13T17:16:37.298 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-13T17:16:38.186 INFO:teuthology.orchestra.run.smithi044.stdout:ceph: stdout ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable) 2024-09-13T17:16:38.186 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph version: ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable) 2024-09-13T17:16:38.186 INFO:teuthology.orchestra.run.smithi044.stdout:Extracting ceph user uid/gid from container image... 2024-09-13T17:16:39.118 INFO:teuthology.orchestra.run.smithi044.stdout:stat: stdout 167 167 2024-09-13T17:16:39.118 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial keys... 2024-09-13T17:16:40.123 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQB3c+RmUb3RGhAAdmT8lIhNN26uCmXeWCOy5A== 2024-09-13T17:16:47.911 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQB5c+RmOx/HGhAAvIVKPGwBiBcXmf1x6pSalQ== 2024-09-13T17:16:51.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQCCc+Rm6PCoMhAAEJX94sODK0nsZvcDcxHlOQ== 2024-09-13T17:16:51.517 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial monmap... 2024-09-13T17:16:52.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T17:16:52.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool for a [v2:172.21.15.44:3300,v1:172.21.15.44:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = pacific 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: set fsid to ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:16:52.448 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mon... 2024-09-13T17:16:53.612 INFO:teuthology.orchestra.run.smithi044.stdout:create mon.a on 2024-09-13T17:16:53.828 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-13T17:16:53.977 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-13T17:16:54.246 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ce020790-71f3-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ce020790-71f3-11ef-bceb-c7b262605968.target. 2024-09-13T17:16:54.246 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ce020790-71f3-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ce020790-71f3-11ef-bceb-c7b262605968.target. 2024-09-13T17:16:54.497 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a 2024-09-13T17:16:54.497 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-13T17:16:54.674 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ce020790-71f3-11ef-bceb-c7b262605968.target.wants/ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ce020790-71f3-11ef-bceb-c7b262605968@.service. 2024-09-13T17:16:55.512 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T17:16:55.512 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-09-13T17:16:55.519 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T17:16:55.520 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-09-13T17:16:55.520 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-09-13T17:16:55.520 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T17:16:55.520 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon to start... 2024-09-13T17:16:55.520 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon... 2024-09-13T17:16:55.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:55 smithi044 systemd[1]: Started Ceph mon.a for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:16:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:56 smithi044 ceph-mon[30807]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T17:16:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:56 smithi044 ceph-mon[30807]: monmap e1: 1 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:16:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:56 smithi044 ceph-mon[30807]: fsmap 2024-09-13T17:16:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:56 smithi044 ceph-mon[30807]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T17:16:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:56 smithi044 ceph-mon[30807]: mgrmap e1: no daemons active 2024-09-13T17:16:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:56 smithi044 ceph-mon[30807]: from='client.? 172.21.15.44:0/3219569111' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-13T17:16:56.922 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout cluster: 2024-09-13T17:16:56.922 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout id: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:16:56.922 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-13T17:16:56.922 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:16:56.922 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout services: 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.736286s) 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout data: 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pgs: 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.971+0000 7ff88897d640 1 Processor -- start 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.971+0000 7ff88897d640 1 -- start start 2024-09-13T17:16:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.971+0000 7ff88897d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff8841084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:56.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.971+0000 7ff88897d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884079e90 con 0x7ff8841060f0 2024-09-13T17:16:56.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.972+0000 7ff88259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff8841084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:56.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.972+0000 7ff88259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff8841084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44366/0 (socket says 172.21.15.44:44366) 2024-09-13T17:16:56.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.972+0000 7ff88259c640 1 -- 172.21.15.44:0/3165839404 learned_addr learned my addr 172.21.15.44:0/3165839404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:16:56.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.972+0000 7ff88259c640 1 -- 172.21.15.44:0/3165839404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884108a20 con 0x7ff8841060f0 2024-09-13T17:16:56.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.974+0000 7ff88259c640 1 --2- 172.21.15.44:0/3165839404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff8841084e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff874009820 tx=0x7ff87402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a304ec52a45e00c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.974+0000 7ff88159a640 1 -- 172.21.15.44:0/3165839404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87402ae70 con 0x7ff8841060f0 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.974+0000 7ff88159a640 1 -- 172.21.15.44:0/3165839404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874032440 con 0x7ff8841060f0 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.974+0000 7ff88159a640 1 -- 172.21.15.44:0/3165839404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff874032700 con 0x7ff8841060f0 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 -- 172.21.15.44:0/3165839404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 msgr2=0x7ff8841084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 --2- 172.21.15.44:0/3165839404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff8841084e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff874009820 tx=0x7ff87402a270 comp rx=0 tx=0).stop 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 -- 172.21.15.44:0/3165839404 shutdown_connections 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 --2- 172.21.15.44:0/3165839404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff8841084e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 -- 172.21.15.44:0/3165839404 >> 172.21.15.44:0/3165839404 conn(0x7ff8840ffcf0 msgr2=0x7ff884102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 -- 172.21.15.44:0/3165839404 shutdown_connections 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.975+0000 7ff88897d640 1 -- 172.21.15.44:0/3165839404 wait complete. 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88897d640 1 Processor -- start 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88897d640 1 -- start start 2024-09-13T17:16:56.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88897d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff884105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88897d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884105eb0 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff884105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff884105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44382/0 (socket says 172.21.15.44:44382) 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88259c640 1 -- 172.21.15.44:0/3219569111 learned_addr learned my addr 172.21.15.44:0/3219569111 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.976+0000 7ff88259c640 1 -- 172.21.15.44:0/3219569111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740094d0 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff88259c640 1 --2- 172.21.15.44:0/3219569111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff884105970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff8740090b0 tx=0x7ff8740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8740308d0 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874030a30 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8841040a0 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff874030cf0 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884104540 con 0x7ff8841060f0 2024-09-13T17:16:56.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff874039040 con 0x7ff8841060f0 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.977+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff8740468c0 con 0x7ff8841060f0 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.978+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884104e90 con 0x7ff8841060f0 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:55.980+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff874037070 con 0x7ff8841060f0 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.040+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff884108cd0 con 0x7ff8841060f0 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.040+0000 7ff8737fe640 1 -- 172.21.15.44:0/3219569111 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff874046ae0 con 0x7ff8841060f0 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 msgr2=0x7ff884105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 --2- 172.21.15.44:0/3219569111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff884105970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff8740090b0 tx=0x7ff8740046c0 comp rx=0 tx=0).stop 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 shutdown_connections 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 --2- 172.21.15.44:0/3219569111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8841060f0 0x7ff884105970 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 >> 172.21.15.44:0/3219569111 conn(0x7ff8840ffcf0 msgr2=0x7ff884100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 shutdown_connections 2024-09-13T17:16:56.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:56.041+0000 7ff88897d640 1 -- 172.21.15.44:0/3219569111 wait complete. 2024-09-13T17:16:56.928 INFO:teuthology.orchestra.run.smithi044.stdout:mon is available 2024-09-13T17:16:56.928 INFO:teuthology.orchestra.run.smithi044.stdout:Assimilating anything we can from ceph.conf... 2024-09-13T17:16:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:57 smithi044 ceph-mon[30807]: from='client.? 172.21.15.44:0/1902478969' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-13T17:16:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:57 smithi044 ceph-mon[30807]: from='client.? 172.21.15.44:0/1902478969' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T17:16:57.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.333+0000 7f5e7768e640 1 Processor -- start 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.333+0000 7f5e7768e640 1 -- start start 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.333+0000 7f5e7768e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e70105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:57.924 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.333+0000 7f5e7768e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70106010 con 0x7f5e70105640 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.334+0000 7f5e7542a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e70105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.334+0000 7f5e7542a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e70105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44394/0 (socket says 172.21.15.44:44394) 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.334+0000 7f5e7542a640 1 -- 172.21.15.44:0/781790513 learned_addr learned my addr 172.21.15.44:0/781790513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.334+0000 7f5e7542a640 1 -- 172.21.15.44:0/781790513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70106840 con 0x7f5e70105640 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e7542a640 1 --2- 172.21.15.44:0/781790513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e70105a40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009820 tx=0x7f5e6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38094274efafd199 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e67fff640 1 -- 172.21.15.44:0/781790513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c02ae70 con 0x7f5e70105640 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e67fff640 1 -- 172.21.15.44:0/781790513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c032440 con 0x7f5e70105640 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e67fff640 1 -- 172.21.15.44:0/781790513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c032700 con 0x7f5e70105640 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e7768e640 1 -- 172.21.15.44:0/781790513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 msgr2=0x7f5e70105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e7768e640 1 --2- 172.21.15.44:0/781790513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e70105a40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009820 tx=0x7f5e6c02a270 comp rx=0 tx=0).stop 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e7768e640 1 -- 172.21.15.44:0/781790513 shutdown_connections 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e7768e640 1 --2- 172.21.15.44:0/781790513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e70105a40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:57.925 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.335+0000 7f5e7768e640 1 -- 172.21.15.44:0/781790513 >> 172.21.15.44:0/781790513 conn(0x7f5e70100df0 msgr2=0x7f5e70103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7768e640 1 -- 172.21.15.44:0/781790513 shutdown_connections 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7768e640 1 -- 172.21.15.44:0/781790513 wait complete. 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7768e640 1 Processor -- start 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7768e640 1 -- start start 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7768e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e7007bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7768e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7007c0d0 con 0x7f5e70105640 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.336+0000 7f5e7542a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e7007bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e7542a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e7007bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44398/0 (socket says 172.21.15.44:44398) 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e7542a640 1 -- 172.21.15.44:0/1902478969 learned_addr learned my addr 172.21.15.44:0/1902478969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e7542a640 1 -- 172.21.15.44:0/1902478969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0094d0 con 0x7f5e70105640 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e7542a640 1 --2- 172.21.15.44:0/1902478969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e7007bb90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0090b0 tx=0x7f5e6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c0308d0 con 0x7f5e70105640 2024-09-13T17:16:57.926 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c030a30 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7007a2c0 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c030cf0 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7007a760 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.337+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5e6c039040 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.338+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5e6c0468c0 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.338+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7007b0b0 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.340+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f5e6c037070 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.399+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5e70073df0 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.403+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5e6c04f8a0 con 0x7f5e70105640 2024-09-13T17:16:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.403+0000 7f5e667fc640 1 -- 172.21.15.44:0/1902478969 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5e6c046b90 con 0x7f5e70105640 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 msgr2=0x7f5e7007bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 --2- 172.21.15.44:0/1902478969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e7007bb90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0090b0 tx=0x7f5e6c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 shutdown_connections 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 --2- 172.21.15.44:0/1902478969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e70105640 0x7f5e7007bb90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 >> 172.21.15.44:0/1902478969 conn(0x7f5e70100df0 msgr2=0x7f5e7018fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 shutdown_connections 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:57.404+0000 7f5e7768e640 1 -- 172.21.15.44:0/1902478969 wait complete. 2024-09-13T17:16:57.928 INFO:teuthology.orchestra.run.smithi044.stdout:Generating new minimal ceph.conf... 2024-09-13T17:16:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:58 smithi044 ceph-mon[30807]: from='client.? 172.21.15.44:0/3057334901' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:16:58.976 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.295+0000 7f9735812640 1 Processor -- start 2024-09-13T17:16:58.976 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.295+0000 7f9735812640 1 -- start start 2024-09-13T17:16:58.976 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.295+0000 7f9735812640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.295+0000 7f9735812640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730106010 con 0x7f9730105640 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.296+0000 7f972effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.296+0000 7f972effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44408/0 (socket says 172.21.15.44:44408) 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.296+0000 7f972effd640 1 -- 172.21.15.44:0/1273070167 learned_addr learned my addr 172.21.15.44:0/1273070167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.296+0000 7f972effd640 1 -- 172.21.15.44:0/1273070167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730106840 con 0x7f9730105640 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.297+0000 7f972effd640 1 --2- 172.21.15.44:0/1273070167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730105a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9720009820 tx=0x7f972002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4432baafaf2c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.297+0000 7f972e7fc640 1 -- 172.21.15.44:0/1273070167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f972002ae70 con 0x7f9730105640 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.297+0000 7f972e7fc640 1 -- 172.21.15.44:0/1273070167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f97200328b0 con 0x7f9730105640 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.297+0000 7f972e7fc640 1 -- 172.21.15.44:0/1273070167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9720030850 con 0x7f9730105640 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- 172.21.15.44:0/1273070167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 msgr2=0x7f9730105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 --2- 172.21.15.44:0/1273070167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730105a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9720009820 tx=0x7f972002a270 comp rx=0 tx=0).stop 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- 172.21.15.44:0/1273070167 shutdown_connections 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 --2- 172.21.15.44:0/1273070167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730105a40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:58.977 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- 172.21.15.44:0/1273070167 >> 172.21.15.44:0/1273070167 conn(0x7f9730100df0 msgr2=0x7f9730103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- 172.21.15.44:0/1273070167 shutdown_connections 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- 172.21.15.44:0/1273070167 wait complete. 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 Processor -- start 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- start start 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.298+0000 7f9735812640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730199cc0 con 0x7f9730105640 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f972effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f972effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44410/0 (socket says 172.21.15.44:44410) 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f972effd640 1 -- 172.21.15.44:0/3057334901 learned_addr learned my addr 172.21.15.44:0/3057334901 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f972effd640 1 -- 172.21.15.44:0/3057334901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97200094d0 con 0x7f9730105640 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f972effd640 1 --2- 172.21.15.44:0/3057334901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730199780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f97200097f0 tx=0x7f9720030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9720032c30 con 0x7f9730105640 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9720032d90 con 0x7f9730105640 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97301076c0 con 0x7f9730105640 2024-09-13T17:16:58.978 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f972003a5b0 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973019a1d0 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9720039040 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.299+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9720044ad0 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.300+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9730105ac0 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.302+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f972003f020 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.360+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9730109b70 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.361+0000 7f9734810640 1 -- 172.21.15.44:0/3057334901 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f9720037070 con 0x7f9730105640 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.361+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 msgr2=0x7f9730199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.361+0000 7f9735812640 1 --2- 172.21.15.44:0/3057334901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730199780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f97200097f0 tx=0x7f9720030d40 comp rx=0 tx=0).stop 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.362+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 shutdown_connections 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.362+0000 7f9735812640 1 --2- 172.21.15.44:0/3057334901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9730105640 0x7f9730199780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.362+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 >> 172.21.15.44:0/3057334901 conn(0x7f9730100df0 msgr2=0x7f97301019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:58.979 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.362+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 shutdown_connections 2024-09-13T17:16:58.980 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:16:58.362+0000 7f9735812640 1 -- 172.21.15.44:0/3057334901 wait complete. 2024-09-13T17:16:58.980 INFO:teuthology.orchestra.run.smithi044.stdout:Restarting the monitor... 2024-09-13T17:16:59.240 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:58 smithi044 systemd[1]: Stopping Ceph mon.a for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:16:59.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a[30803]: 2024-09-13T17:16:59.238+0000 7fb1df9b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:16:59.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a[30803]: 2024-09-13T17:16:59.238+0000 7fb1df9b5640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-13T17:16:59.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:59 smithi044 podman[31048]: 2024-09-13 17:16:59.362217079 +0000 UTC m=+0.218860477 container died 3d89c29f2ce6fdb6ab5b45923fe57af075c37355b199a586f0bc0caa4302f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:16:59.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:59 smithi044 podman[31048]: 2024-09-13 17:16:59.680004415 +0000 UTC m=+0.536647800 container cleanup 3d89c29f2ce6fdb6ab5b45923fe57af075c37355b199a586f0bc0caa4302f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-13T17:16:59.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:59 smithi044 bash[31048]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a 2024-09-13T17:17:00.136 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:16:59 smithi044 podman[31060]: 2024-09-13 17:16:59.872925146 +0000 UTC m=+0.512328476 container remove 3d89c29f2ce6fdb6ab5b45923fe57af075c37355b199a586f0bc0caa4302f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-13T17:17:00.136 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T17:17:00.136 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 systemd[1]: Stopped Ceph mon.a for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:17:00.136 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 systemd[1]: Starting Ceph mon.a for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:17:00.421 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 podman[31155]: 2024-09-13 17:17:00.315667916 +0000 UTC m=+0.111428466 container create 1c73e264c322c4a206bd3b3bfecd885131e6adc5f7b0051336234f2f55e60528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T17:17:00.421 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 podman[31155]: 2024-09-13 17:17:00.220152573 +0000 UTC m=+0.015913091 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:17:00.631 INFO:teuthology.orchestra.run.smithi044.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-13T17:17:00.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 podman[31155]: 2024-09-13 17:17:00.538428793 +0000 UTC m=+0.334189304 container init 1c73e264c322c4a206bd3b3bfecd885131e6adc5f7b0051336234f2f55e60528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:17:00.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 podman[31155]: 2024-09-13 17:17:00.541607885 +0000 UTC m=+0.337368397 container start 1c73e264c322c4a206bd3b3bfecd885131e6adc5f7b0051336234f2f55e60528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:17:00.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T17:17:00.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable), process ceph-mon, pid 2 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: pidfile_write: ignore empty --pid-file 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: load: jerasure load: lrc 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: RocksDB version: 7.9.2 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Git sha 0 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Compile date 2024-09-10 12:59:06 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: DB SUMMARY 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: DB Session ID: QZ4HZP33M1KV4N9EZ3RC 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: CURRENT file: CURRENT 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-13T17:17:00.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100347 ; 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.error_if_exists: 0 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.create_if_missing: 0 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.paranoid_checks: 1 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.env: 0x560a520b2bc0 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.info_log: 0x560a52c98be0 2024-09-13T17:17:00.757 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.statistics: (nil) 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.use_fsync: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_log_file_size: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.allow_fallocate: 1 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.use_direct_reads: 0 2024-09-13T17:17:00.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.db_log_dir: 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.wal_dir: 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T17:17:00.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.write_buffer_manager: 0x560a52ca6320 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.unordered_write: 0 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T17:17:00.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.row_cache: None 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.wal_filter: None 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.two_write_queues: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.wal_compression: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.atomic_flush: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.log_readahead_size: 0 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T17:17:00.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_background_jobs: 2 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_background_compactions: -1 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_subcompactions: 1 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T17:17:00.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_open_files: -1 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_background_flushes: -1 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Compression algorithms supported: 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kZSTD supported: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kXpressCompression supported: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kBZip2Compression supported: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kLZ4Compression supported: 1 2024-09-13T17:17:00.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kZlibCompression supported: 1 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: kSnappyCompression supported: 1 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.merge_operator: 2024-09-13T17:17:00.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_filter: None 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560a52c987e0) 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_type: 0 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_index_type: 0 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_shortening: 1 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: checksum: 4 2024-09-13T17:17:00.766 INFO:journalctl@ceph.mon.a.smithi044.stdout: no_block_cache: 0 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache: 0x560a52c8d350 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_options: 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: capacity : 536870912 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_shard_bits : 4 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: strict_capacity_limit : 0 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_compressed: (nil) 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: persistent_cache: (nil) 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size: 4096 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size_deviation: 10 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_restart_interval: 16 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_block_restart_interval: 1 2024-09-13T17:17:00.767 INFO:journalctl@ceph.mon.a.smithi044.stdout: metadata_block_size: 4096 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: partition_filters: 0 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: use_delta_encoding: 1 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: filter_policy: bloomfilter 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: whole_key_filtering: 1 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: verify_compression: 0 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: format_version: 5 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_index_compression: 1 2024-09-13T17:17:00.768 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_align: 0 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout: max_auto_readahead_size: 262144 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout: prepopulate_block_cache: 0 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout: initial_auto_readahead_size: 8192 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression: NoCompression 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.num_levels: 7 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T17:17:00.769 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T17:17:00.770 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T17:17:00.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T17:17:00.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T17:17:00.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.inplace_update_support: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.bloom_locality: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.max_successive_merges: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.ttl: 2592000 2024-09-13T17:17:00.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.enable_blob_files: false 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.min_blob_size: 0 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-13T17:17:00.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c83a4b60-ca67-41cb-9bdb-deec4dbf2353 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726247820568494, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726247820569667, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95480, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93443, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14007, "raw_average_key_size": 50, "raw_value_size": 85821, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726247820, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c83a4b60-ca67-41cb-9bdb-deec4dbf2353", "db_session_id": "QZ4HZP33M1KV4N9EZ3RC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726247820569766, "job": 1, "event": "recovery_finished"} 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560a52cb8e00 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: DB pointer 0x560a52dac000 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** DB Stats ** 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T17:17:00.776 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: L0 2/0 95.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Sum 2/0 95.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-09-13T17:17:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative compaction: 0.00 GB write, 26.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T17:17:00.778 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval compaction: 0.00 GB write, 26.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache BinnedLRUCache@0x560a52c8d350#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: starting mon.a rank 0 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???) e1 preinit fsid ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).mds e1 new map 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).mds e1 print_map 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: e1 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T17:17:00.779 INFO:journalctl@ceph.mon.a.smithi044.stdout: legacy client fscid: -1 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout: No filesystems configured 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 bash[31155]: 1c73e264c322c4a206bd3b3bfecd885131e6adc5f7b0051336234f2f55e60528 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 systemd[1]: Started Ceph mon.a for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: monmap e1: 1 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: fsmap 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T17:17:00.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:00 smithi044 ceph-mon[31175]: mgrmap e1: no daemons active 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.120+0000 7fa1be252640 1 Processor -- start 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.121+0000 7fa1be252640 1 -- start start 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.121+0000 7fa1be252640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.121+0000 7fa1be252640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8106010 con 0x7fa1b8105640 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.121+0000 7fa1b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.121+0000 7fa1b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52176/0 (socket says 172.21.15.44:52176) 2024-09-13T17:17:01.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.121+0000 7fa1b77fe640 1 -- 172.21.15.44:0/2357005908 learned_addr learned my addr 172.21.15.44:0/2357005908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.122+0000 7fa1b77fe640 1 -- 172.21.15.44:0/2357005908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b8106840 con 0x7fa1b8105640 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.123+0000 7fa1b77fe640 1 --2- 172.21.15.44:0/2357005908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8105a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009820 tx=0x7fa1a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae879015d228207 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.123+0000 7fa1b67fc640 1 -- 172.21.15.44:0/2357005908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1a802ae70 con 0x7fa1b8105640 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1b67fc640 1 -- 172.21.15.44:0/2357005908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa1a80328b0 con 0x7fa1b8105640 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1b67fc640 1 -- 172.21.15.44:0/2357005908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8030850 con 0x7fa1b8105640 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 -- 172.21.15.44:0/2357005908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 msgr2=0x7fa1b8105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 --2- 172.21.15.44:0/2357005908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8105a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009820 tx=0x7fa1a802a270 comp rx=0 tx=0).stop 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 -- 172.21.15.44:0/2357005908 shutdown_connections 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 --2- 172.21.15.44:0/2357005908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8105a40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 -- 172.21.15.44:0/2357005908 >> 172.21.15.44:0/2357005908 conn(0x7fa1b8100df0 msgr2=0x7fa1b8103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 -- 172.21.15.44:0/2357005908 shutdown_connections 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.124+0000 7fa1be252640 1 -- 172.21.15.44:0/2357005908 wait complete. 2024-09-13T17:17:01.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.125+0000 7fa1be252640 1 Processor -- start 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.125+0000 7fa1be252640 1 -- start start 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.125+0000 7fa1be252640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.125+0000 7fa1be252640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8199cb0 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.125+0000 7fa1b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52188/0 (socket says 172.21.15.44:52188) 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b77fe640 1 -- 172.21.15.44:0/564889488 learned_addr learned my addr 172.21.15.44:0/564889488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b77fe640 1 -- 172.21.15.44:0/564889488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80094d0 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b77fe640 1 --2- 172.21.15.44:0/564889488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8199770 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8030560 tx=0x7fa1a8030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8032d60 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa1a803b8a0 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1a803a5f0 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b81076c0 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b819a1c0 con 0x7fa1b8105640 2024-09-13T17:17:01.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.126+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa1a8039040 con 0x7fa1b8105640 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.127+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa1a8044ae0 con 0x7fa1b8105640 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.127+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b8105ac0 con 0x7fa1b8105640 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.130+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa1a803f020 con 0x7fa1b8105640 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.188+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa1b8073f10 con 0x7fa1b8105640 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.190+0000 7fa1b4ff9640 1 -- 172.21.15.44:0/564889488 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa1a8037070 con 0x7fa1b8105640 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 msgr2=0x7fa1b8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 --2- 172.21.15.44:0/564889488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8199770 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8030560 tx=0x7fa1a8030d90 comp rx=0 tx=0).stop 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 shutdown_connections 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 --2- 172.21.15.44:0/564889488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8105640 0x7fa1b8199770 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 >> 172.21.15.44:0/564889488 conn(0x7fa1b8100df0 msgr2=0x7fa1b8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 shutdown_connections 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:01.191+0000 7fa1be252640 1 -- 172.21.15.44:0/564889488 wait complete. 2024-09-13T17:17:01.727 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-13T17:17:01.742 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:17:01.742 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mgr... 2024-09-13T17:17:01.742 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-13T17:17:01.743 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-13T17:17:02.000 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x 2024-09-13T17:17:02.001 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-13T17:17:02.158 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ce020790-71f3-11ef-bceb-c7b262605968.target.wants/ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ce020790-71f3-11ef-bceb-c7b262605968@.service. 2024-09-13T17:17:02.496 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/564889488' entity='client.admin' 2024-09-13T17:17:02.712 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T17:17:02.712 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-09-13T17:17:02.719 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T17:17:02.719 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-09-13T17:17:02.719 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-09-13T17:17:02.719 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T17:17:02.728 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T17:17:02.728 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-09-13T17:17:02.736 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T17:17:02.736 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-09-13T17:17:02.736 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-09-13T17:17:02.736 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-13T17:17:02.736 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr to start... 2024-09-13T17:17:02.736 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr... 2024-09-13T17:17:02.856 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:02.786+0000 7fb7e5c2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:17:03.148 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:02.942+0000 7fb7e5c2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:17:03.148 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:03.012+0000 7fb7e5c2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:17:03.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4019155975' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T17:17:03.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:03.167+0000 7fb7e5c2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:17:03.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:03.318+0000 7fb7e5c2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:17:03.897 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:03.643+0000 7fb7e5c2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:17:03.897 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:03.706+0000 7fb7e5c2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T17:17:04.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T17:17:04.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T17:17:04.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:04.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T17:17:04.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T17:17:04.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T17:17:04.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T17:17:04.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T17:17:04.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T17:17:04.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:04.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T17:17:04.029 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.029 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T17:17:04.029 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T17:16:55.306262+0000", 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.146+0000 7f9bc4f7a640 1 Processor -- start 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.146+0000 7f9bc4f7a640 1 -- start start 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.146+0000 7f9bc4f7a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc01042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.146+0000 7f9bc4f7a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0079e90 con 0x7f9bc0103ee0 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.147+0000 7f9bbe59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc01042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.147+0000 7f9bbe59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc01042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52216/0 (socket says 172.21.15.44:52216) 2024-09-13T17:17:04.030 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.147+0000 7f9bbe59c640 1 -- 172.21.15.44:0/1115214833 learned_addr learned my addr 172.21.15.44:0/1115214833 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.147+0000 7f9bbe59c640 1 -- 172.21.15.44:0/1115214833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc0104820 con 0x7f9bc0103ee0 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.148+0000 7f9bbe59c640 1 --2- 172.21.15.44:0/1115214833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc01042e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009820 tx=0x7f9bb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e3cff395476fb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.148+0000 7f9bbdd9b640 1 -- 172.21.15.44:0/1115214833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bb002ae70 con 0x7f9bc0103ee0 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.148+0000 7f9bbdd9b640 1 -- 172.21.15.44:0/1115214833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9bb00328b0 con 0x7f9bc0103ee0 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.148+0000 7f9bbdd9b640 1 -- 172.21.15.44:0/1115214833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bb0030850 con 0x7f9bc0103ee0 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/1115214833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 msgr2=0x7f9bc01042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 --2- 172.21.15.44:0/1115214833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc01042e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009820 tx=0x7f9bb002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/1115214833 shutdown_connections 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 --2- 172.21.15.44:0/1115214833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc01042e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/1115214833 >> 172.21.15.44:0/1115214833 conn(0x7f9bc00ffcf0 msgr2=0x7f9bc0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/1115214833 shutdown_connections 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.149+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/1115214833 wait complete. 2024-09-13T17:17:04.031 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bc4f7a640 1 Processor -- start 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bc4f7a640 1 -- start start 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bc4f7a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bc4f7a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc019bca0 con 0x7f9bc0103ee0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bbe59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bbe59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52222/0 (socket says 172.21.15.44:52222) 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.150+0000 7f9bbe59c640 1 -- 172.21.15.44:0/4019155975 learned_addr learned my addr 172.21.15.44:0/4019155975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bbe59c640 1 -- 172.21.15.44:0/4019155975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00094d0 con 0x7f9bc0103ee0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bbe59c640 1 --2- 172.21.15.44:0/4019155975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc019b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00097f0 tx=0x7f9bb0030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bb0032c30 con 0x7f9bc0103ee0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9bb0032d90 con 0x7f9bc0103ee0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc019bea0 con 0x7f9bc0103ee0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc019c340 con 0x7f9bc0103ee0 2024-09-13T17:17:04.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bb003a5b0 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9bb0039040 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.151+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9bb0044b30 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.152+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc0195180 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.154+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f9bb003f020 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.215+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9bc0073df0 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.215+0000 7f9bb77fe640 1 -- 172.21.15.44:0/4019155975 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9bb0037070 con 0x7f9bc0103ee0 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 msgr2=0x7f9bc019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 --2- 172.21.15.44:0/4019155975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc019b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00097f0 tx=0x7f9bb0030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 shutdown_connections 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 --2- 172.21.15.44:0/4019155975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bc0103ee0 0x7f9bc019b760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 >> 172.21.15.44:0/4019155975 conn(0x7f9bc00ffcf0 msgr2=0x7f9bc0192350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 shutdown_connections 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:03.216+0000 7f9bc4f7a640 1 -- 172.21.15.44:0/4019155975 wait complete. 2024-09-13T17:17:04.033 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (1/15)... 2024-09-13T17:17:04.356 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:03.896+0000 7fb7e5c2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:17:05.106 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:04.800+0000 7fb7e5c2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:17:05.107 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:04.868+0000 7fb7e5c2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:17:05.882 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:05.616+0000 7fb7e5c2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:17:05.882 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:05.734+0000 7fb7e5c2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:17:05.882 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:05.880+0000 7fb7e5c2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:17:06.143 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:05.943+0000 7fb7e5c2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:17:06.143 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.098+0000 7fb7e5c2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:17:06.398 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.169+0000 7fb7e5c2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:17:06.398 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.230+0000 7fb7e5c2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:17:06.398 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.396+0000 7fb7e5c2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:17:06.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4108784016' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T17:17:06.686 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.466+0000 7fb7e5c2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:17:06.687 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.537+0000 7fb7e5c2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:17:06.687 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.624+0000 7fb7e5c2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:17:06.687 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.685+0000 7fb7e5c2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:17:07.106 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:06.746+0000 7fb7e5c2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T17:17:07.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T17:17:07.126 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:07.126 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T17:17:07.126 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T17:17:07.126 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T17:17:07.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T17:17:07.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T17:17:07.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.135 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T17:17:07.136 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T17:16:55.306262+0000", 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa47e8640 1 Processor -- start 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa47e8640 1 -- start start 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa47e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c108520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa47e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c079e50 con 0x7fef9c106130 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa2584640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c108520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa2584640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c108520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52228/0 (socket says 172.21.15.44:52228) 2024-09-13T17:17:07.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.436+0000 7fefa2584640 1 -- 172.21.15.44:0/397538560 learned_addr learned my addr 172.21.15.44:0/397538560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.437+0000 7fefa2584640 1 -- 172.21.15.44:0/397538560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c108a60 con 0x7fef9c106130 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.437+0000 7fefa2584640 1 --2- 172.21.15.44:0/397538560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c108520 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef98009820 tx=0x7fef9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cc38a13a045fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa1582640 1 -- 172.21.15.44:0/397538560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef9802ae70 con 0x7fef9c106130 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa1582640 1 -- 172.21.15.44:0/397538560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fef980328b0 con 0x7fef9c106130 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa1582640 1 -- 172.21.15.44:0/397538560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef98030850 con 0x7fef9c106130 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa47e8640 1 -- 172.21.15.44:0/397538560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 msgr2=0x7fef9c108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa47e8640 1 --2- 172.21.15.44:0/397538560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c108520 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef98009820 tx=0x7fef9802a270 comp rx=0 tx=0).stop 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa47e8640 1 -- 172.21.15.44:0/397538560 shutdown_connections 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa47e8640 1 --2- 172.21.15.44:0/397538560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c108520 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa47e8640 1 -- 172.21.15.44:0/397538560 >> 172.21.15.44:0/397538560 conn(0x7fef9c0ffcf0 msgr2=0x7fef9c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.438+0000 7fefa47e8640 1 -- 172.21.15.44:0/397538560 shutdown_connections 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa47e8640 1 -- 172.21.15.44:0/397538560 wait complete. 2024-09-13T17:17:07.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa47e8640 1 Processor -- start 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa47e8640 1 -- start start 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa47e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c1059b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa47e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c105ef0 con 0x7fef9c106130 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa2584640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c1059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa2584640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c1059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52244/0 (socket says 172.21.15.44:52244) 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.439+0000 7fefa2584640 1 -- 172.21.15.44:0/4108784016 learned_addr learned my addr 172.21.15.44:0/4108784016 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fefa2584640 1 -- 172.21.15.44:0/4108784016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980094d0 con 0x7fef9c106130 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fefa2584640 1 --2- 172.21.15.44:0/4108784016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c1059b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef98030560 tx=0x7fef98030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef98032d60 con 0x7fef9c106130 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fef9803b8a0 con 0x7fef9c106130 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c1040e0 con 0x7fef9c106130 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef9803a5b0 con 0x7fef9c106130 2024-09-13T17:17:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c104580 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fef98039040 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.440+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fef98044bb0 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.441+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c073df0 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.443+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fef9803f020 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.504+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fef9c109a70 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.505+0000 7fef8f7fe640 1 -- 172.21.15.44:0/4108784016 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fef98037070 con 0x7fef9c106130 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 msgr2=0x7fef9c1059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 --2- 172.21.15.44:0/4108784016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c1059b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef98030560 tx=0x7fef98030d90 comp rx=0 tx=0).stop 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 shutdown_connections 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 --2- 172.21.15.44:0/4108784016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9c106130 0x7fef9c1059b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 >> 172.21.15.44:0/4108784016 conn(0x7fef9c0ffcf0 msgr2=0x7fef9c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 shutdown_connections 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:06.506+0000 7fefa47e8640 1 -- 172.21.15.44:0/4108784016 wait complete. 2024-09-13T17:17:07.140 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (2/15)... 2024-09-13T17:17:07.856 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:07.558+0000 7fb7e5c2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: Activating manager daemon x 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: mgrmap e2: x(active, starting, since 0.00316745s) 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T17:17:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: Manager daemon x is now available 2024-09-13T17:17:07.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:17:07.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:17:07.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' 2024-09-13T17:17:07.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' 2024-09-13T17:17:07.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:07 smithi044 ceph-mon[31175]: from='mgr.14100 172.21.15.44:0/1089873307' entity='mgr.x' 2024-09-13T17:17:09.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:09 smithi044 ceph-mon[31175]: mgrmap e3: x(active, since 1.00763s) 2024-09-13T17:17:10.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:17:10.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:10.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T17:17:10.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T17:17:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T17:17:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T17:17:10.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T17:16:55.306262+0000", 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb10dbe2640 1 Processor -- start 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb10dbe2640 1 -- start start 2024-09-13T17:17:10.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb10dbe2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb108105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb10dbe2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb108106010 con 0x7fb108105640 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb1077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb108105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb1077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb108105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59616/0 (socket says 172.21.15.44:59616) 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.515+0000 7fb1077fe640 1 -- 172.21.15.44:0/3814060231 learned_addr learned my addr 172.21.15.44:0/3814060231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.516+0000 7fb1077fe640 1 -- 172.21.15.44:0/3814060231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108106840 con 0x7fb108105640 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.516+0000 7fb1077fe640 1 --2- 172.21.15.44:0/3814060231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb108105a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009820 tx=0x7fb0f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f716e9452601ce2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb1067fc640 1 -- 172.21.15.44:0/3814060231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f802ae70 con 0x7fb108105640 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb1067fc640 1 -- 172.21.15.44:0/3814060231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb0f80328b0 con 0x7fb108105640 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb1067fc640 1 -- 172.21.15.44:0/3814060231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f8030850 con 0x7fb108105640 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3814060231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 msgr2=0x7fb108105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 --2- 172.21.15.44:0/3814060231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb108105a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009820 tx=0x7fb0f802a270 comp rx=0 tx=0).stop 2024-09-13T17:17:10.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3814060231 shutdown_connections 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 --2- 172.21.15.44:0/3814060231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb108105a40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3814060231 >> 172.21.15.44:0/3814060231 conn(0x7fb108100df0 msgr2=0x7fb108103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3814060231 shutdown_connections 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.517+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3814060231 wait complete. 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.518+0000 7fb10dbe2640 1 Processor -- start 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.518+0000 7fb10dbe2640 1 -- start start 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.518+0000 7fb10dbe2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb10807bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.518+0000 7fb10dbe2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10807c080 con 0x7fb108105640 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb1077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb10807bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb1077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb10807bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59624/0 (socket says 172.21.15.44:59624) 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb1077fe640 1 -- 172.21.15.44:0/3112137373 learned_addr learned my addr 172.21.15.44:0/3112137373 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb1077fe640 1 -- 172.21.15.44:0/3112137373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f80094d0 con 0x7fb108105640 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb1077fe640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb10807bb40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8030560 tx=0x7fb0f8030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:10.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f8032d60 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb0f803b8a0 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f803a5f0 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1081076c0 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.519+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10807a510 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.520+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7fb0f8039040 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.520+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb108105ac0 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.521+0000 7fb104ff9640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7fb0f00385c0 0x7fb0f003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.521+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb0f806b240 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.521+0000 7fb106ffd640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7fb0f00385c0 0x7fb0f003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.521+0000 7fb106ffd640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7fb0f00385c0 0x7fb0f003aa80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0098c0 tx=0x7fb0fc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.526+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb0f80319f0 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.567+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fb0f8038d00 con 0x7fb108105640 2024-09-13T17:17:10.354 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.769+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb108073df0 con 0x7fb108105640 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.770+0000 7fb104ff9640 1 -- 172.21.15.44:0/3112137373 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1237 (secure 0 0 0) 0x7fb0f8037090 con 0x7fb108105640 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7fb0f00385c0 msgr2=0x7fb0f003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7fb0f00385c0 0x7fb0f003aa80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0098c0 tx=0x7fb0fc006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 msgr2=0x7fb10807bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb10807bb40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8030560 tx=0x7fb0f8030d90 comp rx=0 tx=0).stop 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 shutdown_connections 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7fb0f00385c0 0x7fb0f003aa80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 --2- 172.21.15.44:0/3112137373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb108105640 0x7fb10807bb40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 >> 172.21.15.44:0/3112137373 conn(0x7fb108100df0 msgr2=0x7fb1081019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 shutdown_connections 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:09.772+0000 7fb10dbe2640 1 -- 172.21.15.44:0/3112137373 wait complete. 2024-09-13T17:17:10.355 INFO:teuthology.orchestra.run.smithi044.stdout:mgr is available 2024-09-13T17:17:10.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:10 smithi044 ceph-mon[31175]: mgrmap e4: x(active, since 2s) 2024-09-13T17:17:10.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:10 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3112137373' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:17:11.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cc57eb640 1 Processor -- start 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cc57eb640 1 -- start start 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cc57eb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0108520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cc57eb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0079e50 con 0x7f9cc0106130 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0108520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:11.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0108520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59638/0 (socket says 172.21.15.44:59638) 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.736+0000 7f9cbeffd640 1 -- 172.21.15.44:0/1686580708 learned_addr learned my addr 172.21.15.44:0/1686580708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.737+0000 7f9cbeffd640 1 -- 172.21.15.44:0/1686580708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0108a60 con 0x7f9cc0106130 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.737+0000 7f9cbeffd640 1 --2- 172.21.15.44:0/1686580708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0108520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009820 tx=0x7f9cb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8ad999037c8f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cbdffb640 1 -- 172.21.15.44:0/1686580708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb002ae70 con 0x7f9cc0106130 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cbdffb640 1 -- 172.21.15.44:0/1686580708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9cb00328b0 con 0x7f9cc0106130 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cbdffb640 1 -- 172.21.15.44:0/1686580708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb0030850 con 0x7f9cc0106130 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 -- 172.21.15.44:0/1686580708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 msgr2=0x7f9cc0108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 --2- 172.21.15.44:0/1686580708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0108520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009820 tx=0x7f9cb002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 -- 172.21.15.44:0/1686580708 shutdown_connections 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 --2- 172.21.15.44:0/1686580708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0108520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 -- 172.21.15.44:0/1686580708 >> 172.21.15.44:0/1686580708 conn(0x7f9cc00ffcf0 msgr2=0x7f9cc0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:11.503 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 -- 172.21.15.44:0/1686580708 shutdown_connections 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.738+0000 7f9cc57eb640 1 -- 172.21.15.44:0/1686580708 wait complete. 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cc57eb640 1 Processor -- start 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cc57eb640 1 -- start start 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cc57eb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cc57eb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0199a90 con 0x7f9cc0106130 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59650/0 (socket says 172.21.15.44:59650) 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.739+0000 7f9cbeffd640 1 -- 172.21.15.44:0/895386967 learned_addr learned my addr 172.21.15.44:0/895386967 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9cbeffd640 1 -- 172.21.15.44:0/895386967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00094d0 con 0x7f9cc0106130 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9cbeffd640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0199550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00097f0 tx=0x7f9cb0030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb0032c30 con 0x7f9cc0106130 2024-09-13T17:17:11.504 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9cb0032d90 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc0199c90 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb003a5b0 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.740+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc019a130 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.741+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f9cb0039040 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.741+0000 7f9ca7fff640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f9ca80386e0 0x7f9ca803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.741+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9cb006b4a0 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.741+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc0073df0 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.741+0000 7f9cbe7fc640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f9ca80386e0 0x7f9ca803aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.742+0000 7f9cbe7fc640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f9ca80386e0 0x7f9ca803aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40098c0 tx=0x7f9cb4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.746+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9cb0037070 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.917+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9cc019dbf0 con 0x7f9cc0106130 2024-09-13T17:17:11.505 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.918+0000 7f9ca7fff640 1 -- 172.21.15.44:0/895386967 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9cb0068030 con 0x7f9cc0106130 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.920+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f9ca80386e0 msgr2=0x7f9ca803aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.920+0000 7f9cc57eb640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f9ca80386e0 0x7f9ca803aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40098c0 tx=0x7f9cb4006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 msgr2=0x7f9cc0199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0199550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00097f0 tx=0x7f9cb0030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 shutdown_connections 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f9ca80386e0 0x7f9ca803aba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 --2- 172.21.15.44:0/895386967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0106130 0x7f9cc0199550 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 >> 172.21.15.44:0/895386967 conn(0x7f9cc00ffcf0 msgr2=0x7f9cc0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 shutdown_connections 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:10.921+0000 7f9cc57eb640 1 -- 172.21.15.44:0/895386967 wait complete. 2024-09-13T17:17:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling cephadm module... 2024-09-13T17:17:11.835 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:11 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/895386967' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-13T17:17:12.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/893398424' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-13T17:17:12.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ignoring --setuser ceph since I am not root 2024-09-13T17:17:12.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ignoring --setgroup ceph since I am not root 2024-09-13T17:17:12.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:12.714+0000 7f0b7168c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:17:12.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:12.773+0000 7f0b7168c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:17:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76cb5c8640 1 Processor -- start 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76cb5c8640 1 -- start start 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76cb5c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c41084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76cb5c8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4079ed0 con 0x7f76c41060b0 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76c9364640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c41084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76c9364640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c41084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59652/0 (socket says 172.21.15.44:59652) 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.920+0000 7f76c9364640 1 -- 172.21.15.44:0/2192050432 learned_addr learned my addr 172.21.15.44:0/2192050432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.921+0000 7f76c9364640 1 -- 172.21.15.44:0/2192050432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c41089e0 con 0x7f76c41060b0 2024-09-13T17:17:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.921+0000 7f76c9364640 1 --2- 172.21.15.44:0/2192050432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c41084a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009820 tx=0x7f76c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e83fc381c435a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76b7fff640 1 -- 172.21.15.44:0/2192050432 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76c002ae70 con 0x7f76c41060b0 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76b7fff640 1 -- 172.21.15.44:0/2192050432 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f76c00328b0 con 0x7f76c41060b0 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76b7fff640 1 -- 172.21.15.44:0/2192050432 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76c0030850 con 0x7f76c41060b0 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 -- 172.21.15.44:0/2192050432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 msgr2=0x7f76c41084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 --2- 172.21.15.44:0/2192050432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c41084a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009820 tx=0x7f76c002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 -- 172.21.15.44:0/2192050432 shutdown_connections 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 --2- 172.21.15.44:0/2192050432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c41084a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 -- 172.21.15.44:0/2192050432 >> 172.21.15.44:0/2192050432 conn(0x7f76c40ffcf0 msgr2=0x7f76c4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 -- 172.21.15.44:0/2192050432 shutdown_connections 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.922+0000 7f76cb5c8640 1 -- 172.21.15.44:0/2192050432 wait complete. 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76cb5c8640 1 Processor -- start 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76cb5c8640 1 -- start start 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76cb5c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76cb5c8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4199a80 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76c9364640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76c9364640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c4199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59662/0 (socket says 172.21.15.44:59662) 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.923+0000 7f76c9364640 1 -- 172.21.15.44:0/893398424 learned_addr learned my addr 172.21.15.44:0/893398424 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76c9364640 1 -- 172.21.15.44:0/893398424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00094d0 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76c9364640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c4199540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76c00097f0 tx=0x7f76c0030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76c0032c30 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f76c0032d90 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c4199c80 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76c003a5b0 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.924+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c419a120 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.925+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f76c0039040 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.925+0000 7f76b67fc640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f76b00386e0 0x7f76b003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.925+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c4073df0 con 0x7f76c41060b0 2024-09-13T17:17:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.925+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f76c006b600 con 0x7f76c41060b0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.925+0000 7f76c8b63640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f76b00386e0 0x7f76b003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.926+0000 7f76c8b63640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f76b00386e0 0x7f76b003aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76b80098c0 tx=0x7f76b8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:11.930+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f76c003f070 con 0x7f76c41060b0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.149+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f76c41097a0 con 0x7f76c41060b0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.571+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f76c00313a0 con 0x7f76c41060b0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.572+0000 7f76b67fc640 1 -- 172.21.15.44:0/893398424 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f76c005a2b0 con 0x7f76c41060b0 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f76b00386e0 msgr2=0x7f76b003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f76b00386e0 0x7f76b003aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76b80098c0 tx=0x7f76b8006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 msgr2=0x7f76c4199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c4199540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76c00097f0 tx=0x7f76c0030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 shutdown_connections 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f76b00386e0 0x7f76b003aba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 --2- 172.21.15.44:0/893398424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c41060b0 0x7f76c4199540 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 >> 172.21.15.44:0/893398424 conn(0x7f76c40ffcf0 msgr2=0x7f76c41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 shutdown_connections 2024-09-13T17:17:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:12.575+0000 7f76cb5c8640 1 -- 172.21.15.44:0/893398424 wait complete. 2024-09-13T17:17:13.151 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:12.929+0000 7f0b7168c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:17:13.151 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:12.997+0000 7f0b7168c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:17:13.151 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:13.150+0000 7f0b7168c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:17:13.407 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:13.302+0000 7f0b7168c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:17:13.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/893398424' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-13T17:17:13.659 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-mon[31175]: mgrmap e5: x(active, since 5s) 2024-09-13T17:17:13.922 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:13.657+0000 7f0b7168c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:17:13.922 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:13.722+0000 7f0b7168c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:17:13.923 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:13.921+0000 7f0b7168c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.586+0000 7eff0347d640 1 Processor -- start 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff0347d640 1 -- start start 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff0347d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc104320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:14.446 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff0347d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc079e50 con 0x7efefc103f20 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff01219640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff01219640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc104320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59696/0 (socket says 172.21.15.44:59696) 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff01219640 1 -- 172.21.15.44:0/1918168796 learned_addr learned my addr 172.21.15.44:0/1918168796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.587+0000 7eff01219640 1 -- 172.21.15.44:0/1918168796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc104860 con 0x7efefc103f20 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.588+0000 7eff01219640 1 --2- 172.21.15.44:0/1918168796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc104320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efeec009820 tx=0x7efeec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d585a6169ca2f183 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.588+0000 7efef3fff640 1 -- 172.21.15.44:0/1918168796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec02ae70 con 0x7efefc103f20 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.588+0000 7efef3fff640 1 -- 172.21.15.44:0/1918168796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efeec0328b0 con 0x7efefc103f20 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.588+0000 7efef3fff640 1 -- 172.21.15.44:0/1918168796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec030850 con 0x7efefc103f20 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 -- 172.21.15.44:0/1918168796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 msgr2=0x7efefc104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 --2- 172.21.15.44:0/1918168796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc104320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efeec009820 tx=0x7efeec02a270 comp rx=0 tx=0).stop 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 -- 172.21.15.44:0/1918168796 shutdown_connections 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 --2- 172.21.15.44:0/1918168796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc104320 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:14.447 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 -- 172.21.15.44:0/1918168796 >> 172.21.15.44:0/1918168796 conn(0x7efefc0ffcf0 msgr2=0x7efefc102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 -- 172.21.15.44:0/1918168796 shutdown_connections 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 -- 172.21.15.44:0/1918168796 wait complete. 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 Processor -- start 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.589+0000 7eff0347d640 1 -- start start 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff0347d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff0347d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc19bcc0 con 0x7efefc103f20 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff01219640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff01219640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59700/0 (socket says 172.21.15.44:59700) 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff01219640 1 -- 172.21.15.44:0/1175885717 learned_addr learned my addr 172.21.15.44:0/1175885717 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff01219640 1 -- 172.21.15.44:0/1175885717 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0094d0 con 0x7efefc103f20 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff01219640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc19b780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efeec030560 tx=0x7efeec030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec032d60 con 0x7efefc103f20 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc19bec0 con 0x7efefc103f20 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efeec03b8a0 con 0x7efefc103f20 2024-09-13T17:17:14.448 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.590+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec03a5b0 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.591+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc19c360 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.591+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7efeec039040 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.592+0000 7efef27fc640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 0x7efee803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.592+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efeec06b5b0 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.592+0000 7eff00a18640 1 -- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 msgr2=0x7efee803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/3711196986 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.592+0000 7eff00a18640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 0x7efee803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.592+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc105eb0 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.601+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efeec037020 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.793+0000 7eff00a18640 1 -- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 msgr2=0x7efee803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/3711196986 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.793+0000 7eff00a18640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 0x7efee803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.846+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7efefc0008d0 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.846+0000 7efef27fc640 1 -- 172.21.15.44:0/1175885717 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7efeec03f070 con 0x7efefc103f20 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.849+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 msgr2=0x7efee803aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:17:14.449 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.849+0000 7eff0347d640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 0x7efee803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.849+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 msgr2=0x7efefc19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.849+0000 7eff0347d640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc19b780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efeec030560 tx=0x7efeec030d90 comp rx=0 tx=0).stop 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.850+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 shutdown_connections 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.850+0000 7eff0347d640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7efee80386e0 0x7efee803aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.850+0000 7eff0347d640 1 --2- 172.21.15.44:0/1175885717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc103f20 0x7efefc19b780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.850+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 >> 172.21.15.44:0/1175885717 conn(0x7efefc0ffcf0 msgr2=0x7efefc192340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.850+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 shutdown_connections 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:13.850+0000 7eff0347d640 1 -- 172.21.15.44:0/1175885717 wait complete. 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2024-09-13T17:17:14.450 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 5... 2024-09-13T17:17:14.799 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:14 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1175885717' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-13T17:17:15.085 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:14.845+0000 7f0b7168c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:17:15.085 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:14.916+0000 7f0b7168c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:17:15.784 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:15.664+0000 7f0b7168c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:17:16.106 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:15.783+0000 7f0b7168c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:17:16.106 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:15.930+0000 7f0b7168c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:17:16.106 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:15.993+0000 7f0b7168c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:17:16.442 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.146+0000 7f0b7168c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:17:16.442 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.216+0000 7f0b7168c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:17:16.442 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.278+0000 7f0b7168c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:17:16.442 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.440+0000 7f0b7168c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:17:16.725 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.508+0000 7f0b7168c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:17:16.725 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.577+0000 7f0b7168c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:17:16.725 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.663+0000 7f0b7168c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:17:16.726 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.724+0000 7f0b7168c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:17:17.106 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:16.785+0000 7f0b7168c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:17:17.856 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:17.598+0000 7f0b7168c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: Active manager daemon x restarted 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: Activating manager daemon x 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: osdmap e2: 0 total, 0 up, 0 in 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: mgrmap e6: x(active, starting, since 0.00372899s) 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: Manager daemon x is now available 2024-09-13T17:17:17.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: Found migration_current of "None". Setting to last migration. 2024-09-13T17:17:17.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:17.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:17.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:17.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:17.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:17:17.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:17 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.877+0000 7f4692957640 1 Processor -- start 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.877+0000 7f4692957640 1 -- start start 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.878+0000 7f4692957640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c1084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:19.236 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.878+0000 7f4692957640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c079e90 con 0x7f468c1060f0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.878+0000 7f468bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c1084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.878+0000 7f468bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c1084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59702/0 (socket says 172.21.15.44:59702) 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.878+0000 7f468bfff640 1 -- 172.21.15.44:0/4180415659 learned_addr learned my addr 172.21.15.44:0/4180415659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.878+0000 7f468bfff640 1 -- 172.21.15.44:0/4180415659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c108a20 con 0x7f468c1060f0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.879+0000 7f468bfff640 1 --2- 172.21.15.44:0/4180415659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c1084e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f467c009820 tx=0x7f467c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a52f487ef21b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.879+0000 7f468affd640 1 -- 172.21.15.44:0/4180415659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c02ae70 con 0x7f468c1060f0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.879+0000 7f468affd640 1 -- 172.21.15.44:0/4180415659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f467c0328b0 con 0x7f468c1060f0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.879+0000 7f468affd640 1 -- 172.21.15.44:0/4180415659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c030850 con 0x7f468c1060f0 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 -- 172.21.15.44:0/4180415659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 msgr2=0x7f468c1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 --2- 172.21.15.44:0/4180415659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c1084e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f467c009820 tx=0x7f467c02a270 comp rx=0 tx=0).stop 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 -- 172.21.15.44:0/4180415659 shutdown_connections 2024-09-13T17:17:19.237 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 --2- 172.21.15.44:0/4180415659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c1084e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 -- 172.21.15.44:0/4180415659 >> 172.21.15.44:0/4180415659 conn(0x7f468c0ffcf0 msgr2=0x7f468c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 -- 172.21.15.44:0/4180415659 shutdown_connections 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 -- 172.21.15.44:0/4180415659 wait complete. 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 Processor -- start 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.880+0000 7f4692957640 1 -- start start 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f4692957640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f4692957640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c199ad0 con 0x7f468c1060f0 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f468bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f468bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59712/0 (socket says 172.21.15.44:59712) 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f468bfff640 1 -- 172.21.15.44:0/1419575156 learned_addr learned my addr 172.21.15.44:0/1419575156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f468bfff640 1 -- 172.21.15.44:0/1419575156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c0094d0 con 0x7f468c1060f0 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f468bfff640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c199590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097f0 tx=0x7f467c030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:19.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c032c30 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f467c032d90 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c199cd0 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c03a5b0 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.881+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c19a170 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.882+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f467c039040 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.883+0000 7f46897fa640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.883+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 --> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f467403b2b0 con 0x7f46740386e0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.883+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f467c06b540 con 0x7f468c1060f0 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.883+0000 7f468b7fe640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 msgr2=0x7f467403aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/3711196986 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:14.883+0000 7f468b7fe640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:15.084+0000 7f468b7fe640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 msgr2=0x7f467403aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/3711196986 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:15.084+0000 7f468b7fe640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:17:19.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:15.485+0000 7f468b7fe640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 msgr2=0x7f467403aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/3711196986 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:15.485+0000 7f468b7fe640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:16.286+0000 7f468b7fe640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 msgr2=0x7f467403aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/3711196986 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:16.286+0000 7f468b7fe640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:17.602+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f467c06a6c0 con 0x7f468c1060f0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:17.602+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 msgr2=0x7f467403aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:17.602+0000 7f46897fa640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.602+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f467c06d0f0 con 0x7f468c1060f0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.602+0000 7f46897fa640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f467403bfc0 0x7f467403e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.602+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f467403b2b0 con 0x7f467403bfc0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.603+0000 7f468b7fe640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f467403bfc0 0x7f467403e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.604+0000 7f468b7fe640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f467403bfc0 0x7f467403e3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4680003980 tx=0x7f4680006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.604+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f467403b2b0 con 0x7f467403bfc0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f468c073df0 con 0x7f467403bfc0 2024-09-13T17:17:19.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f46897fa640 1 -- 172.21.15.44:0/1419575156 <== mgr.14118 v2:172.21.15.44:6800/2265862199 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f468c073df0 con 0x7f467403bfc0 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f467403bfc0 msgr2=0x7f467403e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f467403bfc0 0x7f467403e3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4680003980 tx=0x7f4680006e10 comp rx=0 tx=0).stop 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 msgr2=0x7f468c199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c199590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097f0 tx=0x7f467c030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 shutdown_connections 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f467403bfc0 0x7f467403e3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:6800/3711196986,v1:172.21.15.44:6801/3711196986] conn(0x7f46740386e0 0x7f467403aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 --2- 172.21.15.44:0/1419575156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c1060f0 0x7f468c199590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:19.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 >> 172.21.15.44:0/1419575156 conn(0x7f468c0ffcf0 msgr2=0x7f468c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:19.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 shutdown_connections 2024-09-13T17:17:19.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:18.610+0000 7f4692957640 1 -- 172.21.15.44:0/1419575156 wait complete. 2024-09-13T17:17:19.242 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 5 is available 2024-09-13T17:17:19.242 INFO:teuthology.orchestra.run.smithi044.stdout:Setting orchestrator backend to cephadm... 2024-09-13T17:17:19.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:19 smithi044 ceph-mon[31175]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T17:17:19.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:19 smithi044 ceph-mon[31175]: mgrmap e7: x(active, since 1.00511s) 2024-09-13T17:17:19.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:19 smithi044 ceph-mon[31175]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T17:17:19.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:19 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:19.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:19 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:20.382 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc346227640 1 Processor -- start 2024-09-13T17:17:20.382 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc346227640 1 -- start start 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc346227640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc340104320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc346227640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340079e50 con 0x7fc340103f20 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc33f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc340104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc33f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc340104320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45590/0 (socket says 172.21.15.44:45590) 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc33f7fe640 1 -- 172.21.15.44:0/4240425888 learned_addr learned my addr 172.21.15.44:0/4240425888 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.575+0000 7fc33f7fe640 1 -- 172.21.15.44:0/4240425888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340104860 con 0x7fc340103f20 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.576+0000 7fc33f7fe640 1 --2- 172.21.15.44:0/4240425888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc340104320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc330009820 tx=0x7fc33002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f081983cd6610178 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.576+0000 7fc33e7fc640 1 -- 172.21.15.44:0/4240425888 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc33002ae70 con 0x7fc340103f20 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.576+0000 7fc33e7fc640 1 -- 172.21.15.44:0/4240425888 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc3300328b0 con 0x7fc340103f20 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.576+0000 7fc33e7fc640 1 -- 172.21.15.44:0/4240425888 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc330030850 con 0x7fc340103f20 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 -- 172.21.15.44:0/4240425888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 msgr2=0x7fc340104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 --2- 172.21.15.44:0/4240425888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc340104320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc330009820 tx=0x7fc33002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 -- 172.21.15.44:0/4240425888 shutdown_connections 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 --2- 172.21.15.44:0/4240425888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc340104320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 -- 172.21.15.44:0/4240425888 >> 172.21.15.44:0/4240425888 conn(0x7fc3400ffcf0 msgr2=0x7fc340102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 -- 172.21.15.44:0/4240425888 shutdown_connections 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 -- 172.21.15.44:0/4240425888 wait complete. 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.577+0000 7fc346227640 1 Processor -- start 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc346227640 1 -- start start 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc346227640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc34019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc346227640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34019bca0 con 0x7fc340103f20 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc33f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc34019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc33f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc34019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45602/0 (socket says 172.21.15.44:45602) 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc33f7fe640 1 -- 172.21.15.44:0/2843222383 learned_addr learned my addr 172.21.15.44:0/2843222383 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc33f7fe640 1 -- 172.21.15.44:0/2843222383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3300094d0 con 0x7fc340103f20 2024-09-13T17:17:20.384 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.578+0000 7fc33f7fe640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc34019b760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc3300097f0 tx=0x7fc330030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.579+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc330032c30 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.579+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc330032d90 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.579+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34019bea0 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.579+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc33003a5b0 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.579+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34019c340 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.580+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fc330039040 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.580+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3401951c0 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.580+0000 7fc33cff9640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fc3280385c0 0x7fc32803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.580+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc33006b850 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.580+0000 7fc33effd640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fc3280385c0 0x7fc32803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.581+0000 7fc33effd640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fc3280385c0 0x7fc32803aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc3340098c0 tx=0x7fc334006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.585+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc33004b6c0 con 0x7fc340103f20 2024-09-13T17:17:20.385 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.697+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fc330044400 con 0x7fc340103f20 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.771+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3400608f0 con 0x7fc3280385c0 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.780+0000 7fc33cff9640 1 -- 172.21.15.44:0/2843222383 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc3400608f0 con 0x7fc3280385c0 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fc3280385c0 msgr2=0x7fc32803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fc3280385c0 0x7fc32803aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc3340098c0 tx=0x7fc334006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 msgr2=0x7fc34019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc34019b760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc3300097f0 tx=0x7fc330030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 shutdown_connections 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fc3280385c0 0x7fc32803aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 --2- 172.21.15.44:0/2843222383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc340103f20 0x7fc34019b760 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 >> 172.21.15.44:0/2843222383 conn(0x7fc3400ffcf0 msgr2=0x7fc340192390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 shutdown_connections 2024-09-13T17:17:20.386 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:19.782+0000 7fc346227640 1 -- 172.21.15.44:0/2843222383 wait complete. 2024-09-13T17:17:20.697 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:20.448+0000 7f0adeb77640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:17:17:20] ENGINE Error in HTTPServer.serve 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: Traceback (most recent call last): 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._connections.run(self.expiration_interval) 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._run(expiration_interval) 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: s = self.context.wrap_socket( 2024-09-13T17:17:20.698 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: return self.sslsocket_class._create( 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self.do_handshake() 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._sslobj.do_handshake() 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:17:20.699 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: mgrmap e8: x(active, since 2s) 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:20] ENGINE Bus STARTING 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:20] ENGINE Serving on https://172.21.15.44:7150 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:20] ENGINE Error in HTTPServer.serve 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout: Traceback (most recent call last): 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._connections.run(self.expiration_interval) 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:17:20.964 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._run(expiration_interval) 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: s = self.context.wrap_socket( 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: return self.sslsocket_class._create( 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: self.do_handshake() 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:17:20.965 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._sslobj.do_handshake() 2024-09-13T17:17:20.966 INFO:journalctl@ceph.mon.a.smithi044.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:17:20.966 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:20] ENGINE Serving on http://172.21.15.44:8765 2024-09-13T17:17:20.966 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:20] ENGINE Bus STARTED 2024-09-13T17:17:20.966 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:20 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:21.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-13T17:17:21.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec26366640 1 Processor -- start 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec26366640 1 -- start start 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec26366640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec20104940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec26366640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20104f10 con 0x7fec20104540 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec20104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec20104940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45610/0 (socket says 172.21.15.44:45610) 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.781+0000 7fec1ffff640 1 -- 172.21.15.44:0/3156856311 learned_addr learned my addr 172.21.15.44:0/3156856311 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.782+0000 7fec1ffff640 1 -- 172.21.15.44:0/3156856311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec20105740 con 0x7fec20104540 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.782+0000 7fec1ffff640 1 --2- 172.21.15.44:0/3156856311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec20104940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec1000bac0 tx=0x7fec1002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3145b25ae894e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec1effd640 1 -- 172.21.15.44:0/3156856311 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec1002f070 con 0x7fec20104540 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec1effd640 1 -- 172.21.15.44:0/3156856311 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fec10032b40 con 0x7fec20104540 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec1effd640 1 -- 172.21.15.44:0/3156856311 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec10033ba0 con 0x7fec20104540 2024-09-13T17:17:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 -- 172.21.15.44:0/3156856311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 msgr2=0x7fec20104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 --2- 172.21.15.44:0/3156856311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec20104940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec1000bac0 tx=0x7fec1002c920 comp rx=0 tx=0).stop 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 -- 172.21.15.44:0/3156856311 shutdown_connections 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 --2- 172.21.15.44:0/3156856311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec20104940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 -- 172.21.15.44:0/3156856311 >> 172.21.15.44:0/3156856311 conn(0x7fec200ffcf0 msgr2=0x7fec20102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 -- 172.21.15.44:0/3156856311 shutdown_connections 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.783+0000 7fec26366640 1 -- 172.21.15.44:0/3156856311 wait complete. 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec26366640 1 Processor -- start 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec26366640 1 -- start start 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec26366640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec201973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec26366640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20197900 con 0x7fec20104540 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec201973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.560 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec201973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45618/0 (socket says 172.21.15.44:45618) 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec1ffff640 1 -- 172.21.15.44:0/1470855523 learned_addr learned my addr 172.21.15.44:0/1470855523 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.784+0000 7fec1ffff640 1 -- 172.21.15.44:0/1470855523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1000b770 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.785+0000 7fec1ffff640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec201973c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec1000bbe0 tx=0x7fec100327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.785+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec1003c600 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.785+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fec1003cbd0 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.785+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec20197b00 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.785+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec1003b9f0 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.785+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec20198000 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.786+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fec1003bb50 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.786+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec20078ff0 con 0x7fec20104540 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.786+0000 7fec1d7fa640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fec08038730 0x7fec0803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.786+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec1006ea80 con 0x7fec20104540 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.786+0000 7fec1f7fe640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fec08038730 0x7fec0803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.787+0000 7fec1f7fe640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fec08038730 0x7fec0803abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec140098c0 tx=0x7fec14006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.791+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fec1004fce0 con 0x7fec20104540 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.962+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fec200608f0 con 0x7fec08038730 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.963+0000 7fec1d7fa640 1 -- 172.21.15.44:0/1470855523 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fec200608f0 con 0x7fec08038730 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fec08038730 msgr2=0x7fec0803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fec08038730 0x7fec0803abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec140098c0 tx=0x7fec14006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 msgr2=0x7fec201973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec201973c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec1000bbe0 tx=0x7fec100327d0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 shutdown_connections 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fec08038730 0x7fec0803abf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 --2- 172.21.15.44:0/1470855523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20104540 0x7fec201973c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 >> 172.21.15.44:0/1470855523 conn(0x7fec200ffcf0 msgr2=0x7fec2018dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:21.563 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 shutdown_connections 2024-09-13T17:17:21.563 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:20.965+0000 7fec26366640 1 -- 172.21.15.44:0/1470855523 wait complete. 2024-09-13T17:17:21.563 INFO:teuthology.orchestra.run.smithi044.stdout:Generating ssh key... 2024-09-13T17:17:21.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:21 smithi044 ceph-mon[31175]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: Generating public/private rsa key pair. 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: Your identification has been saved in /tmp/tmpud26o0zf/key 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: Your public key has been saved in /tmp/tmpud26o0zf/key.pub 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: The key fingerprint is: 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: SHA256:ooJqqu6JzLovVCAQvnTMXhz1coz18Ars67Y77vYwfbg ceph-ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: The key's randomart image is: 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: +---[RSA 3072]----+ 2024-09-13T17:17:22.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |+. ... o | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |o.o . o = + | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |.o.+ o = + o | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |. +.. . + . | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: | ... ..S. | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: | o . .o . | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |o . . + o . | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |*... .+o o | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: |#X. =*=E | 2024-09-13T17:17:22.858 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: +----[SHA256]-----+ 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff138f00640 1 Processor -- start 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff138f00640 1 -- start start 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff138f00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff138f00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff134106010 con 0x7ff134105640 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff13259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff13259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45632/0 (socket says 172.21.15.44:45632) 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.010+0000 7ff13259c640 1 -- 172.21.15.44:0/2599164621 learned_addr learned my addr 172.21.15.44:0/2599164621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.011+0000 7ff13259c640 1 -- 172.21.15.44:0/2599164621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134106840 con 0x7ff134105640 2024-09-13T17:17:23.130 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.011+0000 7ff13259c640 1 --2- 172.21.15.44:0/2599164621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134105a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff124009820 tx=0x7ff12402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1364773e04a1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff13159a640 1 -- 172.21.15.44:0/2599164621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff12402ae70 con 0x7ff134105640 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff13159a640 1 -- 172.21.15.44:0/2599164621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff1240328b0 con 0x7ff134105640 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff13159a640 1 -- 172.21.15.44:0/2599164621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff124030850 con 0x7ff134105640 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 -- 172.21.15.44:0/2599164621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 msgr2=0x7ff134105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 --2- 172.21.15.44:0/2599164621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134105a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff124009820 tx=0x7ff12402a270 comp rx=0 tx=0).stop 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 -- 172.21.15.44:0/2599164621 shutdown_connections 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 --2- 172.21.15.44:0/2599164621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134105a40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 -- 172.21.15.44:0/2599164621 >> 172.21.15.44:0/2599164621 conn(0x7ff134100df0 msgr2=0x7ff134103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 -- 172.21.15.44:0/2599164621 shutdown_connections 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.012+0000 7ff138f00640 1 -- 172.21.15.44:0/2599164621 wait complete. 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.013+0000 7ff138f00640 1 Processor -- start 2024-09-13T17:17:23.131 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.013+0000 7ff138f00640 1 -- start start 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.013+0000 7ff138f00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.013+0000 7ff138f00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff134199cc0 con 0x7ff134105640 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.013+0000 7ff13259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.013+0000 7ff13259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45648/0 (socket says 172.21.15.44:45648) 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff13259c640 1 -- 172.21.15.44:0/810515704 learned_addr learned my addr 172.21.15.44:0/810515704 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff13259c640 1 -- 172.21.15.44:0/810515704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1240094d0 con 0x7ff134105640 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff13259c640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134199780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff1240097f0 tx=0x7ff124030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff124032c30 con 0x7ff134105640 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff124032d90 con 0x7ff134105640 2024-09-13T17:17:23.132 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff12403a5b0 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1341076c0 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.014+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13419a1d0 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.015+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7ff124039040 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.015+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff134105ac0 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.016+0000 7ff1237fe640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7ff11c038730 0x7ff11c03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.016+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff12406b860 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.016+0000 7ff131d9b640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7ff11c038730 0x7ff11c03abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.016+0000 7ff131d9b640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7ff11c038730 0x7ff11c03abf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1280098c0 tx=0x7ff128006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.020+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff1240313a0 con 0x7ff134105640 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.192+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff134109c70 con 0x7ff11c038730 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.565+0000 7ff1237fe640 1 -- 172.21.15.44:0/810515704 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff134109c70 con 0x7ff11c038730 2024-09-13T17:17:23.133 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7ff11c038730 msgr2=0x7ff11c03abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7ff11c038730 0x7ff11c03abf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1280098c0 tx=0x7ff128006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 msgr2=0x7ff134199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134199780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff1240097f0 tx=0x7ff124030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 shutdown_connections 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7ff11c038730 0x7ff11c03abf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 --2- 172.21.15.44:0/810515704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff134105640 0x7ff134199780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.567+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 >> 172.21.15.44:0/810515704 conn(0x7ff134100df0 msgr2=0x7ff1341019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.568+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 shutdown_connections 2024-09-13T17:17:23.134 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:22.568+0000 7ff138f00640 1 -- 172.21.15.44:0/810515704 wait complete. 2024-09-13T17:17:23.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:23 smithi044 ceph-mon[31175]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:23.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:23 smithi044 ceph-mon[31175]: Generating ssh key... 2024-09-13T17:17:23.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:23 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:23.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:23 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:24.253 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:24.253 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f7941487640 1 Processor -- start 2024-09-13T17:17:24.253 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f7941487640 1 -- start start 2024-09-13T17:17:24.253 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f7941487640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 0x7f793c105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.253 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f7941487640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c106010 con 0x7f793c105640 2024-09-13T17:17:24.253 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f793affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 0x7f793c105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f793affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 0x7f793c105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45664/0 (socket says 172.21.15.44:45664) 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.507+0000 7f793affd640 1 -- 172.21.15.44:0/3514334024 learned_addr learned my addr 172.21.15.44:0/3514334024 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.508+0000 7f793affd640 1 -- 172.21.15.44:0/3514334024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c106840 con 0x7f793c105640 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.508+0000 7f793affd640 1 --2- 172.21.15.44:0/3514334024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 0x7f793c105a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f792c009820 tx=0x7f792c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24bbfc8cd0c0f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7939ffb640 1 -- 172.21.15.44:0/3514334024 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f792c02ae70 con 0x7f793c105640 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7939ffb640 1 -- 172.21.15.44:0/3514334024 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f792c0328b0 con 0x7f793c105640 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7939ffb640 1 -- 172.21.15.44:0/3514334024 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f792c030850 con 0x7f793c105640 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 -- 172.21.15.44:0/3514334024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 msgr2=0x7f793c105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 --2- 172.21.15.44:0/3514334024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 0x7f793c105a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f792c009820 tx=0x7f792c02a270 comp rx=0 tx=0).stop 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 -- 172.21.15.44:0/3514334024 shutdown_connections 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 --2- 172.21.15.44:0/3514334024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c105640 0x7f793c105a40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 -- 172.21.15.44:0/3514334024 >> 172.21.15.44:0/3514334024 conn(0x7f793c100df0 msgr2=0x7f793c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 -- 172.21.15.44:0/3514334024 shutdown_connections 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.509+0000 7f7941487640 1 -- 172.21.15.44:0/3514334024 wait complete. 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f7941487640 1 Processor -- start 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f7941487640 1 -- start start 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f7941487640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 0x7f793c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f7941487640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c07bf70 con 0x7f793c07bb50 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f793affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 0x7f793c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f793affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 0x7f793c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45666/0 (socket says 172.21.15.44:45666) 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.510+0000 7f793affd640 1 -- 172.21.15.44:0/512543504 learned_addr learned my addr 172.21.15.44:0/512543504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f793affd640 1 -- 172.21.15.44:0/512543504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0094d0 con 0x7f793c07bb50 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f793affd640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 0x7f793c07a2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f792c0097f0 tx=0x7f792c030870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f792c030ca0 con 0x7f793c07bb50 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f792c030e00 con 0x7f793c07bb50 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c1076c0 con 0x7f793c07bb50 2024-09-13T17:17:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f792c03b580 con 0x7f793c07bb50 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.511+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c07aa50 con 0x7f793c07bb50 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.512+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f792c039040 con 0x7f793c07bb50 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.512+0000 7f7923fff640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f79240386e0 0x7f792403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.512+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c105a40 con 0x7f793c07bb50 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.513+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f792c06bbe0 con 0x7f793c07bb50 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.513+0000 7f793a7fc640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f79240386e0 0x7f792403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.513+0000 7f793a7fc640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f79240386e0 0x7f792403aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79300098c0 tx=0x7f7930006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.517+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f792c0317b0 con 0x7f793c07bb50 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.689+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f793c07ada0 con 0x7f79240386e0 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.690+0000 7f7923fff640 1 -- 172.21.15.44:0/512543504 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f793c07ada0 con 0x7f79240386e0 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f79240386e0 msgr2=0x7f792403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f79240386e0 0x7f792403aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79300098c0 tx=0x7f7930006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 msgr2=0x7f793c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 0x7f793c07a2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f792c0097f0 tx=0x7f792c030870 comp rx=0 tx=0).stop 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 shutdown_connections 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f79240386e0 0x7f792403aba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 --2- 172.21.15.44:0/512543504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f793c07bb50 0x7f793c07a2c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 >> 172.21.15.44:0/512543504 conn(0x7f793c100df0 msgr2=0x7f793c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 shutdown_connections 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:23.692+0000 7f7941487640 1 -- 172.21.15.44:0/512543504 wait complete. 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-13T17:17:24.257 INFO:teuthology.orchestra.run.smithi044.stdout:Adding key to root@localhost authorized_keys... 2024-09-13T17:17:24.258 INFO:teuthology.orchestra.run.smithi044.stdout:Adding host smithi044... 2024-09-13T17:17:24.841 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:24 smithi044 ceph-mon[31175]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:25.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:25 smithi044 ceph-mon[31175]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "addr": "172.21.15.44", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:26.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:26 smithi044 ceph-mon[31175]: Deploying cephadm binary to smithi044 2024-09-13T17:17:28.828 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Added host 'smithi044' with addr '172.21.15.44' 2024-09-13T17:17:28.828 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.653+0000 7f3bcc89c640 1 Processor -- start 2024-09-13T17:17:28.828 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.653+0000 7f3bcc89c640 1 -- start start 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.654+0000 7f3bcc89c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc8104940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.654+0000 7f3bcc89c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc8104f10 con 0x7f3bc8104540 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.654+0000 7f3bc659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc8104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.654+0000 7f3bc659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc8104940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45670/0 (socket says 172.21.15.44:45670) 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.654+0000 7f3bc659c640 1 -- 172.21.15.44:0/3680545414 learned_addr learned my addr 172.21.15.44:0/3680545414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.654+0000 7f3bc659c640 1 -- 172.21.15.44:0/3680545414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8105740 con 0x7f3bc8104540 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.655+0000 7f3bc659c640 1 --2- 172.21.15.44:0/3680545414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc8104940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009e80 tx=0x7f3bb802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6220b5841a58b10e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.655+0000 7f3bc559a640 1 -- 172.21.15.44:0/3680545414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb8031400 con 0x7f3bc8104540 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bc559a640 1 -- 172.21.15.44:0/3680545414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3bb802d050 con 0x7f3bc8104540 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bc559a640 1 -- 172.21.15.44:0/3680545414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb8007880 con 0x7f3bc8104540 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 -- 172.21.15.44:0/3680545414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 msgr2=0x7f3bc8104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.829 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 --2- 172.21.15.44:0/3680545414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc8104940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009e80 tx=0x7f3bb802a7a0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 -- 172.21.15.44:0/3680545414 shutdown_connections 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 --2- 172.21.15.44:0/3680545414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc8104940 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 -- 172.21.15.44:0/3680545414 >> 172.21.15.44:0/3680545414 conn(0x7f3bc80ffcf0 msgr2=0x7f3bc8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 -- 172.21.15.44:0/3680545414 shutdown_connections 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.656+0000 7f3bcc89c640 1 -- 172.21.15.44:0/3680545414 wait complete. 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bcc89c640 1 Processor -- start 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bcc89c640 1 -- start start 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bcc89c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc818f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bcc89c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc818fd40 con 0x7f3bc8104540 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bc659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc818f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.830 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bc659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc818f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45680/0 (socket says 172.21.15.44:45680) 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.657+0000 7f3bc659c640 1 -- 172.21.15.44:0/1411837336 learned_addr learned my addr 172.21.15.44:0/1411837336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bc659c640 1 -- 172.21.15.44:0/1411837336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8009b30 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bc659c640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc818f800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bb802acc0 tx=0x7f3bb8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb8031ac0 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3bb8031c20 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc81065c0 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb803b590 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.658+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc818e1e0 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.659+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3bb803b800 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.659+0000 7f3bb77fe640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3bb00386e0 0x7f3bb003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.659+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc8104a00 con 0x7f3bc8104540 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.660+0000 7f3bc5d9b640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3bb00386e0 0x7f3bb003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.831 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.660+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3bb806bdc0 con 0x7f3bc8104540 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.660+0000 7f3bc5d9b640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3bb00386e0 0x7f3bb003aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098c0 tx=0x7f3bbc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.664+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3bb80398f0 con 0x7f3bc8104540 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:24.839+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "addr": "172.21.15.44", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc8108c30 con 0x7f3bb00386e0 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.189+0000 7f3bb77fe640 1 -- 172.21.15.44:0/1411837336 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f3bc8108c30 con 0x7f3bb00386e0 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3bb00386e0 msgr2=0x7f3bb003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3bb00386e0 0x7f3bb003aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098c0 tx=0x7f3bbc006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 msgr2=0x7f3bc818f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc818f800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bb802acc0 tx=0x7f3bb8007590 comp rx=0 tx=0).stop 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 shutdown_connections 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3bb00386e0 0x7f3bb003aba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 --2- 172.21.15.44:0/1411837336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8104540 0x7f3bc818f800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 >> 172.21.15.44:0/1411837336 conn(0x7f3bc80ffcf0 msgr2=0x7f3bc8100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:28.832 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 shutdown_connections 2024-09-13T17:17:28.833 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:28.192+0000 7f3bcc89c640 1 -- 172.21.15.44:0/1411837336 wait complete. 2024-09-13T17:17:28.833 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mon service... 2024-09-13T17:17:29.425 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:29 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:29.425 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:29 smithi044 ceph-mon[31175]: Added host smithi044 2024-09-13T17:17:29.425 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:29 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:30.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:30 smithi044 ceph-mon[31175]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:30 smithi044 ceph-mon[31175]: Saving service mon spec with placement count:5 2024-09-13T17:17:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:30 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:30 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:30.766 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e29a21640 1 Processor -- start 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e29a21640 1 -- start start 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e29a21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24108520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e29a21640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24079e50 con 0x7f8e24106130 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e22ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24108520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e22ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24108520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45682/0 (socket says 172.21.15.44:45682) 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e22ffd640 1 -- 172.21.15.44:0/3381250300 learned_addr learned my addr 172.21.15.44:0/3381250300 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.231+0000 7f8e22ffd640 1 -- 172.21.15.44:0/3381250300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e24108a60 con 0x7f8e24106130 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.232+0000 7f8e22ffd640 1 --2- 172.21.15.44:0/3381250300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24108520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009820 tx=0x7f8e1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeaab1c6768e047 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.232+0000 7f8e227fc640 1 -- 172.21.15.44:0/3381250300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e1402ae70 con 0x7f8e24106130 2024-09-13T17:17:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.232+0000 7f8e227fc640 1 -- 172.21.15.44:0/3381250300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8e140328b0 con 0x7f8e24106130 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.232+0000 7f8e227fc640 1 -- 172.21.15.44:0/3381250300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e14030850 con 0x7f8e24106130 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 -- 172.21.15.44:0/3381250300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 msgr2=0x7f8e24108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 --2- 172.21.15.44:0/3381250300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24108520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009820 tx=0x7f8e1402a270 comp rx=0 tx=0).stop 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 -- 172.21.15.44:0/3381250300 shutdown_connections 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 --2- 172.21.15.44:0/3381250300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24108520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 -- 172.21.15.44:0/3381250300 >> 172.21.15.44:0/3381250300 conn(0x7f8e240ffcf0 msgr2=0x7f8e24102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 -- 172.21.15.44:0/3381250300 shutdown_connections 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.233+0000 7f8e29a21640 1 -- 172.21.15.44:0/3381250300 wait complete. 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e29a21640 1 Processor -- start 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e29a21640 1 -- start start 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e29a21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:30.768 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e29a21640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24199a80 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e22ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e22ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45694/0 (socket says 172.21.15.44:45694) 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e22ffd640 1 -- 172.21.15.44:0/3956399577 learned_addr learned my addr 172.21.15.44:0/3956399577 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e22ffd640 1 -- 172.21.15.44:0/3956399577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140094d0 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.234+0000 7f8e22ffd640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24199540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e140097f0 tx=0x7f8e14030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.235+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e14032c30 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.235+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e24199c80 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.235+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8e14032d90 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.235+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e1403a5b0 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.235+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2419a120 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.235+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f8e14039040 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.236+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e24073df0 con 0x7f8e24106130 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.236+0000 7f8e28a1f640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f8e100386e0 0x7f8e1003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:30.769 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.236+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8e1406bb90 con 0x7f8e24106130 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.236+0000 7f8e1bfff640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f8e100386e0 0x7f8e1003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.236+0000 7f8e1bfff640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f8e100386e0 0x7f8e1003aba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098c0 tx=0x7f8e0c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.241+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8e1404b6c0 con 0x7f8e24106130 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.422+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8e24101000 con 0x7f8e100386e0 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.428+0000 7f8e28a1f640 1 -- 172.21.15.44:0/3956399577 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8e24101000 con 0x7f8e100386e0 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f8e100386e0 msgr2=0x7f8e1003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f8e100386e0 0x7f8e1003aba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098c0 tx=0x7f8e0c006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 msgr2=0x7f8e24199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24199540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e140097f0 tx=0x7f8e14030d40 comp rx=0 tx=0).stop 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 shutdown_connections 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f8e100386e0 0x7f8e1003aba0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:30.770 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 --2- 172.21.15.44:0/3956399577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e24106130 0x7f8e24199540 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:30.771 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 >> 172.21.15.44:0/3956399577 conn(0x7f8e240ffcf0 msgr2=0x7f8e241008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:30.771 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 shutdown_connections 2024-09-13T17:17:30.771 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:29.430+0000 7f8e29a21640 1 -- 172.21.15.44:0/3956399577 wait complete. 2024-09-13T17:17:30.771 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mgr service... 2024-09-13T17:17:31.988 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-13T17:17:31.988 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.140+0000 7f4adfaac640 1 Processor -- start 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.140+0000 7f4adfaac640 1 -- start start 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.140+0000 7f4adfaac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad8105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.140+0000 7f4adfaac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad8106010 con 0x7f4ad8105640 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.141+0000 7f4add848640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad8105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.141+0000 7f4add848640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad8105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36472/0 (socket says 172.21.15.44:36472) 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.141+0000 7f4add848640 1 -- 172.21.15.44:0/1215683643 learned_addr learned my addr 172.21.15.44:0/1215683643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.141+0000 7f4add848640 1 -- 172.21.15.44:0/1215683643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8106840 con 0x7f4ad8105640 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4add848640 1 --2- 172.21.15.44:0/1215683643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad8105a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009820 tx=0x7f4ad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5194f32ccdb89b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adc846640 1 -- 172.21.15.44:0/1215683643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad002ae70 con 0x7f4ad8105640 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adc846640 1 -- 172.21.15.44:0/1215683643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ad00328d0 con 0x7f4ad8105640 2024-09-13T17:17:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adc846640 1 -- 172.21.15.44:0/1215683643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad00307a0 con 0x7f4ad8105640 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adfaac640 1 -- 172.21.15.44:0/1215683643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 msgr2=0x7f4ad8105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adfaac640 1 --2- 172.21.15.44:0/1215683643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad8105a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009820 tx=0x7f4ad002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adfaac640 1 -- 172.21.15.44:0/1215683643 shutdown_connections 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adfaac640 1 --2- 172.21.15.44:0/1215683643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad8105a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.142+0000 7f4adfaac640 1 -- 172.21.15.44:0/1215683643 >> 172.21.15.44:0/1215683643 conn(0x7f4ad8100df0 msgr2=0x7f4ad8103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.143+0000 7f4adfaac640 1 -- 172.21.15.44:0/1215683643 shutdown_connections 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.143+0000 7f4adfaac640 1 -- 172.21.15.44:0/1215683643 wait complete. 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.143+0000 7f4adfaac640 1 Processor -- start 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.143+0000 7f4adfaac640 1 -- start start 2024-09-13T17:17:31.990 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.143+0000 7f4adfaac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.143+0000 7f4adfaac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad819bcf0 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4add848640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4add848640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36488/0 (socket says 172.21.15.44:36488) 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4add848640 1 -- 172.21.15.44:0/2673694692 learned_addr learned my addr 172.21.15.44:0/2673694692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4add848640 1 -- 172.21.15.44:0/2673694692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00094d0 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4add848640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad819b7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0030410 tx=0x7f4ad0030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad0032c50 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ad0032db0 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad819bef0 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad003a520 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.144+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad819c390 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.145+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4ad0039040 con 0x7f4ad8105640 2024-09-13T17:17:31.991 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.145+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad8073df0 con 0x7f4ad8105640 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.146+0000 7f4ad6ffd640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f4ac0038730 0x7f4ac003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.146+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ad006b7f0 con 0x7f4ad8105640 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.146+0000 7f4add047640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f4ac0038730 0x7f4ac003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.146+0000 7f4add047640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f4ac0038730 0x7f4ac003abf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0098c0 tx=0x7f4acc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.150+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4ad0031580 con 0x7f4ad8105640 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.333+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad8102050 con 0x7f4ac0038730 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.338+0000 7f4ad6ffd640 1 -- 172.21.15.44:0/2673694692 <== mgr.14118 v2:172.21.15.44:6800/2265862199 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4ad8102050 con 0x7f4ac0038730 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f4ac0038730 msgr2=0x7f4ac003abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f4ac0038730 0x7f4ac003abf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0098c0 tx=0x7f4acc006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 msgr2=0x7f4ad819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad819b7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0030410 tx=0x7f4ad0030d50 comp rx=0 tx=0).stop 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 shutdown_connections 2024-09-13T17:17:31.992 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f4ac0038730 0x7f4ac003abf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.993 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 --2- 172.21.15.44:0/2673694692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ad8105640 0x7f4ad819b7b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.993 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 >> 172.21.15.44:0/2673694692 conn(0x7f4ad8100df0 msgr2=0x7f4ad8101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:31.993 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 shutdown_connections 2024-09-13T17:17:31.993 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:31.340+0000 7f4adfaac640 1 -- 172.21.15.44:0/2673694692 wait complete. 2024-09-13T17:17:32.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:32 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:32.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:32 smithi044 ceph-mon[31175]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:32.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:32 smithi044 ceph-mon[31175]: Saving service mgr spec with placement count:2 2024-09-13T17:17:32.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:32 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:33.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:33 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1371613882' entity='client.admin' 2024-09-13T17:17:33.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:33 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:33.834 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.434+0000 7f096c65c640 1 Processor -- start 2024-09-13T17:17:33.834 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.434+0000 7f096c65c640 1 -- start start 2024-09-13T17:17:33.834 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.435+0000 7f096c65c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964108500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.834 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.435+0000 7f096c65c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964079e70 con 0x7f0964106110 2024-09-13T17:17:33.834 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.435+0000 7f096a3f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.435+0000 7f096a3f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964108500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36504/0 (socket says 172.21.15.44:36504) 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.435+0000 7f096a3f8640 1 -- 172.21.15.44:0/210007342 learned_addr learned my addr 172.21.15.44:0/210007342 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.435+0000 7f096a3f8640 1 -- 172.21.15.44:0/210007342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964108a40 con 0x7f0964106110 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.436+0000 7f096a3f8640 1 --2- 172.21.15.44:0/210007342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964108500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0960009820 tx=0x7f096002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5443c9fb8bf5723b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.436+0000 7f09693f6640 1 -- 172.21.15.44:0/210007342 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f096002ae70 con 0x7f0964106110 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.436+0000 7f09693f6640 1 -- 172.21.15.44:0/210007342 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09600328d0 con 0x7f0964106110 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.436+0000 7f09693f6640 1 -- 172.21.15.44:0/210007342 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09600307a0 con 0x7f0964106110 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 -- 172.21.15.44:0/210007342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 msgr2=0x7f0964108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 --2- 172.21.15.44:0/210007342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964108500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0960009820 tx=0x7f096002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 -- 172.21.15.44:0/210007342 shutdown_connections 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 --2- 172.21.15.44:0/210007342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964108500 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 -- 172.21.15.44:0/210007342 >> 172.21.15.44:0/210007342 conn(0x7f09640ffcf0 msgr2=0x7f0964102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.835 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 -- 172.21.15.44:0/210007342 shutdown_connections 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.437+0000 7f096c65c640 1 -- 172.21.15.44:0/210007342 wait complete. 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096c65c640 1 Processor -- start 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096c65c640 1 -- start start 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096c65c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096c65c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964199b40 con 0x7f0964106110 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096a3f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096a3f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36514/0 (socket says 172.21.15.44:36514) 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096a3f8640 1 -- 172.21.15.44:0/1371613882 learned_addr learned my addr 172.21.15.44:0/1371613882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096a3f8640 1 -- 172.21.15.44:0/1371613882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600094d0 con 0x7f0964106110 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.438+0000 7f096a3f8640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964199600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f09600097f0 tx=0x7f0960032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.439+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0960030c70 con 0x7f0964106110 2024-09-13T17:17:33.836 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.439+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0960030dd0 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.439+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0964199d40 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.439+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f096003a5b0 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.439+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096419a1e0 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.440+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0960039040 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.440+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09641933c0 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.440+0000 7f095b7fe640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f0950038730 0x7f095003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.440+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f096006b9e0 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.440+0000 7f0969bf7640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f0950038730 0x7f095003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.440+0000 7f0969bf7640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f0950038730 0x7f095003abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09540098c0 tx=0x7f0954006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.444+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0960031d70 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.644+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0964073df0 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.648+0000 7f095b7fe640 1 -- 172.21.15.44:0/1371613882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f096003f030 con 0x7f0964106110 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f0950038730 msgr2=0x7f095003abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.837 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f0950038730 0x7f095003abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09540098c0 tx=0x7f0954006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 msgr2=0x7f0964199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964199600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f09600097f0 tx=0x7f0960032c70 comp rx=0 tx=0).stop 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 shutdown_connections 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f0950038730 0x7f095003abf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 --2- 172.21.15.44:0/1371613882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0964106110 0x7f0964199600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 >> 172.21.15.44:0/1371613882 conn(0x7f09640ffcf0 msgr2=0x7f09641009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 shutdown_connections 2024-09-13T17:17:33.838 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:32.650+0000 7f096c65c640 1 -- 172.21.15.44:0/1371613882 wait complete. 2024-09-13T17:17:35.036 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7b19f640 1 Processor -- start 2024-09-13T17:17:35.036 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7b19f640 1 -- start start 2024-09-13T17:17:35.036 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7b19f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7407a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.036 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7b19f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7407c030 con 0x7f3a7407a300 2024-09-13T17:17:35.036 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7a19d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7407a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.037 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7a19d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7407a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36518/0 (socket says 172.21.15.44:36518) 2024-09-13T17:17:35.037 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.243+0000 7f3a7a19d640 1 -- 172.21.15.44:0/625284605 learned_addr learned my addr 172.21.15.44:0/625284605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:35.037 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.244+0000 7f3a7a19d640 1 -- 172.21.15.44:0/625284605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7407c170 con 0x7f3a7407a300 2024-09-13T17:17:35.037 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.244+0000 7f3a7a19d640 1 --2- 172.21.15.44:0/625284605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7407a700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009820 tx=0x7f3a7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cd0bd0e854d47e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.244+0000 7f3a7919b640 1 -- 172.21.15.44:0/625284605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a7002ae70 con 0x7f3a7407a300 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7919b640 1 -- 172.21.15.44:0/625284605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a700328d0 con 0x7f3a7407a300 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7919b640 1 -- 172.21.15.44:0/625284605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a700307a0 con 0x7f3a7407a300 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 -- 172.21.15.44:0/625284605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 msgr2=0x7f3a7407a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 --2- 172.21.15.44:0/625284605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7407a700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009820 tx=0x7f3a7002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 -- 172.21.15.44:0/625284605 shutdown_connections 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 --2- 172.21.15.44:0/625284605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7407a700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 -- 172.21.15.44:0/625284605 >> 172.21.15.44:0/625284605 conn(0x7f3a74100c50 msgr2=0x7f3a74103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 -- 172.21.15.44:0/625284605 shutdown_connections 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.245+0000 7f3a7b19f640 1 -- 172.21.15.44:0/625284605 wait complete. 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7b19f640 1 Processor -- start 2024-09-13T17:17:35.038 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7b19f640 1 -- start start 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7b19f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7b19f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7419bbc0 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7a19d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7a19d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36526/0 (socket says 172.21.15.44:36526) 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.246+0000 7f3a7a19d640 1 -- 172.21.15.44:0/1817081645 learned_addr learned my addr 172.21.15.44:0/1817081645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a7a19d640 1 -- 172.21.15.44:0/1817081645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a700094d0 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a7a19d640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7419b680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a700097f0 tx=0x7f3a70032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a70030da0 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7419bdc0 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a7003b8c0 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a7003a570 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.247+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7419c260 con 0x7f3a7407a300 2024-09-13T17:17:35.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.248+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3a70039040 con 0x7f3a7407a300 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.248+0000 7f3a6b7fe640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3a600386e0 0x7f3a6003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.248+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a74073df0 con 0x7f3a7407a300 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.248+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a7006b720 con 0x7f3a7407a300 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.248+0000 7f3a7999c640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3a600386e0 0x7f3a6003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.249+0000 7f3a7999c640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3a600386e0 0x7f3a6003aba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098c0 tx=0x7f3a64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.253+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3a7003f070 con 0x7f3a7407a300 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.425+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f3a74073f80 con 0x7f3a7407a300 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.428+0000 7f3a6b7fe640 1 -- 172.21.15.44:0/1817081645 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f3a700690c0 con 0x7f3a7407a300 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3a600386e0 msgr2=0x7f3a6003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3a600386e0 0x7f3a6003aba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098c0 tx=0x7f3a64006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 msgr2=0x7f3a7419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7419b680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a700097f0 tx=0x7f3a70032db0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 shutdown_connections 2024-09-13T17:17:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f3a600386e0 0x7f3a6003aba0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 --2- 172.21.15.44:0/1817081645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7407a300 0x7f3a7419b680 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.430+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 >> 172.21.15.44:0/1817081645 conn(0x7f3a74100c50 msgr2=0x7f3a74101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.431+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 shutdown_connections 2024-09-13T17:17:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:34.431+0000 7f3a7b19f640 1 -- 172.21.15.44:0/1817081645 wait complete. 2024-09-13T17:17:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling the dashboard module... 2024-09-13T17:17:35.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1817081645' entity='client.admin' 2024-09-13T17:17:35.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:35 smithi044 ceph-mon[31175]: from='mgr.14118 172.21.15.44:0/1938676771' entity='mgr.x' 2024-09-13T17:17:36.760 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ignoring --setuser ceph since I am not root 2024-09-13T17:17:36.760 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ignoring --setgroup ceph since I am not root 2024-09-13T17:17:36.760 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:36.586+0000 7f6866250040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:17:36.760 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:36.645+0000 7f6866250040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:17:36.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1140041261' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-13T17:17:37.027 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:36.802+0000 7f6866250040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:17:37.027 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:36 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:36.871+0000 7f6866250040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:17:37.027 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:37.024+0000 7f6866250040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:17:37.356 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:37.172+0000 7f6866250040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:17:37.369 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.608+0000 7fcf2175c640 1 Processor -- start 2024-09-13T17:17:37.369 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf2175c640 1 -- start start 2024-09-13T17:17:37.369 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf2175c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf2175c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c1042b0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf1affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf1affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c103ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36528/0 (socket says 172.21.15.44:36528) 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf1affd640 1 -- 172.21.15.44:0/2329538623 learned_addr learned my addr 172.21.15.44:0/2329538623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.609+0000 7fcf1affd640 1 -- 172.21.15.44:0/2329538623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c1043f0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.610+0000 7fcf1affd640 1 --2- 172.21.15.44:0/2329538623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c103ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009820 tx=0x7fcf0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9211d5e4599e22ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.610+0000 7fcf19ffb640 1 -- 172.21.15.44:0/2329538623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf0c02ae70 con 0x7fcf1c1038e0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.610+0000 7fcf19ffb640 1 -- 172.21.15.44:0/2329538623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf0c0328d0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.610+0000 7fcf19ffb640 1 -- 172.21.15.44:0/2329538623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf0c0307a0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 -- 172.21.15.44:0/2329538623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 msgr2=0x7fcf1c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 --2- 172.21.15.44:0/2329538623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c103ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009820 tx=0x7fcf0c02a270 comp rx=0 tx=0).stop 2024-09-13T17:17:37.370 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 -- 172.21.15.44:0/2329538623 shutdown_connections 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 --2- 172.21.15.44:0/2329538623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c103ce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 -- 172.21.15.44:0/2329538623 >> 172.21.15.44:0/2329538623 conn(0x7fcf1c0ff470 msgr2=0x7fcf1c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 -- 172.21.15.44:0/2329538623 shutdown_connections 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.611+0000 7fcf2175c640 1 -- 172.21.15.44:0/2329538623 wait complete. 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf2175c640 1 Processor -- start 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf2175c640 1 -- start start 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf2175c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c106140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf2175c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c106680 con 0x7fcf1c1038e0 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf1affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c106140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf1affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c106140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36534/0 (socket says 172.21.15.44:36534) 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.612+0000 7fcf1affd640 1 -- 172.21.15.44:0/1140041261 learned_addr learned my addr 172.21.15.44:0/1140041261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf1affd640 1 -- 172.21.15.44:0/1140041261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0094d0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf1affd640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c106140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097f0 tx=0x7fcf0c032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf0c030c70 con 0x7fcf1c1038e0 2024-09-13T17:17:37.371 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf0c030dd0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c104870 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf0c03a5b0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.613+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c104d10 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.614+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fcf0c039040 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.614+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c103d60 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.614+0000 7fcf03fff640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fcf04038730 0x7fcf0403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.614+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcf0c06b8f0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.614+0000 7fcf1a7fc640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fcf04038730 0x7fcf0403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.615+0000 7fcf1a7fc640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fcf04038730 0x7fcf0403abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000ad30 tx=0x7fcf100093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.619+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcf0c04b710 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:35.846+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fcf1c109af0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.436+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fcf0c03ba50 con 0x7fcf1c1038e0 2024-09-13T17:17:37.372 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.437+0000 7fcf03fff640 1 -- 172.21.15.44:0/1140041261 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fcf0c049cd0 con 0x7fcf1c1038e0 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fcf04038730 msgr2=0x7fcf0403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fcf04038730 0x7fcf0403abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000ad30 tx=0x7fcf100093f0 comp rx=0 tx=0).stop 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 msgr2=0x7fcf1c106140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c106140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097f0 tx=0x7fcf0c032c70 comp rx=0 tx=0).stop 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 shutdown_connections 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fcf04038730 0x7fcf0403abf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 --2- 172.21.15.44:0/1140041261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf1c1038e0 0x7fcf1c106140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 >> 172.21.15.44:0/1140041261 conn(0x7fcf1c0ff470 msgr2=0x7fcf1c100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 shutdown_connections 2024-09-13T17:17:37.373 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:36.440+0000 7fcf2175c640 1 -- 172.21.15.44:0/1140041261 wait complete. 2024-09-13T17:17:37.616 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1140041261' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-13T17:17:37.616 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-mon[31175]: mgrmap e9: x(active, since 18s) 2024-09-13T17:17:37.616 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:37.499+0000 7f6866250040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:17:37.616 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:37.562+0000 7f6866250040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:17:38.020 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:37.757+0000 7f6866250040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:17:38.661 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:38.661 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-13T17:17:38.661 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T17:17:38.661 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.810+0000 7fe995888640 1 Processor -- start 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.810+0000 7fe995888640 1 -- start start 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.811+0000 7fe995888640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990104940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.811+0000 7fe995888640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990104f10 con 0x7fe990104540 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.811+0000 7fe98effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.811+0000 7fe98effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990104940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36546/0 (socket says 172.21.15.44:36546) 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.811+0000 7fe98effd640 1 -- 172.21.15.44:0/3902142245 learned_addr learned my addr 172.21.15.44:0/3902142245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.811+0000 7fe98effd640 1 -- 172.21.15.44:0/3902142245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990105740 con 0x7fe990104540 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.812+0000 7fe98effd640 1 --2- 172.21.15.44:0/3902142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990104940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe980009820 tx=0x7fe98002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cbb6d9b776280a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.812+0000 7fe98dffb640 1 -- 172.21.15.44:0/3902142245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98002ae70 con 0x7fe990104540 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.812+0000 7fe98dffb640 1 -- 172.21.15.44:0/3902142245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9800328d0 con 0x7fe990104540 2024-09-13T17:17:38.662 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.812+0000 7fe98dffb640 1 -- 172.21.15.44:0/3902142245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9800307a0 con 0x7fe990104540 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 -- 172.21.15.44:0/3902142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 msgr2=0x7fe990104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 --2- 172.21.15.44:0/3902142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990104940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe980009820 tx=0x7fe98002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 -- 172.21.15.44:0/3902142245 shutdown_connections 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 --2- 172.21.15.44:0/3902142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990104940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 -- 172.21.15.44:0/3902142245 >> 172.21.15.44:0/3902142245 conn(0x7fe9900ffcf0 msgr2=0x7fe990102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 -- 172.21.15.44:0/3902142245 shutdown_connections 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.813+0000 7fe995888640 1 -- 172.21.15.44:0/3902142245 wait complete. 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.814+0000 7fe995888640 1 Processor -- start 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.814+0000 7fe995888640 1 -- start start 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe995888640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe995888640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901978c0 con 0x7fe990104540 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe98effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe98effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36548/0 (socket says 172.21.15.44:36548) 2024-09-13T17:17:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe98effd640 1 -- 172.21.15.44:0/4235732703 learned_addr learned my addr 172.21.15.44:0/4235732703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe98effd640 1 -- 172.21.15.44:0/4235732703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800094d0 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.815+0000 7fe98effd640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990197380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe980030410 tx=0x7fe980030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.816+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe980032c50 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.816+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe980032db0 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.816+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98003a520 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.816+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe990197ac0 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.816+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe990197f60 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.817+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fe980039040 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.817+0000 7fe994886640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 0x7fe97803f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.817+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe98006c700 con 0x7fe990104540 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.817+0000 7fe98e7fc640 1 -- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 msgr2=0x7fe97803f030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2265862199 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.817+0000 7fe98e7fc640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 0x7fe97803f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:17:38.664 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.817+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990104940 con 0x7fe990104540 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:37.822+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe980037070 con 0x7fe990104540 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.018+0000 7fe98e7fc640 1 -- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 msgr2=0x7fe97803f030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2265862199 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.018+0000 7fe98e7fc640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 0x7fe97803f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.040+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe990190a20 con 0x7fe990104540 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.041+0000 7fe994886640 1 -- 172.21.15.44:0/4235732703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe98003f030 con 0x7fe990104540 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 msgr2=0x7fe97803f030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 0x7fe97803f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 msgr2=0x7fe990197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990197380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe980030410 tx=0x7fe980030d50 comp rx=0 tx=0).stop 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 shutdown_connections 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7fe97803cb70 0x7fe97803f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 --2- 172.21.15.44:0/4235732703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990104540 0x7fe990197380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 >> 172.21.15.44:0/4235732703 conn(0x7fe9900ffcf0 msgr2=0x7fe99018db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:38.665 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 shutdown_connections 2024-09-13T17:17:38.666 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:38.043+0000 7fe995888640 1 -- 172.21.15.44:0/4235732703 wait complete. 2024-09-13T17:17:38.666 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2024-09-13T17:17:38.666 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 9... 2024-09-13T17:17:38.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:38 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4235732703' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-13T17:17:38.972 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:38 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:38.698+0000 7f6866250040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:17:38.972 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:38 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:38.767+0000 7f6866250040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:17:39.796 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:39.526+0000 7f6866250040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:17:39.796 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:39.646+0000 7f6866250040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:17:40.084 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:39.794+0000 7f6866250040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:17:40.084 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:39.857+0000 7f6866250040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:17:40.084 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.011+0000 7f6866250040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:17:40.084 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.081+0000 7f6866250040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:17:40.356 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.144+0000 7f6866250040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:17:40.356 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.308+0000 7f6866250040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:17:40.656 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.375+0000 7f6866250040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:17:40.656 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.445+0000 7f6866250040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:17:40.656 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.531+0000 7f6866250040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:17:40.656 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.593+0000 7f6866250040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:17:40.656 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:40.654+0000 7f6866250040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:17:41.723 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:41.470+0000 7f6866250040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:17:41.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: Active manager daemon x restarted 2024-09-13T17:17:41.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: Activating manager daemon x 2024-09-13T17:17:41.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: osdmap e3: 0 total, 0 up, 0 in 2024-09-13T17:17:41.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: mgrmap e10: x(active, starting, since 0.00316273s) 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: Manager daemon x is now available 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:17:41.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67ce8a7640 1 Processor -- start 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67ce8a7640 1 -- start start 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67ce8a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c8104300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67ce8a7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8079e70 con 0x7f67c8103f00 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c8104300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c8104300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36564/0 (socket says 172.21.15.44:36564) 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67c7fff640 1 -- 172.21.15.44:0/1861483668 learned_addr learned my addr 172.21.15.44:0/1861483668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:43.238 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.052+0000 7f67c7fff640 1 -- 172.21.15.44:0/1861483668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8104840 con 0x7f67c8103f00 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.053+0000 7f67c7fff640 1 --2- 172.21.15.44:0/1861483668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c8104300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009820 tx=0x7f67b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda8e1dfbe6ac114 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.053+0000 7f67c6ffd640 1 -- 172.21.15.44:0/1861483668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67b802ae70 con 0x7f67c8103f00 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.053+0000 7f67c6ffd640 1 -- 172.21.15.44:0/1861483668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67b80328d0 con 0x7f67c8103f00 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.053+0000 7f67c6ffd640 1 -- 172.21.15.44:0/1861483668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67b80307a0 con 0x7f67c8103f00 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 -- 172.21.15.44:0/1861483668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 msgr2=0x7f67c8104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/1861483668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c8104300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009820 tx=0x7f67b802a270 comp rx=0 tx=0).stop 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 -- 172.21.15.44:0/1861483668 shutdown_connections 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/1861483668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c8104300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 -- 172.21.15.44:0/1861483668 >> 172.21.15.44:0/1861483668 conn(0x7f67c80ffcf0 msgr2=0x7f67c8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 -- 172.21.15.44:0/1861483668 shutdown_connections 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.054+0000 7f67ce8a7640 1 -- 172.21.15.44:0/1861483668 wait complete. 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67ce8a7640 1 Processor -- start 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67ce8a7640 1 -- start start 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67ce8a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.239 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67ce8a7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819bc90 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36572/0 (socket says 172.21.15.44:36572) 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.055+0000 7f67c7fff640 1 -- 172.21.15.44:0/2451261868 learned_addr learned my addr 172.21.15.44:0/2451261868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67c7fff640 1 -- 172.21.15.44:0/2451261868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80094d0 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67c7fff640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c819b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67b8030410 tx=0x7f67b8030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67b8032c50 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c819be90 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67b8032db0 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67b803a5b0 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.056+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c819c330 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.057+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f67b8039040 con 0x7f67c8103f00 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.057+0000 7f67c57fa640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.240 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.057+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 --> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67b003b300 con 0x7f67b0038730 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.057+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f67b806b8a0 con 0x7f67c8103f00 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.057+0000 7f67c77fe640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 msgr2=0x7f67b003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2265862199 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.057+0000 7f67c77fe640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.261+0000 7f67c77fe640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 msgr2=0x7f67b003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2265862199 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.261+0000 7f67c77fe640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.662+0000 7f67c77fe640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 msgr2=0x7f67b003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2265862199 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:39.662+0000 7f67c77fe640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:40.463+0000 7f67c77fe640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 msgr2=0x7f67b003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2265862199 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:40.463+0000 7f67c77fe640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:41.473+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f67b803a940 con 0x7f67c8103f00 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:41.473+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 msgr2=0x7f67b003abf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:41.473+0000 7f67c57fa640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.241 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.475+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f67b803ba50 con 0x7f67c8103f00 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.475+0000 7f67c57fa640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67b003bfd0 0x7f67b003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.475+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67b003b300 con 0x7f67b003bfd0 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.476+0000 7f67c77fe640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67b003bfd0 0x7f67b003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.477+0000 7f67c77fe640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67b003bfd0 0x7f67b003e3c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f67bc003980 tx=0x7f67bc006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.477+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f67b003b300 con 0x7f67b003bfd0 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.483+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f67c8073df0 con 0x7f67b003bfd0 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.483+0000 7f67c57fa640 1 -- 172.21.15.44:0/2451261868 <== mgr.14150 v2:172.21.15.44:6800/322597156 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f67c8073df0 con 0x7f67b003bfd0 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67b003bfd0 msgr2=0x7f67b003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67b003bfd0 0x7f67b003e3c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f67bc003980 tx=0x7f67bc006e10 comp rx=0 tx=0).stop 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 msgr2=0x7f67c819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c819b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67b8030410 tx=0x7f67b8030d50 comp rx=0 tx=0).stop 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 shutdown_connections 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67b003bfd0 0x7f67b003e3c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.242 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:6800/2265862199,v1:172.21.15.44:6801/2265862199] conn(0x7f67b0038730 0x7f67b003abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.243 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 --2- 172.21.15.44:0/2451261868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67c8103f00 0x7f67c819b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.243 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 >> 172.21.15.44:0/2451261868 conn(0x7f67c80ffcf0 msgr2=0x7f67c8192360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:43.243 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 shutdown_connections 2024-09-13T17:17:43.243 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:42.484+0000 7f67ce8a7640 1 -- 172.21.15.44:0/2451261868 wait complete. 2024-09-13T17:17:43.243 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 9 is available 2024-09-13T17:17:43.243 INFO:teuthology.orchestra.run.smithi044.stdout:Generating a dashboard self-signed certificate... 2024-09-13T17:17:43.497 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-mon[31175]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-mon[31175]: mgrmap e11: x(active, since 1.00632s) 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-mon[31175]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:43.449+0000 7f67d1ee5640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:17:17:43] ENGINE Error in HTTPServer.serve 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: Traceback (most recent call last): 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._connections.run(self.expiration_interval) 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._run(expiration_interval) 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:17:43.498 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: s = self.context.wrap_socket( 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: return self.sslsocket_class._create( 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self.do_handshake() 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._sslobj.do_handshake() 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:17:43.499 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:17:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:17:44.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-13T17:17:44.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.676+0000 7fd05142d640 1 Processor -- start 2024-09-13T17:17:44.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.676+0000 7fd05142d640 1 -- start start 2024-09-13T17:17:44.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.676+0000 7fd05142d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c108500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.676+0000 7fd05142d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c079e70 con 0x7fd04c106110 2024-09-13T17:17:44.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.676+0000 7fd04affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.677+0000 7fd04affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c108500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53576/0 (socket says 172.21.15.44:53576) 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.677+0000 7fd04affd640 1 -- 172.21.15.44:0/4094031796 learned_addr learned my addr 172.21.15.44:0/4094031796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.677+0000 7fd04affd640 1 -- 172.21.15.44:0/4094031796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c108a40 con 0x7fd04c106110 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.677+0000 7fd04affd640 1 --2- 172.21.15.44:0/4094031796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c108500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd044009820 tx=0x7fd04402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8075289b15de7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd049ffb640 1 -- 172.21.15.44:0/4094031796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd04402ae70 con 0x7fd04c106110 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd049ffb640 1 -- 172.21.15.44:0/4094031796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0440328d0 con 0x7fd04c106110 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd049ffb640 1 -- 172.21.15.44:0/4094031796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0440307a0 con 0x7fd04c106110 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 -- 172.21.15.44:0/4094031796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 msgr2=0x7fd04c108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 --2- 172.21.15.44:0/4094031796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c108500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd044009820 tx=0x7fd04402a270 comp rx=0 tx=0).stop 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 -- 172.21.15.44:0/4094031796 shutdown_connections 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 --2- 172.21.15.44:0/4094031796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c108500 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 -- 172.21.15.44:0/4094031796 >> 172.21.15.44:0/4094031796 conn(0x7fd04c0ffcf0 msgr2=0x7fd04c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 -- 172.21.15.44:0/4094031796 shutdown_connections 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.678+0000 7fd05142d640 1 -- 172.21.15.44:0/4094031796 wait complete. 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd05142d640 1 Processor -- start 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd05142d640 1 -- start start 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd05142d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c105990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd05142d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c105ed0 con 0x7fd04c106110 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd04affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd04affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53588/0 (socket says 172.21.15.44:53588) 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.679+0000 7fd04affd640 1 -- 172.21.15.44:0/2108038685 learned_addr learned my addr 172.21.15.44:0/2108038685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd04affd640 1 -- 172.21.15.44:0/2108038685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440094d0 con 0x7fd04c106110 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd04affd640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c105990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0440097f0 tx=0x7fd044032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd044030c70 con 0x7fd04c106110 2024-09-13T17:17:44.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd044030dd0 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c1040c0 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd04403a5b0 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.680+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c104560 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.681+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fd044039040 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.681+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c106570 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.681+0000 7fd043fff640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd038038780 0x7fd03803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.681+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd04406bbf0 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.681+0000 7fd04a7fc640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd038038780 0x7fd03803ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.682+0000 7fd04a7fc640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd038038780 0x7fd03803ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0340098c0 tx=0x7fd034006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.686+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd04404b710 con 0x7fd04c106110 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.864+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd04c1047a0 con 0x7fd038038780 2024-09-13T17:17:44.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.974+0000 7fd043fff640 1 -- 172.21.15.44:0/2108038685 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd04c1047a0 con 0x7fd038038780 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd038038780 msgr2=0x7fd03803ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd038038780 0x7fd03803ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0340098c0 tx=0x7fd034006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 msgr2=0x7fd04c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c105990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0440097f0 tx=0x7fd044032c70 comp rx=0 tx=0).stop 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 shutdown_connections 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd038038780 0x7fd03803ac40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 --2- 172.21.15.44:0/2108038685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd04c106110 0x7fd04c105990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 >> 172.21.15.44:0/2108038685 conn(0x7fd04c0ffcf0 msgr2=0x7fd04c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 shutdown_connections 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:43.977+0000 7fd05142d640 1 -- 172.21.15.44:0/2108038685 wait complete. 2024-09-13T17:17:44.522 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial admin user... 2024-09-13T17:17:44.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:43] ENGINE Bus STARTING 2024-09-13T17:17:44.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:43] ENGINE Serving on https://172.21.15.44:7150 2024-09-13T17:17:44.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:43] ENGINE Error in HTTPServer.serve 2024-09-13T17:17:44.688 INFO:journalctl@ceph.mon.a.smithi044.stdout: Traceback (most recent call last): 2024-09-13T17:17:44.688 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:17:44.688 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._connections.run(self.expiration_interval) 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._run(expiration_interval) 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: s = self.context.wrap_socket( 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: return self.sslsocket_class._create( 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: self.do_handshake() 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:17:44.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._sslobj.do_handshake() 2024-09-13T17:17:44.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:17:44.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: mgrmap e12: x(active, since 2s) 2024-09-13T17:17:44.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:44.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:44.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:45 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:43] ENGINE Serving on http://172.21.15.44:8765 2024-09-13T17:17:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:45 smithi044 ceph-mon[31175]: [13/Sep/2024:17:17:43] ENGINE Bus STARTED 2024-09-13T17:17:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:45 smithi044 ceph-mon[31175]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:46.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$eR9Qdr9EFwB5sKRB4gBh.O.vQe4mArJe9JDkl2PGc4cLiB1PlnRUG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726247865, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.870+0000 7f11ec11f640 1 Processor -- start 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11ec11f640 1 -- start start 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11ec11f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e41084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11ec11f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4079ed0 con 0x7f11e41060b0 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11e9ebb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e41084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11e9ebb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e41084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53600/0 (socket says 172.21.15.44:53600) 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11e9ebb640 1 -- 172.21.15.44:0/4169334457 learned_addr learned my addr 172.21.15.44:0/4169334457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.871+0000 7f11e9ebb640 1 -- 172.21.15.44:0/4169334457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e41089e0 con 0x7f11e41060b0 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.872+0000 7f11e9ebb640 1 --2- 172.21.15.44:0/4169334457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e41084a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009820 tx=0x7f11e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d5b567039d8c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.872+0000 7f11e8eb9640 1 -- 172.21.15.44:0/4169334457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e002ae70 con 0x7f11e41060b0 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.872+0000 7f11e8eb9640 1 -- 172.21.15.44:0/4169334457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11e00328d0 con 0x7f11e41060b0 2024-09-13T17:17:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.872+0000 7f11e8eb9640 1 -- 172.21.15.44:0/4169334457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e00307a0 con 0x7f11e41060b0 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 -- 172.21.15.44:0/4169334457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 msgr2=0x7f11e41084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 --2- 172.21.15.44:0/4169334457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e41084a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009820 tx=0x7f11e002a270 comp rx=0 tx=0).stop 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 -- 172.21.15.44:0/4169334457 shutdown_connections 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 --2- 172.21.15.44:0/4169334457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e41084a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 -- 172.21.15.44:0/4169334457 >> 172.21.15.44:0/4169334457 conn(0x7f11e40ffcf0 msgr2=0x7f11e4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 -- 172.21.15.44:0/4169334457 shutdown_connections 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 -- 172.21.15.44:0/4169334457 wait complete. 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.873+0000 7f11ec11f640 1 Processor -- start 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11ec11f640 1 -- start start 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11ec11f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11ec11f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4199ab0 con 0x7f11e41060b0 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11e9ebb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11e9ebb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e4199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53604/0 (socket says 172.21.15.44:53604) 2024-09-13T17:17:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11e9ebb640 1 -- 172.21.15.44:0/1402467979 learned_addr learned my addr 172.21.15.44:0/1402467979 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11e9ebb640 1 -- 172.21.15.44:0/1402467979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00094d0 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11e9ebb640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e4199570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11e00097f0 tx=0x7f11e0032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e0030c70 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11e0030dd0 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e4199cb0 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.874+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e003a5b0 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.875+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e419a150 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.875+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f11e0039040 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.875+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e4073df0 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.876+0000 7f11daffd640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f11d0038780 0x7f11d003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.876+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f11e006bb10 con 0x7f11e41060b0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.876+0000 7f11e96ba640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f11d0038780 0x7f11d003ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.876+0000 7f11e96ba640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f11d0038780 0x7f11d003ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098c0 tx=0x7f11d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:44.880+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f11e003ae60 con 0x7f11e41060b0 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.079+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f11e4061900 con 0x7f11d0038780 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.332+0000 7f11daffd640 1 -- 172.21.15.44:0/1402467979 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f11e4061900 con 0x7f11d0038780 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f11d0038780 msgr2=0x7f11d003ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f11d0038780 0x7f11d003ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098c0 tx=0x7f11d4006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 msgr2=0x7f11e4199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e4199570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11e00097f0 tx=0x7f11e0032c70 comp rx=0 tx=0).stop 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 shutdown_connections 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f11d0038780 0x7f11d003ac40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 --2- 172.21.15.44:0/1402467979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41060b0 0x7f11e4199570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 >> 172.21.15.44:0/1402467979 conn(0x7f11e40ffcf0 msgr2=0x7f11e4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 shutdown_connections 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:45.334+0000 7f11ec11f640 1 -- 172.21.15.44:0/1402467979 wait complete. 2024-09-13T17:17:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:Fetching dashboard port number... 2024-09-13T17:17:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:46 smithi044 ceph-mon[31175]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:47.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:47 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1740194140' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 8443 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.265+0000 7fab1a4a5640 1 Processor -- start 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.265+0000 7fab1a4a5640 1 -- start start 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.265+0000 7fab1a4a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab14105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.266+0000 7fab1a4a5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14106010 con 0x7fab14105640 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.266+0000 7fab13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab14105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.056 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.266+0000 7fab13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab14105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53608/0 (socket says 172.21.15.44:53608) 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.266+0000 7fab13fff640 1 -- 172.21.15.44:0/103896679 learned_addr learned my addr 172.21.15.44:0/103896679 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.266+0000 7fab13fff640 1 -- 172.21.15.44:0/103896679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14106840 con 0x7fab14105640 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.267+0000 7fab13fff640 1 --2- 172.21.15.44:0/103896679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab14105a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab04009820 tx=0x7fab0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b467ec8f55bc54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.267+0000 7fab12ffd640 1 -- 172.21.15.44:0/103896679 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0402ae70 con 0x7fab14105640 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.267+0000 7fab12ffd640 1 -- 172.21.15.44:0/103896679 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab040328d0 con 0x7fab14105640 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.267+0000 7fab12ffd640 1 -- 172.21.15.44:0/103896679 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab040307a0 con 0x7fab14105640 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 -- 172.21.15.44:0/103896679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 msgr2=0x7fab14105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 --2- 172.21.15.44:0/103896679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab14105a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab04009820 tx=0x7fab0402a270 comp rx=0 tx=0).stop 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 -- 172.21.15.44:0/103896679 shutdown_connections 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 --2- 172.21.15.44:0/103896679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab14105a40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 -- 172.21.15.44:0/103896679 >> 172.21.15.44:0/103896679 conn(0x7fab14100df0 msgr2=0x7fab14103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 -- 172.21.15.44:0/103896679 shutdown_connections 2024-09-13T17:17:48.057 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 -- 172.21.15.44:0/103896679 wait complete. 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 Processor -- start 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.268+0000 7fab1a4a5640 1 -- start start 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab1a4a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab1407bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab1a4a5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1407c0d0 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab1407bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab1407bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53620/0 (socket says 172.21.15.44:53620) 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab13fff640 1 -- 172.21.15.44:0/1740194140 learned_addr learned my addr 172.21.15.44:0/1740194140 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab13fff640 1 -- 172.21.15.44:0/1740194140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040094d0 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab13fff640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab1407bb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fab04030410 tx=0x7fab04030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab04032c50 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.269+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab04032db0 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.270+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1407a2c0 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.270+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0403a520 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.270+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1407a760 con 0x7fab14105640 2024-09-13T17:17:48.058 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.270+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fab04039040 con 0x7fab14105640 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.271+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab14105ac0 con 0x7fab14105640 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.271+0000 7fab117fa640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faafc038780 0x7faafc03ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.271+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fab0406b8f0 con 0x7fab14105640 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.271+0000 7fab137fe640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faafc038780 0x7faafc03ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.271+0000 7fab137fe640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faafc038780 0x7faafc03ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fab080098c0 tx=0x7fab08006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.275+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fab0404b710 con 0x7fab14105640 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.448+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fab14073df0 con 0x7fab14105640 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.448+0000 7fab117fa640 1 -- 172.21.15.44:0/1740194140 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fab04037030 con 0x7fab14105640 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.450+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faafc038780 msgr2=0x7faafc03ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.450+0000 7fab1a4a5640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faafc038780 0x7faafc03ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fab080098c0 tx=0x7fab08006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.450+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 msgr2=0x7fab1407bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.450+0000 7fab1a4a5640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab1407bb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fab04030410 tx=0x7fab04030d50 comp rx=0 tx=0).stop 2024-09-13T17:17:48.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.451+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 shutdown_connections 2024-09-13T17:17:48.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.451+0000 7fab1a4a5640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faafc038780 0x7faafc03ac40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.451+0000 7fab1a4a5640 1 --2- 172.21.15.44:0/1740194140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14105640 0x7fab1407bb90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.451+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 >> 172.21.15.44:0/1740194140 conn(0x7fab14100df0 msgr2=0x7fab141019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.451+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 shutdown_connections 2024-09-13T17:17:48.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:47.451+0000 7fab1a4a5640 1 -- 172.21.15.44:0/1740194140 wait complete. 2024-09-13T17:17:48.062 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T17:17:48.062 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-09-13T17:17:48.069 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T17:17:48.070 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-09-13T17:17:48.070 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-09-13T17:17:48.070 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-13T17:17:48.071 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph Dashboard is now available at: 2024-09-13T17:17:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: URL: https://smithi044.front.sepia.ceph.com:8443/ 2024-09-13T17:17:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: User: admin 2024-09-13T17:17:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: Password: 3khdygcqm4 2024-09-13T17:17:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:48.072 INFO:teuthology.orchestra.run.smithi044.stdout:Saving cluster configuration to /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config directory 2024-09-13T17:17:48.072 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling autotune for osd_memory_target 2024-09-13T17:17:48.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:48 smithi044 ceph-mon[31175]: mgrmap e13: x(active, since 6s) 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.489+0000 7efe97b34640 1 Processor -- start 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.489+0000 7efe97b34640 1 -- start start 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.490+0000 7efe97b34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe901084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.490+0000 7efe97b34640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90079ed0 con 0x7efe901060b0 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.490+0000 7efe958d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe901084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.490+0000 7efe958d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe901084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53632/0 (socket says 172.21.15.44:53632) 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.490+0000 7efe958d0640 1 -- 172.21.15.44:0/2606317291 learned_addr learned my addr 172.21.15.44:0/2606317291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.490+0000 7efe958d0640 1 -- 172.21.15.44:0/2606317291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe901089e0 con 0x7efe901060b0 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe958d0640 1 --2- 172.21.15.44:0/2606317291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe901084a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009820 tx=0x7efe8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fcaad0e872b8f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe948ce640 1 -- 172.21.15.44:0/2606317291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe8c02ae70 con 0x7efe901060b0 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe948ce640 1 -- 172.21.15.44:0/2606317291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe8c0328d0 con 0x7efe901060b0 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe948ce640 1 -- 172.21.15.44:0/2606317291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe8c0307a0 con 0x7efe901060b0 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe97b34640 1 -- 172.21.15.44:0/2606317291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 msgr2=0x7efe901084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe97b34640 1 --2- 172.21.15.44:0/2606317291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe901084a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009820 tx=0x7efe8c02a270 comp rx=0 tx=0).stop 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe97b34640 1 -- 172.21.15.44:0/2606317291 shutdown_connections 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.491+0000 7efe97b34640 1 --2- 172.21.15.44:0/2606317291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe901084a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 -- 172.21.15.44:0/2606317291 >> 172.21.15.44:0/2606317291 conn(0x7efe900ffcf0 msgr2=0x7efe90102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 -- 172.21.15.44:0/2606317291 shutdown_connections 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 -- 172.21.15.44:0/2606317291 wait complete. 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 Processor -- start 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 -- start start 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe90199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.492+0000 7efe97b34640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90199aa0 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe958d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe90199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe958d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe90199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53638/0 (socket says 172.21.15.44:53638) 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe958d0640 1 -- 172.21.15.44:0/3756024728 learned_addr learned my addr 172.21.15.44:0/3756024728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe958d0640 1 -- 172.21.15.44:0/3756024728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0094d0 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe958d0640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe90199560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe8c030410 tx=0x7efe8c030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe8c032c50 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe8c032db0 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe90199ca0 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe8c03a5b0 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.493+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9019a140 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.494+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7efe8c039040 con 0x7efe901060b0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.494+0000 7efe86ffd640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7efe7c038820 0x7efe7c03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.494+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe90073df0 con 0x7efe901060b0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.494+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe8c06ba10 con 0x7efe901060b0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.494+0000 7efe950cf640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7efe7c038820 0x7efe7c03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.495+0000 7efe950cf640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7efe7c038820 0x7efe7c03ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe800098c0 tx=0x7efe80006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.499+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efe8c03f080 con 0x7efe901060b0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.670+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7efe901097e0 con 0x7efe901060b0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.670+0000 7efe86ffd640 1 -- 172.21.15.44:0/3756024728 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7efe8c037020 con 0x7efe901060b0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7efe7c038820 msgr2=0x7efe7c03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7efe7c038820 0x7efe7c03ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe800098c0 tx=0x7efe80006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 msgr2=0x7efe90199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe90199560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe8c030410 tx=0x7efe8c030d50 comp rx=0 tx=0).stop 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 shutdown_connections 2024-09-13T17:17:49.266 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7efe7c038820 0x7efe7c03ace0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.266 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 --2- 172.21.15.44:0/3756024728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe901060b0 0x7efe90199560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.266 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 >> 172.21.15.44:0/3756024728 conn(0x7efe900ffcf0 msgr2=0x7efe901008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.266 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 shutdown_connections 2024-09-13T17:17:49.266 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:48.672+0000 7efe97b34640 1 -- 172.21.15.44:0/3756024728 wait complete. 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac80da4640 1 Processor -- start 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac80da4640 1 -- start start 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac80da4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c108520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac80da4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c079e50 con 0x7fac7c106130 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c108520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c108520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48944/0 (socket says 172.21.15.44:48944) 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.668+0000 7fac7a59c640 1 -- 172.21.15.44:0/3297777942 learned_addr learned my addr 172.21.15.44:0/3297777942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.669+0000 7fac7a59c640 1 -- 172.21.15.44:0/3297777942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c108a60 con 0x7fac7c106130 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.669+0000 7fac7a59c640 1 --2- 172.21.15.44:0/3297777942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c108520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009820 tx=0x7fac6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650c7291943b4274 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:50.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac79d9b640 1 -- 172.21.15.44:0/3297777942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac6c02ae70 con 0x7fac7c106130 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac79d9b640 1 -- 172.21.15.44:0/3297777942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac6c0328d0 con 0x7fac7c106130 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac79d9b640 1 -- 172.21.15.44:0/3297777942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac6c0307a0 con 0x7fac7c106130 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 -- 172.21.15.44:0/3297777942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 msgr2=0x7fac7c108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 --2- 172.21.15.44:0/3297777942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c108520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009820 tx=0x7fac6c02a270 comp rx=0 tx=0).stop 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 -- 172.21.15.44:0/3297777942 shutdown_connections 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 --2- 172.21.15.44:0/3297777942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c108520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 -- 172.21.15.44:0/3297777942 >> 172.21.15.44:0/3297777942 conn(0x7fac7c0ffcf0 msgr2=0x7fac7c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 -- 172.21.15.44:0/3297777942 shutdown_connections 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.670+0000 7fac80da4640 1 -- 172.21.15.44:0/3297777942 wait complete. 2024-09-13T17:17:50.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.671+0000 7fac80da4640 1 Processor -- start 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.671+0000 7fac80da4640 1 -- start start 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.671+0000 7fac80da4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.671+0000 7fac80da4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c199aa0 con 0x7fac7c106130 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48956/0 (socket says 172.21.15.44:48956) 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac7a59c640 1 -- 172.21.15.44:0/1844584661 learned_addr learned my addr 172.21.15.44:0/1844584661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac7a59c640 1 -- 172.21.15.44:0/1844584661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0094d0 con 0x7fac7c106130 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac7a59c640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c199560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac6c030410 tx=0x7fac6c030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac6c032c50 con 0x7fac7c106130 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac6c032db0 con 0x7fac7c106130 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c199ca0 con 0x7fac7c106130 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac6c03a5b0 con 0x7fac7c106130 2024-09-13T17:17:50.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.672+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c19a140 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.673+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fac6c039040 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.673+0000 7fac737fe640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac68038820 0x7fac6803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.674+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac6c06bb80 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.674+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7c073df0 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.674+0000 7fac73fff640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac68038820 0x7fac6803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.674+0000 7fac73fff640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac68038820 0x7fac6803ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac640098c0 tx=0x7fac64006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.678+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fac6c037070 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.936+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fac7c109800 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.939+0000 7fac737fe640 1 -- 172.21.15.44:0/1844584661 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fac6c03f030 con 0x7fac7c106130 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac68038820 msgr2=0x7fac6803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:50.521 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac68038820 0x7fac6803ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac640098c0 tx=0x7fac64006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 msgr2=0x7fac7c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c199560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac6c030410 tx=0x7fac6c030d50 comp rx=0 tx=0).stop 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 shutdown_connections 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac68038820 0x7fac6803ace0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 --2- 172.21.15.44:0/1844584661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac7c106130 0x7fac7c199560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 >> 172.21.15.44:0/1844584661 conn(0x7fac7c0ffcf0 msgr2=0x7fac7c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 shutdown_connections 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-09-13T17:17:49.941+0000 7fac80da4640 1 -- 172.21.15.44:0/1844584661 wait complete. 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:Or, if you are only running a single cluster on this host: 2024-09-13T17:17:50.522 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: ceph telemetry on 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout:For more information see: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:17:50.523 INFO:teuthology.orchestra.run.smithi044.stdout:Bootstrap complete. 2024-09-13T17:17:50.554 INFO:tasks.cephadm:Fetching config... 2024-09-13T17:17:50.554 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:17:50.554 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-13T17:17:50.572 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-13T17:17:50.572 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:17:50.572 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-13T17:17:50.628 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-13T17:17:50.628 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:17:50.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-13T17:17:50.695 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-13T17:17:50.695 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:17:50.695 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-13T17:17:50.751 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-13T17:17:50.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T17:17:50.829 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:50.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T17:17:50.978 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:50.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T17:17:51.039 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeKrnLQfheyrff9dSCf+kM7dPz1ZHiaVo5NfgJehwdW1zDN8NsxGyxG2fsv59zi298ABMNdBSsogr5drfTexObuGDILx9ps0jx3WbEf8jfvpnVUDeJwf/ZjTqdDbUiTS47m0BeJvw3QmTJoEjx/qMCsepmq9KxeI5WeNZR9G4mDyBJzg2RThveaveCuQRDI0o9RYR4SlZ/njqIdTpDp0L/Geq3uYd7YyMFNocKuhYtRON7tYB+ozMAUy2ObiOrMIKSkxbTE+A0MAxkI8YGqj/HhIMDRhycHd1ovrcrdCbI2j9HQ1WHP8B3ffHnGO94PnFYH0Jo+ImHIN7Wz8uH8FxOez+6IOIUbJsHv8iOa2ySi7rLa2bDb2dVUlLm5lAiKh185d6W1B+mjGqPLSRkqnw5PROuddX/I7d/R6RDhmrI6cdQpQDGONeHVUHVgI4xtFANVeBrIYdeeJneUCB2M4nxFHVZZgEDYeRbYk5uWsEvzMYlD1jlgRzQn7xWWIaT47c= ceph-ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:17:51.116 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1844584661' entity='client.admin' 2024-09-13T17:17:51.120 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-13T17:17:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:17:53.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 -- 172.21.15.44:0/1308392538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 msgr2=0x7f5c700ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:53.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 --2- 172.21.15.44:0/1308392538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c700ffbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009880 tx=0x7f5c6002a5d0 comp rx=0 tx=0).stop 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 -- 172.21.15.44:0/1308392538 shutdown_connections 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 --2- 172.21.15.44:0/1308392538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c700ffbe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 -- 172.21.15.44:0/1308392538 >> 172.21.15.44:0/1308392538 conn(0x7f5c700faf90 msgr2=0x7f5c700fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 -- 172.21.15.44:0/1308392538 shutdown_connections 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 -- 172.21.15.44:0/1308392538 wait complete. 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 Processor -- start 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.789+0000 7f5c766dc640 1 -- start start 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c766dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c70194c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c766dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70195140 con 0x7f5c700ff7e0 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c70194c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c70194c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48988/0 (socket says 172.21.15.44:48988) 2024-09-13T17:17:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c6ffff640 1 -- 172.21.15.44:0/3553210868 learned_addr learned my addr 172.21.15.44:0/3553210868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c6ffff640 1 -- 172.21.15.44:0/3553210868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009560 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c6ffff640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c70194c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005230 tx=0x7f5c60005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.790+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c60004b70 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.791+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c60003a80 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.791+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c6003c970 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.791+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c70195340 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.791+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c701957e0 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.792+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f5c60003bf0 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.792+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c700ffc60 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.792+0000 7f5c6dffb640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5c5c038710 0x7f5c5c03abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.792+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c6006c470 con 0x7f5c700ff7e0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.792+0000 7f5c67fff640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5c5c038710 0x7f5c5c03abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.793+0000 7f5c67fff640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5c5c038710 0x7f5c5c03abd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098c0 tx=0x7f5c58006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.797+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5c6004d530 con 0x7f5c700ff7e0 2024-09-13T17:17:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.973+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5c7018e3b0 con 0x7f5c700ff7e0 2024-09-13T17:17:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.976+0000 7f5c6dffb640 1 -- 172.21.15.44:0/3553210868 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5c60035020 con 0x7f5c700ff7e0 2024-09-13T17:17:53.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5c5c038710 msgr2=0x7f5c5c03abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:53.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5c5c038710 0x7f5c5c03abd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098c0 tx=0x7f5c58006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 msgr2=0x7f5c70194c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c70194c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005230 tx=0x7f5c60005260 comp rx=0 tx=0).stop 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 shutdown_connections 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5c5c038710 0x7f5c5c03abd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 --2- 172.21.15.44:0/3553210868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c700ff7e0 0x7f5c70194c00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 >> 172.21.15.44:0/3553210868 conn(0x7f5c700faf90 msgr2=0x7f5c700fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 shutdown_connections 2024-09-13T17:17:53.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:53.978+0000 7f5c766dc640 1 -- 172.21.15.44:0/3553210868 wait complete. 2024-09-13T17:17:54.753 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-13T17:17:54.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-13T17:17:55.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:54 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3553210868' entity='client.admin' 2024-09-13T17:17:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:17:56.319 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:56.319 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:56.319 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:17:56.319 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:56.319 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:56.319 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:56.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 -- 172.21.15.44:0/2115609248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 msgr2=0x7f283c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 --2- 172.21.15.44:0/2115609248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c0ffbe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f282c009880 tx=0x7f282c02a5d0 comp rx=0 tx=0).stop 2024-09-13T17:17:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 -- 172.21.15.44:0/2115609248 shutdown_connections 2024-09-13T17:17:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 --2- 172.21.15.44:0/2115609248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c0ffbe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 -- 172.21.15.44:0/2115609248 >> 172.21.15.44:0/2115609248 conn(0x7f283c0faf90 msgr2=0x7f283c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 -- 172.21.15.44:0/2115609248 shutdown_connections 2024-09-13T17:17:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.610+0000 7f2840fbd640 1 -- 172.21.15.44:0/2115609248 wait complete. 2024-09-13T17:17:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f2840fbd640 1 Processor -- start 2024-09-13T17:17:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f2840fbd640 1 -- start start 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f2840fbd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f2840fbd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1950c0 con 0x7f283c0ff7e0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f283a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f283a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49012/0 (socket says 172.21.15.44:49012) 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.611+0000 7f283a59c640 1 -- 172.21.15.44:0/4118398483 learned_addr learned my addr 172.21.15.44:0/4118398483 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f283a59c640 1 -- 172.21.15.44:0/4118398483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009560 con 0x7f283c0ff7e0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f283a59c640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c194b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f282c005230 tx=0x7f282c005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f282c004b70 con 0x7f283c0ff7e0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f282c003a80 con 0x7f283c0ff7e0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c1952c0 con 0x7f283c0ff7e0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f282c03c970 con 0x7f283c0ff7e0 2024-09-13T17:17:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.612+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c195760 con 0x7f283c0ff7e0 2024-09-13T17:17:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.613+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f282c03cbe0 con 0x7f283c0ff7e0 2024-09-13T17:17:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.613+0000 7f2833fff640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2828038760 0x7f282803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.613+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f282c06c3f0 con 0x7f283c0ff7e0 2024-09-13T17:17:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.613+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c0ffc60 con 0x7f283c0ff7e0 2024-09-13T17:17:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.613+0000 7f2831d9b640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2828038760 0x7f282803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.614+0000 7f2831d9b640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2828038760 0x7f282803ac20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f28240098c0 tx=0x7f2824006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:56.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.618+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f282c035080 con 0x7f283c0ff7e0 2024-09-13T17:17:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.795+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f283c0608f0 con 0x7f2828038760 2024-09-13T17:17:56.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.798+0000 7f2833fff640 1 -- 172.21.15.44:0/4118398483 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f283c0608f0 con 0x7f2828038760 2024-09-13T17:17:56.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2828038760 msgr2=0x7f282803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:56.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2828038760 0x7f282803ac20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f28240098c0 tx=0x7f2824006db0 comp rx=0 tx=0).stop 2024-09-13T17:17:56.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 msgr2=0x7f283c194b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c194b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f282c005230 tx=0x7f282c005260 comp rx=0 tx=0).stop 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 shutdown_connections 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2828038760 0x7f282803ac20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 --2- 172.21.15.44:0/4118398483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f283c0ff7e0 0x7f283c194b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 >> 172.21.15.44:0/4118398483 conn(0x7f283c0faf90 msgr2=0x7f283c0fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 shutdown_connections 2024-09-13T17:17:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:56.801+0000 7f2840fbd640 1 -- 172.21.15.44:0/4118398483 wait complete. 2024-09-13T17:17:57.560 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi070 2024-09-13T17:17:57.560 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:17:57.560 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/etc/ceph/ceph.conf 2024-09-13T17:17:57.579 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:17:57.579 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:17:57.638 INFO:tasks.cephadm:Adding host smithi070 to orchestrator... 2024-09-13T17:17:57.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch host add smithi070 2024-09-13T17:17:57.839 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:17:58.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:17:58.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:58.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:17:58.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:17:58.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:17:58.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: Updating smithi044:/etc/ceph/ceph.conf 2024-09-13T17:17:58.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:17:58.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:17:58.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:17:58.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:58.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:58.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:17:59.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 -- 172.21.15.44:0/4170835609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 msgr2=0x7fa65c100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:59.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 --2- 172.21.15.44:0/4170835609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c100db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009880 tx=0x7fa64c02a670 comp rx=0 tx=0).stop 2024-09-13T17:17:59.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 -- 172.21.15.44:0/4170835609 shutdown_connections 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 --2- 172.21.15.44:0/4170835609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c100db0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 -- 172.21.15.44:0/4170835609 >> 172.21.15.44:0/4170835609 conn(0x7fa65c075490 msgr2=0x7fa65c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 -- 172.21.15.44:0/4170835609 shutdown_connections 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.293+0000 7fa663024640 1 -- 172.21.15.44:0/4170835609 wait complete. 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa663024640 1 Processor -- start 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa663024640 1 -- start start 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa663024640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa663024640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c100540 con 0x7fa65c1009d0 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa660dc0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa660dc0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c0fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49028/0 (socket says 172.21.15.44:49028) 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa660dc0640 1 -- 172.21.15.44:0/1645839322 learned_addr learned my addr 172.21.15.44:0/1645839322 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.294+0000 7fa660dc0640 1 -- 172.21.15.44:0/1645839322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c009560 con 0x7fa65c1009d0 2024-09-13T17:17:59.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.295+0000 7fa660dc0640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c0fe630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005ab0 tx=0x7fa64c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.295+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa64c005440 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.295+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa64c0055a0 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.295+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c0feb70 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.295+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa64c03c740 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.295+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c0fef90 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.296+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa64c0035f0 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.296+0000 7fa659ffb640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa648038710 0x7fa64803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.296+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa64c06c3a0 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.296+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c106b50 con 0x7fa65c1009d0 2024-09-13T17:17:59.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.296+0000 7fa65bfff640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa648038710 0x7fa64803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:59.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.297+0000 7fa65bfff640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa648038710 0x7fa64803abd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa6440098c0 tx=0x7fa644006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.301+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa64c035080 con 0x7fa65c1009d0 2024-09-13T17:17:59.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:17:59.476+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi070", "target": ["mon-mgr", ""]}) v1 -- 0x7fa65c0ffbd0 con 0x7fa648038710 2024-09-13T17:17:59.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:17:59 smithi044 ceph-mon[31175]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:02.781 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:02 smithi044 ceph-mon[31175]: Deploying cephadm binary to smithi070 2024-09-13T17:18:02.781 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:02 smithi044 ceph-mon[31175]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.779+0000 7fa659ffb640 1 -- 172.21.15.44:0/1645839322 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fa65c0ffbd0 con 0x7fa648038710 2024-09-13T17:18:02.781 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi070' with addr '172.21.15.70' 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa648038710 msgr2=0x7fa64803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa648038710 0x7fa64803abd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa6440098c0 tx=0x7fa644006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 msgr2=0x7fa65c0fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c0fe630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005ab0 tx=0x7fa64c005e70 comp rx=0 tx=0).stop 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 shutdown_connections 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa648038710 0x7fa64803abd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 --2- 172.21.15.44:0/1645839322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65c1009d0 0x7fa65c0fe630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 >> 172.21.15.44:0/1645839322 conn(0x7fa65c075490 msgr2=0x7fa65c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 shutdown_connections 2024-09-13T17:18:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:02.782+0000 7fa663024640 1 -- 172.21.15.44:0/1645839322 wait complete. 2024-09-13T17:18:03.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T17:18:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:03.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:03.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:03 smithi044 ceph-mon[31175]: Added host smithi070 2024-09-13T17:18:03.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:03.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:03.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:03.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:03 smithi044 ceph-mon[31175]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:05.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.081+0000 7fcec7841640 1 -- 172.21.15.44:0/3204726977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 msgr2=0x7fcec0100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.081+0000 7fcec7841640 1 --2- 172.21.15.44:0/3204726977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0100cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009880 tx=0x7fcebc02a670 comp rx=0 tx=0).stop 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 -- 172.21.15.44:0/3204726977 shutdown_connections 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 --2- 172.21.15.44:0/3204726977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0100cf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 -- 172.21.15.44:0/3204726977 >> 172.21.15.44:0/3204726977 conn(0x7fcec0075490 msgr2=0x7fcec00fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 -- 172.21.15.44:0/3204726977 shutdown_connections 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 -- 172.21.15.44:0/3204726977 wait complete. 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 Processor -- start 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.082+0000 7fcec7841640 1 -- start start 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec7841640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec7841640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0195550 con 0x7fcec0100910 2024-09-13T17:18:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec55dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec55dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0195010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58032/0 (socket says 172.21.15.44:58032) 2024-09-13T17:18:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec55dd640 1 -- 172.21.15.44:0/1436924272 learned_addr learned my addr 172.21.15.44:0/1436924272 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec55dd640 1 -- 172.21.15.44:0/1436924272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc009560 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec55dd640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0195010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0099a0 tx=0x7fcebc004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcebc0053a0 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec0195750 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcebc005500 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec0195b70 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.083+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcebc03c7c0 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.084+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fcebc03ca20 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.084+0000 7fceb67fc640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceac038710 0x7fceac03abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.085+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcebc06d1c0 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.085+0000 7fcec4ddc640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceac038710 0x7fceac03abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.085+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec0106a90 con 0x7fcec0100910 2024-09-13T17:18:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.085+0000 7fcec4ddc640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceac038710 0x7fceac03abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fceb00098c0 tx=0x7fceb0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.089+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcebc040080 con 0x7fcec0100910 2024-09-13T17:18:05.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.264+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcec01963e0 con 0x7fceac038710 2024-09-13T17:18:05.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.265+0000 7fceb67fc640 1 -- 172.21.15.44:0/1436924272 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fcec01963e0 con 0x7fceac038710 2024-09-13T17:18:05.266 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:18:05.266 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}] 2024-09-13T17:18:05.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceac038710 msgr2=0x7fceac03abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceac038710 0x7fceac03abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fceb00098c0 tx=0x7fceb0006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 msgr2=0x7fcec0195010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0195010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0099a0 tx=0x7fcebc004e50 comp rx=0 tx=0).stop 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 shutdown_connections 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceac038710 0x7fceac03abd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 --2- 172.21.15.44:0/1436924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0100910 0x7fcec0195010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.267+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 >> 172.21.15.44:0/1436924272 conn(0x7fcec0075490 msgr2=0x7fcec01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.268+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 shutdown_connections 2024-09-13T17:18:05.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:05.268+0000 7fcec7841640 1 -- 172.21.15.44:0/1436924272 wait complete. 2024-09-13T17:18:05.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:06.000 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-09-13T17:18:06.000 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:18:06.001 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-09-13T17:18:06.020 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:18:06.020 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:18:06.078 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-09-13T17:18:06.079 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch host add smithi100 2024-09-13T17:18:06.279 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:06 smithi044 ceph-mon[31175]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:18:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:06 smithi044 ceph-mon[31175]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 -- 172.21.15.44:0/4009844225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 msgr2=0x7f85c8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 --2- 172.21.15.44:0/4009844225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8104a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009c20 tx=0x7f85b802aa90 comp rx=0 tx=0).stop 2024-09-13T17:18:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 -- 172.21.15.44:0/4009844225 shutdown_connections 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 --2- 172.21.15.44:0/4009844225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8104a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 -- 172.21.15.44:0/4009844225 >> 172.21.15.44:0/4009844225 conn(0x7f85c8075490 msgr2=0x7f85c80fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 -- 172.21.15.44:0/4009844225 shutdown_connections 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.869+0000 7f85d0277640 1 -- 172.21.15.44:0/4009844225 wait complete. 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85d0277640 1 Processor -- start 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85d0277640 1 -- start start 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85d0277640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85d0277640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81954d0 con 0x7f85c8104650 2024-09-13T17:18:07.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85ce013640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85ce013640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58058/0 (socket says 172.21.15.44:58058) 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85ce013640 1 -- 172.21.15.44:0/679039374 learned_addr learned my addr 172.21.15.44:0/679039374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85ce013640 1 -- 172.21.15.44:0/679039374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8009900 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85ce013640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8194f90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005e30 tx=0x7f85b802e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.870+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85b803e070 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.871+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c81956d0 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.871+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85b8003dc0 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.871+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85b8004700 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.871+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c8195b50 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.871+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f85b800a040 con 0x7f85c8104650 2024-09-13T17:18:07.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.872+0000 7f85c77fe640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85b0038710 0x7f85b003abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:07.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.872+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c80ffd90 con 0x7f85c8104650 2024-09-13T17:18:07.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.872+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f85b806c640 con 0x7f85c8104650 2024-09-13T17:18:07.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.872+0000 7f85cd812640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85b0038710 0x7f85b003abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:07.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.872+0000 7f85cd812640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85b0038710 0x7f85b003abd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0098c0 tx=0x7f85bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:07.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:07.876+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f85b8004860 con 0x7f85c8104650 2024-09-13T17:18:08.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:08.052+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7f85c8108870 con 0x7f85b0038710 2024-09-13T17:18:08.053 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:08.053 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:08.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:09.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:08 smithi044 ceph-mon[31175]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:10.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:09 smithi044 ceph-mon[31175]: Deploying cephadm binary to smithi100 2024-09-13T17:18:10.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:09 smithi044 ceph-mon[31175]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:11.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.337+0000 7f85c77fe640 1 -- 172.21.15.44:0/679039374 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f85c8108870 con 0x7f85b0038710 2024-09-13T17:18:11.339 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-09-13T17:18:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85b0038710 msgr2=0x7f85b003abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85b0038710 0x7f85b003abd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0098c0 tx=0x7f85bc006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 msgr2=0x7f85c8194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8194f90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005e30 tx=0x7f85b802e960 comp rx=0 tx=0).stop 2024-09-13T17:18:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 shutdown_connections 2024-09-13T17:18:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85b0038710 0x7f85b003abd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:11.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 --2- 172.21.15.44:0/679039374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85c8104650 0x7f85c8194f90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:11.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 >> 172.21.15.44:0/679039374 conn(0x7f85c8075490 msgr2=0x7f85c80fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:11.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 shutdown_connections 2024-09-13T17:18:11.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:11.340+0000 7f85d0277640 1 -- 172.21.15.44:0/679039374 wait complete. 2024-09-13T17:18:11.895 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T17:18:12.093 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:12 smithi044 ceph-mon[31175]: Added host smithi100 2024-09-13T17:18:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:12 smithi044 ceph-mon[31175]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:12.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 -- 172.21.15.44:0/2634073628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 msgr2=0x7f3e8c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 --2- 172.21.15.44:0/2634073628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c105b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009880 tx=0x7f3e7c02a5d0 comp rx=0 tx=0).stop 2024-09-13T17:18:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 -- 172.21.15.44:0/2634073628 shutdown_connections 2024-09-13T17:18:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 --2- 172.21.15.44:0/2634073628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c105b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 -- 172.21.15.44:0/2634073628 >> 172.21.15.44:0/2634073628 conn(0x7f3e8c076660 msgr2=0x7f3e8c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 -- 172.21.15.44:0/2634073628 shutdown_connections 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.659+0000 7f3e92570640 1 -- 172.21.15.44:0/2634073628 wait complete. 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e92570640 1 Processor -- start 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e92570640 1 -- start start 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e92570640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e92570640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0746c0 con 0x7f3e8c105770 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e8bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e8bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48108/0 (socket says 172.21.15.44:48108) 2024-09-13T17:18:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.660+0000 7f3e8bfff640 1 -- 172.21.15.44:0/2369885925 learned_addr learned my addr 172.21.15.44:0/2369885925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e8bfff640 1 -- 172.21.15.44:0/2369885925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c009560 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e8bfff640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c0727b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005ab0 tx=0x7f3e7c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e7c005440 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c072cf0 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e7c0055a0 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e7c03c740 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.661+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c073170 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.662+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f3e7c03ca00 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.662+0000 7f3e89ffb640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e78038710 0x7f3e7803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.662+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c0fd360 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.662+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e7c06c3d0 con 0x7f3e8c105770 2024-09-13T17:18:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.662+0000 7f3e83fff640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e78038710 0x7f3e7803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.663+0000 7f3e83fff640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e78038710 0x7f3e7803abd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098c0 tx=0x7f3e74006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:13.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.667+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3e7c04d350 con 0x7f3e8c105770 2024-09-13T17:18:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.842+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e8c073420 con 0x7f3e78038710 2024-09-13T17:18:13.844 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:13 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:13.844 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:13 smithi044 ceph-mon[31175]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.843+0000 7f3e89ffb640 1 -- 172.21.15.44:0/2369885925 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f3e8c073420 con 0x7f3e78038710 2024-09-13T17:18:13.844 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:18:13.845 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-09-13T17:18:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e78038710 msgr2=0x7f3e7803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e78038710 0x7f3e7803abd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098c0 tx=0x7f3e74006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 msgr2=0x7f3e8c0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c0727b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005ab0 tx=0x7f3e7c005e70 comp rx=0 tx=0).stop 2024-09-13T17:18:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 shutdown_connections 2024-09-13T17:18:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e78038710 0x7f3e7803abd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 --2- 172.21.15.44:0/2369885925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c105770 0x7f3e8c0727b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 >> 172.21.15.44:0/2369885925 conn(0x7f3e8c076660 msgr2=0x7f3e8c0fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.845+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 shutdown_connections 2024-09-13T17:18:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:13.846+0000 7f3e92570640 1 -- 172.21.15.44:0/2369885925 wait complete. 2024-09-13T17:18:14.432 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-13T17:18:14.433 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-13T17:18:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:14.753 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:14 smithi044 ceph-mon[31175]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:18:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.271+0000 7fef46f2b640 1 -- 172.21.15.44:0/261621295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 msgr2=0x7fef40105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.271+0000 7fef46f2b640 1 --2- 172.21.15.44:0/261621295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40105c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009880 tx=0x7fef3c02a5d0 comp rx=0 tx=0).stop 2024-09-13T17:18:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.271+0000 7fef46f2b640 1 -- 172.21.15.44:0/261621295 shutdown_connections 2024-09-13T17:18:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.271+0000 7fef46f2b640 1 --2- 172.21.15.44:0/261621295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40105c10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.271+0000 7fef46f2b640 1 -- 172.21.15.44:0/261621295 >> 172.21.15.44:0/261621295 conn(0x7fef40076500 msgr2=0x7fef40076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef46f2b640 1 -- 172.21.15.44:0/261621295 shutdown_connections 2024-09-13T17:18:16.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef46f2b640 1 -- 172.21.15.44:0/261621295 wait complete. 2024-09-13T17:18:16.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef46f2b640 1 Processor -- start 2024-09-13T17:18:16.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef46f2b640 1 -- start start 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef46f2b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40194ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef46f2b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40195400 con 0x7fef40105830 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.272+0000 7fef44cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40194ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef44cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40194ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48136/0 (socket says 172.21.15.44:48136) 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef44cc7640 1 -- 172.21.15.44:0/4126406953 learned_addr learned my addr 172.21.15.44:0/4126406953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef44cc7640 1 -- 172.21.15.44:0/4126406953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c009560 con 0x7fef40105830 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef44cc7640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40194ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef3c005ab0 tx=0x7fef3c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef3c005440 con 0x7fef40105830 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef3c0055a0 con 0x7fef40105830 2024-09-13T17:18:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef40195660 con 0x7fef40105830 2024-09-13T17:18:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef3c03c740 con 0x7fef40105830 2024-09-13T17:18:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.273+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef40195b60 con 0x7fef40105830 2024-09-13T17:18:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.274+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fef3c03ca00 con 0x7fef40105830 2024-09-13T17:18:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.275+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef40100ed0 con 0x7fef40105830 2024-09-13T17:18:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.275+0000 7fef35ffb640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fef28038710 0x7fef2803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.275+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef3c06c250 con 0x7fef40105830 2024-09-13T17:18:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.275+0000 7fef37fff640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fef28038710 0x7fef2803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.275+0000 7fef37fff640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fef28038710 0x7fef2803abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef300098c0 tx=0x7fef30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:16.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.279+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fef3c03a720 con 0x7fef40105830 2024-09-13T17:18:16.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:16.457+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fef40195b60 con 0x7fef40105830 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:16.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:17.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.347+0000 7fef35ffb640 1 -- 172.21.15.44:0/4126406953 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fef40195b60 con 0x7fef40105830 2024-09-13T17:18:17.349 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-09-13T17:18:17.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fef28038710 msgr2=0x7fef2803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:17.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fef28038710 0x7fef2803abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef300098c0 tx=0x7fef30006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:17.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 msgr2=0x7fef40194ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40194ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef3c005ab0 tx=0x7fef3c005e70 comp rx=0 tx=0).stop 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 shutdown_connections 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fef28038710 0x7fef2803abd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 --2- 172.21.15.44:0/4126406953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef40105830 0x7fef40194ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 >> 172.21.15.44:0/4126406953 conn(0x7fef40076500 msgr2=0x7fef400fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 shutdown_connections 2024-09-13T17:18:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:17.350+0000 7fef46f2b640 1 -- 172.21.15.44:0/4126406953 wait complete. 2024-09-13T17:18:17.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:17 smithi044 ceph-mon[31175]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:17.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:17 smithi044 ceph-mon[31175]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:18:17.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:17 smithi044 ceph-mon[31175]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:18:17.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:17 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4126406953' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-13T17:18:17.959 INFO:tasks.cephadm:Adding mon.a on smithi044 2024-09-13T17:18:17.960 INFO:tasks.cephadm:Adding mon.b on smithi070 2024-09-13T17:18:17.960 INFO:tasks.cephadm:Adding mon.c on smithi100 2024-09-13T17:18:17.960 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi044:172.21.15.44=a;smithi070:172.21.15.70=b;smithi100:172.21.15.100=c' 2024-09-13T17:18:18.158 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:18 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4126406953' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T17:18:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:18 smithi044 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:18 smithi044 ceph-mon[31175]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.544+0000 7fbec7278640 1 -- 172.21.15.100:0/3837541041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 msgr2=0x7fbec0072ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.544+0000 7fbec7278640 1 --2- 172.21.15.100:0/3837541041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec0072ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009880 tx=0x7fbebc02a5d0 comp rx=0 tx=0).stop 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.544+0000 7fbec7278640 1 -- 172.21.15.100:0/3837541041 shutdown_connections 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.544+0000 7fbec7278640 1 --2- 172.21.15.100:0/3837541041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec0072ac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.544+0000 7fbec7278640 1 -- 172.21.15.100:0/3837541041 >> 172.21.15.100:0/3837541041 conn(0x7fbec0076500 msgr2=0x7fbec0076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.545+0000 7fbec7278640 1 -- 172.21.15.100:0/3837541041 shutdown_connections 2024-09-13T17:18:19.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.545+0000 7fbec7278640 1 -- 172.21.15.100:0/3837541041 wait complete. 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.545+0000 7fbec7278640 1 Processor -- start 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.545+0000 7fbec7278640 1 -- start start 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.545+0000 7fbec7278640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.545+0000 7fbec7278640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec01976f0 con 0x7fbec00726e0 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec5014640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec5014640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.100:60722/0 (socket says 172.21.15.100:60722) 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec5014640 1 -- 172.21.15.100:0/1356743801 learned_addr learned my addr 172.21.15.100:0/1356743801 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:18:19.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec5014640 1 -- 172.21.15.100:0/1356743801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc009560 con 0x7fbec00726e0 2024-09-13T17:18:19.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec5014640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec01971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0099a0 tx=0x7fbebc004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:19.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbebc0053a0 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbebc005500 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec01978f0 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbebc03c7c0 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.546+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec0197d10 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.547+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec01090e0 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.547+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fbebc0035f0 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.548+0000 7fbeb67fc640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbeac038710 0x7fbeac03abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.548+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbebc06c3f0 con 0x7fbec00726e0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.548+0000 7fbec4813640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbeac038710 0x7fbeac03abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:19.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.548+0000 7fbec4813640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbeac038710 0x7fbeac03abd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00098c0 tx=0x7fbeb0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:19.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.552+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbebc04d350 con 0x7fbec00726e0 2024-09-13T17:18:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.722+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi070:172.21.15.70=b;smithi100:172.21.15.100=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fbec010bbf0 con 0x7fbeac038710 2024-09-13T17:18:19.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.728+0000 7fbeb67fc640 1 -- 172.21.15.100:0/1356743801 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbec010bbf0 con 0x7fbeac038710 2024-09-13T17:18:19.730 INFO:teuthology.orchestra.run.smithi100.stdout:Scheduled mon update... 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.730+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbeac038710 msgr2=0x7fbeac03abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.730+0000 7fbec7278640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbeac038710 0x7fbeac03abd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00098c0 tx=0x7fbeb0006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.730+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 msgr2=0x7fbec01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.730+0000 7fbec7278640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec01971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0099a0 tx=0x7fbebc004e50 comp rx=0 tx=0).stop 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.731+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 shutdown_connections 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.731+0000 7fbec7278640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbeac038710 0x7fbeac03abd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.731+0000 7fbec7278640 1 --2- 172.21.15.100:0/1356743801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbec00726e0 0x7fbec01971b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.731+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 >> 172.21.15.100:0/1356743801 conn(0x7fbec0076500 msgr2=0x7fbec0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.731+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 shutdown_connections 2024-09-13T17:18:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:19.731+0000 7fbec7278640 1 -- 172.21.15.100:0/1356743801 wait complete. 2024-09-13T17:18:19.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:19 smithi044 ceph-mon[31175]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:20.297 DEBUG:teuthology.orchestra.run.smithi070:mon.b> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.b.service 2024-09-13T17:18:20.300 DEBUG:teuthology.orchestra.run.smithi100:mon.c> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.c.service 2024-09-13T17:18:20.302 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T17:18:20.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T17:18:21.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi070:172.21.15.70=b;smithi100:172.21.15.100=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi070:172.21.15.70=b;smithi100:172.21.15.100=c;count:3 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:21.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:20 smithi044 ceph-mon[31175]: Deploying daemon mon.c on smithi100 2024-09-13T17:18:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:22.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:21 smithi044 ceph-mon[31175]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:23.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.578+0000 7f72e45c4640 1 -- 172.21.15.100:0/2514994332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 msgr2=0x7f72dc100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:23.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.578+0000 7f72e45c4640 1 --2- 172.21.15.100:0/2514994332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc100cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009880 tx=0x7f72d802a670 comp rx=0 tx=0).stop 2024-09-13T17:18:23.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.578+0000 7f72e45c4640 1 -- 172.21.15.100:0/2514994332 shutdown_connections 2024-09-13T17:18:23.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.578+0000 7f72e45c4640 1 --2- 172.21.15.100:0/2514994332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc100cf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:23.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.578+0000 7f72e45c4640 1 -- 172.21.15.100:0/2514994332 >> 172.21.15.100:0/2514994332 conn(0x7f72dc075490 msgr2=0x7f72dc0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:23.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.579+0000 7f72e45c4640 1 -- 172.21.15.100:0/2514994332 shutdown_connections 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.579+0000 7f72e45c4640 1 -- 172.21.15.100:0/2514994332 wait complete. 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.579+0000 7f72e45c4640 1 Processor -- start 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.579+0000 7f72e45c4640 1 -- start start 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.579+0000 7f72e45c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.579+0000 7f72e45c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc195550 con 0x7f72dc100910 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.580+0000 7f72e2360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.580+0000 7f72e2360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc195010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.100:60740/0 (socket says 172.21.15.100:60740) 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.580+0000 7f72e2360640 1 -- 172.21.15.100:0/559000979 learned_addr learned my addr 172.21.15.100:0/559000979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:18:23.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.580+0000 7f72e2360640 1 -- 172.21.15.100:0/559000979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8009560 con 0x7f72dc100910 2024-09-13T17:18:23.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.580+0000 7f72e2360640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc195010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72d80099a0 tx=0x7f72d8004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:23.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.580+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d80053a0 con 0x7f72dc100910 2024-09-13T17:18:23.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.581+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72d8005500 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.581+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc195750 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.581+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d803c7c0 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.581+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc195b70 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.582+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc106a90 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.582+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f72d80035f0 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.582+0000 7f72d37fe640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f72c8038760 0x7f72c803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.582+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f72d806c380 con 0x7f72dc100910 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.582+0000 7f72e1b5f640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f72c8038760 0x7f72c803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:23.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.583+0000 7f72e1b5f640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f72c8038760 0x7f72c803ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0098c0 tx=0x7f72cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:23.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.587+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f72d804d350 con 0x7f72dc100910 2024-09-13T17:18:23.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.836+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f72dc067f80 con 0x7f72dc100910 2024-09-13T17:18:23.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.836+0000 7f72d37fe640 1 -- 172.21.15.100:0/559000979 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f72d8035070 con 0x7f72dc100910 2024-09-13T17:18:23.838 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:18:23.838 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","modified":"2024-09-13T17:16:51.854882Z","created":"2024-09-13T17:16:51.854882Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-13T17:18:23.838 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f72c8038760 msgr2=0x7f72c803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f72c8038760 0x7f72c803ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0098c0 tx=0x7f72cc006db0 comp rx=0 tx=0).stop 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 msgr2=0x7f72dc195010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc195010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72d80099a0 tx=0x7f72d8004e50 comp rx=0 tx=0).stop 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 shutdown_connections 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f72c8038760 0x7f72c803ac20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 --2- 172.21.15.100:0/559000979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72dc100910 0x7f72dc195010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:23.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 >> 172.21.15.100:0/559000979 conn(0x7f72dc075490 msgr2=0x7f72dc1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:23.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 shutdown_connections 2024-09-13T17:18:23.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:23.839+0000 7f72e45c4640 1 -- 172.21.15.100:0/559000979 wait complete. 2024-09-13T17:18:23.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:23 smithi044 ceph-mon[31175]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:24.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:24 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/559000979' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T17:18:25.175 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:25 smithi100 ceph-mon[35055]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-13T17:18:25.175 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:25 smithi100 bash[35036]: 7b5c424e19e5a454de093fe54b23f85e0f2742b0d663f0bd7a86ebcf92d49c54 2024-09-13T17:18:25.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:25 smithi100 systemd[1]: Started Ceph mon.c for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:18:25.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T17:18:25.752 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T17:18:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:18:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.294+0000 7ff58b974640 1 -- 172.21.15.100:0/990596504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570003530 msgr2=0x7ff5700059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.294+0000 7ff58b974640 1 --2- 172.21.15.100:0/990596504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570003530 0x7ff5700059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff584067900 tx=0x7ff58002bce0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.295+0000 7ff58b974640 1 -- 172.21.15.100:0/990596504 shutdown_connections 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.295+0000 7ff58b974640 1 --2- 172.21.15.100:0/990596504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570003530 0x7ff5700059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.295+0000 7ff58b974640 1 -- 172.21.15.100:0/990596504 >> 172.21.15.100:0/990596504 conn(0x7ff5840f9c10 msgr2=0x7ff5840fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.295+0000 7ff58b974640 1 -- 172.21.15.100:0/990596504 shutdown_connections 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.295+0000 7ff58b974640 1 -- 172.21.15.100:0/990596504 wait complete. 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.295+0000 7ff58b974640 1 Processor -- start 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff58b974640 1 -- start start 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff58b974640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff58b974640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5840ff400 0x7ff5840ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff58b974640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584100dd0 con 0x7ff5840ff400 2024-09-13T17:18:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff58b974640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5841a1320 con 0x7ff570003530 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49662/0 (socket says 172.21.15.100:49662) 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 -- 172.21.15.100:0/2702683289 learned_addr learned my addr 172.21.15.100:0/2702683289 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff588f0f640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5840ff400 0x7ff5840ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 -- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 msgr2=0x7ff5840feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 -- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 msgr2=0x7ff5840feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:18:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff589710640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff588f0f640 1 -- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 msgr2=0x7ff5840feec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.296+0000 7ff588f0f640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff588f0f640 1 -- 172.21.15.100:0/2702683289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580011870 con 0x7ff5840ff400 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff588f0f640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5840ff400 0x7ff5840ff860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b3d0 tx=0x7ff57400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff574011630 con 0x7ff5840ff400 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff574011c20 con 0x7ff5840ff400 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5841a14c0 con 0x7ff5840ff400 2024-09-13T17:18:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff574010bf0 con 0x7ff5840ff400 2024-09-13T17:18:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.297+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5841a19e0 con 0x7ff5840ff400 2024-09-13T17:18:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.298+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7ff57401a400 con 0x7ff5840ff400 2024-09-13T17:18:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.298+0000 7ff57a7fc640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff568038550 0x7ff56803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.298+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff57404af30 con 0x7ff5840ff400 2024-09-13T17:18:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.298+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5841087c0 con 0x7ff5840ff400 2024-09-13T17:18:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.299+0000 7ff589710640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff568038550 0x7ff56803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.299+0000 7ff589710640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff568038550 0x7ff56803aa10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff580005940 tx=0x7ff5800058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.304+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff574010e60 con 0x7ff5840ff400 2024-09-13T17:18:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:18:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: mon.a calling monitor election 2024-09-13T17:18:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: mon.c calling monitor election 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: monmap e2: 2 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: fsmap 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: mgrmap e13: x(active, since 48s) 2024-09-13T17:18:30.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: overall HEALTH_OK 2024-09-13T17:18:30.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:30.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:30.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:30.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:30.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:30.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.529+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff584100fa0 con 0x7ff5840ff400 2024-09-13T17:18:30.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.529+0000 7ff57a7fc640 1 -- 172.21.15.100:0/2702683289 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7ff57403d020 con 0x7ff5840ff400 2024-09-13T17:18:30.531 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:18:30.531 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","modified":"2024-09-13T17:18:25.173076Z","created":"2024-09-13T17:16:51.854882Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T17:18:30.531 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 2 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff568038550 msgr2=0x7ff56803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff568038550 0x7ff56803aa10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff580005940 tx=0x7ff5800058b0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5840ff400 msgr2=0x7ff5840ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5840ff400 0x7ff5840ff860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b3d0 tx=0x7ff57400b890 comp rx=0 tx=0).stop 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 shutdown_connections 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff568038550 0x7ff56803aa10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5840ff400 0x7ff5840ff860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 --2- 172.21.15.100:0/2702683289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff570003530 0x7ff5840feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 >> 172.21.15.100:0/2702683289 conn(0x7ff5840f9c10 msgr2=0x7ff5840fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:30.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 shutdown_connections 2024-09-13T17:18:30.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:30.531+0000 7ff58b974640 1 -- 172.21.15.100:0/2702683289 wait complete. 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: mon.a calling monitor election 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: mon.c calling monitor election 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:30.534 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: monmap e2: 2 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: fsmap 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: mgrmap e13: x(active, since 48s) 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: overall HEALTH_OK 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:31 smithi100 ceph-mon[35055]: Deploying daemon mon.b on smithi070 2024-09-13T17:18:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:31 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/2702683289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T17:18:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:31 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:31.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:31 smithi044 ceph-mon[31175]: Deploying daemon mon.b on smithi070 2024-09-13T17:18:31.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:31 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/2702683289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T17:18:31.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:31 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:32.077 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T17:18:32.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T17:18:32.282 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:18:32.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:32 smithi100 ceph-mon[35055]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:32.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:32 smithi044 ceph-mon[31175]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:33.624 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:33 smithi100 ceph-mon[35055]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.798+0000 7fa645ac1640 1 -- 172.21.15.100:0/3404920265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 msgr2=0x7fa640104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.798+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3404920265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640104b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa630009880 tx=0x7fa63002a5d0 comp rx=0 tx=0).stop 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.798+0000 7fa645ac1640 1 -- 172.21.15.100:0/3404920265 shutdown_connections 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.798+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3404920265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640104b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.798+0000 7fa645ac1640 1 -- 172.21.15.100:0/3404920265 >> 172.21.15.100:0/3404920265 conn(0x7fa6400fa440 msgr2=0x7fa6400fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.798+0000 7fa645ac1640 1 -- 172.21.15.100:0/3404920265 shutdown_connections 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 -- 172.21.15.100:0/3404920265 wait complete. 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 Processor -- start 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 -- start start 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa640195bb0 con 0x7fa640104780 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.799+0000 7fa645ac1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa640195d20 con 0x7fa640195640 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49676/0 (socket says 172.21.15.100:49676) 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 -- 172.21.15.100:0/3920411157 learned_addr learned my addr 172.21.15.100:0/3920411157 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63f7fe640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 -- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 msgr2=0x7fa640199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 -- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 msgr2=0x7fa640199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:18:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:18:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63effd640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:18:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63f7fe640 1 -- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 msgr2=0x7fa640199a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:18:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63f7fe640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63f7fe640 1 -- 172.21.15.100:0/3920411157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630009560 con 0x7fa640104780 2024-09-13T17:18:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63f7fe640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640195100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa6300045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.800+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa6300052d0 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.801+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa640199f80 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.801+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa630005430 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.801+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa63003b460 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.801+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64019a4d0 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.802+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa6300035f0 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.802+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6400fc460 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.802+0000 7fa63cff9640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa62c0384f0 0x7fa62c03a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.802+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa63006bf70 con 0x7fa640104780 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.802+0000 7fa63effd640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa62c0384f0 0x7fa62c03a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.803+0000 7fa63effd640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa62c0384f0 0x7fa62c03a9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa628009100 tx=0x7fa628008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:33.807+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa63009fe20 con 0x7fa640104780 2024-09-13T17:18:33.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:33 smithi044 ceph-mon[31175]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:34.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.032+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa64019a840 con 0x7fa640104780 2024-09-13T17:18:34.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.032+0000 7fa63cff9640 1 -- 172.21.15.100:0/3920411157 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fa63003aa10 con 0x7fa640104780 2024-09-13T17:18:34.035 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:18:34.035 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","modified":"2024-09-13T17:18:25.173076Z","created":"2024-09-13T17:16:51.854882Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T17:18:34.035 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 2 2024-09-13T17:18:34.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa62c0384f0 msgr2=0x7fa62c03a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:34.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa62c0384f0 0x7fa62c03a9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa628009100 tx=0x7fa628008040 comp rx=0 tx=0).stop 2024-09-13T17:18:34.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 msgr2=0x7fa640195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:34.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640195100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa6300045e0 comp rx=0 tx=0).stop 2024-09-13T17:18:34.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 shutdown_connections 2024-09-13T17:18:34.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa62c0384f0 0x7fa62c03a9b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:34.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640195640 0x7fa640199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:34.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 --2- 172.21.15.100:0/3920411157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa640104780 0x7fa640195100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:34.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 >> 172.21.15.100:0/3920411157 conn(0x7fa6400fa440 msgr2=0x7fa6400fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:34.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 shutdown_connections 2024-09-13T17:18:34.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:34.035+0000 7fa645ac1640 1 -- 172.21.15.100:0/3920411157 wait complete. 2024-09-13T17:18:34.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:34 smithi070 systemd[1]: Started Ceph mon.b for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:18:35.540 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T17:18:35.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T17:18:35.754 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: mon.a calling monitor election 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: mon.c calling monitor election 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: fsmap 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: mgrmap e13: x(active, since 57s) 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: mon.b (rank 2) addr [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] is down (out of quorum) 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:18:39.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: mon.a calling monitor election 2024-09-13T17:18:39.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: mon.c calling monitor election 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: fsmap 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: mgrmap e13: x(active, since 57s) 2024-09-13T17:18:39.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: mon.b (rank 2) addr [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] is down (out of quorum) 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:39.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.178+0000 7f3eb37f9640 1 -- 172.21.15.100:0/2592460113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e98003530 msgr2=0x7f3e980059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.178+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/2592460113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e98003530 0x7f3e980059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3eac100d30 tx=0x7f3ea402bc80 comp rx=0 tx=0).stop 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 -- 172.21.15.100:0/2592460113 shutdown_connections 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/2592460113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e98003530 0x7f3e980059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 -- 172.21.15.100:0/2592460113 >> 172.21.15.100:0/2592460113 conn(0x7f3eac0fb5a0 msgr2=0x7f3eac0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 -- 172.21.15.100:0/2592460113 shutdown_connections 2024-09-13T17:18:40.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 -- 172.21.15.100:0/2592460113 wait complete. 2024-09-13T17:18:40.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 Processor -- start 2024-09-13T17:18:40.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.179+0000 7f3eb37f9640 1 -- start start 2024-09-13T17:18:40.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb37f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 0x7f3eac0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:40.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb37f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eac072cf0 0x7f3eac073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:40.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb37f9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eac1a13c0 0x7f3eac1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb37f9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a3cf0 con 0x7f3eac1a13c0 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb37f9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a3e60 con 0x7f3eac072cf0 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb37f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a4020 con 0x7f3e98003530 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 0x7f3eac0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 0x7f3eac0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60680/0 (socket says 172.21.15.100:60680) 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 -- 172.21.15.100:0/1394744902 learned_addr learned my addr 172.21.15.100:0/1394744902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1d96640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eac1a13c0 0x7f3eac1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb0d94640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eac072cf0 0x7f3eac073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 -- 172.21.15.100:0/1394744902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eac072cf0 msgr2=0x7f3eac073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eac072cf0 0x7f3eac073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 -- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eac1a13c0 msgr2=0x7f3eac1a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eac1a13c0 0x7f3eac1a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 -- 172.21.15.100:0/1394744902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009a20 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3eb1595640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 0x7f3eac0727b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40113f0 tx=0x7f3ea4004c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.180+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea402ecc0 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.181+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ea402ee20 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.181+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1a42a0 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.181+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4038c10 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.181+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1a47f0 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.181+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f3ea400a040 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.181+0000 7f3ea27fc640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e90038c10 0x7f3e9003b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.182+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ea406d860 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.182+0000 7f3eb0d94640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e90038c10 0x7f3e9003b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.182+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac100fe0 con 0x7f3e98003530 2024-09-13T17:18:40.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.182+0000 7f3eb0d94640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e90038c10 0x7f3e9003b0d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009950 tx=0x7f3e9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:40.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.187+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3ea4004eb0 con 0x7f3e98003530 2024-09-13T17:18:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.421+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3eac18eda0 con 0x7f3e98003530 2024-09-13T17:18:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.422+0000 7f3ea27fc640 1 -- 172.21.15.100:0/1394744902 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f3ea4033030 con 0x7f3e98003530 2024-09-13T17:18:40.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-13T17:18:40.424 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","modified":"2024-09-13T17:18:34.221226Z","created":"2024-09-13T17:16:51.854882Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T17:18:40.425 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 3 2024-09-13T17:18:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e90038c10 msgr2=0x7f3e9003b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e90038c10 0x7f3e9003b0d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009950 tx=0x7f3e9c008040 comp rx=0 tx=0).stop 2024-09-13T17:18:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 msgr2=0x7f3eac0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 0x7f3eac0727b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40113f0 tx=0x7f3ea4004c90 comp rx=0 tx=0).stop 2024-09-13T17:18:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 shutdown_connections 2024-09-13T17:18:40.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e90038c10 0x7f3e9003b0d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eac1a13c0 0x7f3eac1a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eac072cf0 0x7f3eac073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 --2- 172.21.15.100:0/1394744902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e98003530 0x7f3eac0727b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:40.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 >> 172.21.15.100:0/1394744902 conn(0x7f3eac0fb5a0 msgr2=0x7f3eac104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:40.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 shutdown_connections 2024-09-13T17:18:40.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:40.424+0000 7f3eb37f9640 1 -- 172.21.15.100:0/1394744902 wait complete. 2024-09-13T17:18:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:40 smithi100 ceph-mon[35055]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:40.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:40 smithi044 ceph-mon[31175]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:40.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:40.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:41.070 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-13T17:18:41.070 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-13T17:18:41.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:18:41.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.a calling monitor election 2024-09-13T17:18:41.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.c calling monitor election 2024-09-13T17:18:41.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: fsmap 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:41.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mgrmap e13: x(active, since 57s) 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.b (rank 2) addr [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] is down (out of quorum) 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:41.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.b calling monitor election 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Reconfiguring daemon mon.a on smithi044 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.b calling monitor election 2024-09-13T17:18:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.c calling monitor election 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.a calling monitor election 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: fsmap 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: mgrmap e13: x(active, since 59s) 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: overall HEALTH_OK 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:42.229 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: mon.b calling monitor election 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Reconfiguring daemon mon.a on smithi044 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: mon.b calling monitor election 2024-09-13T17:18:42.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: mon.c calling monitor election 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: mon.a calling monitor election 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: fsmap 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: mgrmap e13: x(active, since 59s) 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: overall HEALTH_OK 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:42.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: mon.b calling monitor election 2024-09-13T17:18:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:18:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T17:18:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Reconfiguring daemon mon.a on smithi044 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: mon.b calling monitor election 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: mon.c calling monitor election 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: mon.a calling monitor election 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: fsmap 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: mgrmap e13: x(active, since 59s) 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: overall HEALTH_OK 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.416 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.416 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.416 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: Reconfiguring mon.b (monmap changed)... 2024-09-13T17:18:43.416 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:43.417 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T17:18:43.417 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:43.417 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: Reconfiguring daemon mon.b on smithi070 2024-09-13T17:18:43.417 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: Reconfiguring mon.b (monmap changed)... 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: Reconfiguring daemon mon.b on smithi070 2024-09-13T17:18:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: Reconfiguring mon.b (monmap changed)... 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: Reconfiguring daemon mon.b on smithi070 2024-09-13T17:18:43.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 -- 172.21.15.44:0/3422517922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044101850 msgr2=0x7f6044109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 --2- 172.21.15.44:0/3422517922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044101850 0x7f6044109d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603802a480 comp rx=0 tx=0).stop 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 -- 172.21.15.44:0/3422517922 shutdown_connections 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 --2- 172.21.15.44:0/3422517922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604410a360 0x7f604410c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 --2- 172.21.15.44:0/3422517922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044101850 0x7f6044109d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 --2- 172.21.15.44:0/3422517922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6044100f30 0x7f6044101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 -- 172.21.15.44:0/3422517922 >> 172.21.15.44:0/3422517922 conn(0x7f60440fa9b0 msgr2=0x7f60440fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 -- 172.21.15.44:0/3422517922 shutdown_connections 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.629+0000 7f60498e7640 1 -- 172.21.15.44:0/3422517922 wait complete. 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 Processor -- start 2024-09-13T17:18:43.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 -- start start 2024-09-13T17:18:43.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 0x7f60441961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6044101850 0x7f60441966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604410a360 0x7f604419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604419b0e0 con 0x7f6044100f30 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604419b250 con 0x7f604410a360 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60498e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604419b3c0 con 0x7f6044101850 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f6042ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 0x7f60441961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f6042ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 0x7f60441961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39230/0 (socket says 172.21.15.44:39230) 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f6042ffd640 1 -- 172.21.15.44:0/1368921339 learned_addr learned my addr 172.21.15.44:0/1368921339 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60427fc640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6044101850 0x7f60441966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.630+0000 7f60437fe640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604410a360 0x7f604419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f6042ffd640 1 -- 172.21.15.44:0/1368921339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6044101850 msgr2=0x7f60441966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f6042ffd640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6044101850 0x7f60441966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f6042ffd640 1 -- 172.21.15.44:0/1368921339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604410a360 msgr2=0x7f604419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f6042ffd640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604410a360 0x7f604419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f6042ffd640 1 -- 172.21.15.44:0/1368921339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038009560 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f6042ffd640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 0x7f60441961a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f603400db80 tx=0x7f603400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034003b70 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f603400de80 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604419b6a0 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340106c0 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.631+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60441a2f60 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.632+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f6034010820 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.632+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60440687e0 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.632+0000 7f60488e5640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f602c038930 0x7f602c03adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.632+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f603404b010 con 0x7f6044100f30 2024-09-13T17:18:43.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.632+0000 7f60427fc640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f602c038930 0x7f602c03adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:43.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.633+0000 7f60427fc640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f602c038930 0x7f602c03adf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f603802aa70 tx=0x7f6038005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.637+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6034021c70 con 0x7f6044100f30 2024-09-13T17:18:43.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.812+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6044101310 con 0x7f6044100f30 2024-09-13T17:18:43.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.812+0000 7f60488e5640 1 -- 172.21.15.44:0/1368921339 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f6034015110 con 0x7f6044100f30 2024-09-13T17:18:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:# minimal ceph.conf for ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:18:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-09-13T17:18:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:18:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: mon_host = [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.814+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f602c038930 msgr2=0x7f602c03adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.814+0000 7f60498e7640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f602c038930 0x7f602c03adf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f603802aa70 tx=0x7f6038005a20 comp rx=0 tx=0).stop 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.814+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 msgr2=0x7f60441961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.814+0000 7f60498e7640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 0x7f60441961a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f603400db80 tx=0x7f603400b630 comp rx=0 tx=0).stop 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 shutdown_connections 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f602c038930 0x7f602c03adf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6044101850 0x7f60441966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604410a360 0x7f604419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 --2- 172.21.15.44:0/1368921339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6044100f30 0x7f60441961a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:43.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 >> 172.21.15.44:0/1368921339 conn(0x7f60440fa9b0 msgr2=0x7f60440fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:43.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 shutdown_connections 2024-09-13T17:18:43.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:43.815+0000 7f60498e7640 1 -- 172.21.15.44:0/1368921339 wait complete. 2024-09-13T17:18:44.417 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-13T17:18:44.417 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:18:44.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T17:18:44.450 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:18:44.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:18:44.513 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:44.513 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:44.513 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: Reconfiguring mon.c (monmap changed)... 2024-09-13T17:18:44.513 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:44.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T17:18:44.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:44.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: Reconfiguring daemon mon.c on smithi100 2024-09-13T17:18:44.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:44.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1368921339' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:44.517 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:18:44.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T17:18:44.551 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:18:44.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:18:44.620 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:18:44.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T17:18:44.649 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:18:44.649 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:18:44.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:44.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:44.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: Reconfiguring mon.c (monmap changed)... 2024-09-13T17:18:44.711 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:44.711 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T17:18:44.711 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:44.711 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: Reconfiguring daemon mon.c on smithi100 2024-09-13T17:18:44.711 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:44.711 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1368921339' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:44.714 INFO:tasks.cephadm:Adding mgr.x on smithi044 2024-09-13T17:18:44.714 INFO:tasks.cephadm:Adding mgr.y on smithi070 2024-09-13T17:18:44.714 INFO:tasks.cephadm:Adding mgr.z on smithi100 2024-09-13T17:18:44.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi044=x;smithi070=y;smithi100=z' 2024-09-13T17:18:44.939 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:18:44.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: Reconfiguring mon.c (monmap changed)... 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: Reconfiguring daemon mon.c on smithi100 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1368921339' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:45 smithi070 ceph-mon[34502]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:45.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:45.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:45.963 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:45.963 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:45.963 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:45.963 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:45.963 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:45 smithi100 ceph-mon[35055]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:45 smithi044 ceph-mon[31175]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 -- 172.21.15.100:0/436217622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d8101dd0 msgr2=0x7f95d80fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 --2- 172.21.15.100:0/436217622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d8101dd0 0x7f95d80fde80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009900 tx=0x7f95cc02a400 comp rx=0 tx=0).stop 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 -- 172.21.15.100:0/436217622 shutdown_connections 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 --2- 172.21.15.100:0/436217622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d8101dd0 0x7f95d80fde80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 --2- 172.21.15.100:0/436217622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d80fe520 0x7f95d81083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 --2- 172.21.15.100:0/436217622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95d8101400 0x7f95d8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 -- 172.21.15.100:0/436217622 >> 172.21.15.100:0/436217622 conn(0x7f95d80f9c10 msgr2=0x7f95d80fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 -- 172.21.15.100:0/436217622 shutdown_connections 2024-09-13T17:18:46.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.395+0000 7f95dca23640 1 -- 172.21.15.100:0/436217622 wait complete. 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 Processor -- start 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 -- start start 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 0x7f95d8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95d8101400 0x7f95d8196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d8101dd0 0x7f95d819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819d830 con 0x7f95d8101400 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819d9a0 con 0x7f95d8101dd0 2024-09-13T17:18:46.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95dca23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819db60 con 0x7f95d80fe520 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95d659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 0x7f95d8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95d659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 0x7f95d8195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52586/0 (socket says 172.21.15.100:52586) 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.396+0000 7f95d659c640 1 -- 172.21.15.100:0/2807299229 learned_addr learned my addr 172.21.15.100:0/2807299229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d5d9b640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95d8101400 0x7f95d8196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d6d9d640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d8101dd0 0x7f95d819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d659c640 1 -- 172.21.15.100:0/2807299229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d8101dd0 msgr2=0x7f95d819d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:46.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d659c640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d8101dd0 0x7f95d819d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d659c640 1 -- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95d8101400 msgr2=0x7f95d8196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d659c640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95d8101400 0x7f95d8196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d659c640 1 -- 172.21.15.100:0/2807299229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc009560 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d6d9d640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d8101dd0 0x7f95d819d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95d659c640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 0x7f95d8195d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f95c800e9c0 tx=0x7f95c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8009700 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d80ff0b0 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95c80044b0 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8003e40 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.397+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d80ff580 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.398+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f95c8018430 con 0x7f95d80fe520 2024-09-13T17:18:46.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.398+0000 7f95c77fe640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95c0038960 0x7f95c003ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:46.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.398+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f95c8014070 con 0x7f95d80fe520 2024-09-13T17:18:46.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.398+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d8101800 con 0x7f95d80fe520 2024-09-13T17:18:46.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.398+0000 7f95d5d9b640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95c0038960 0x7f95c003ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:46.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.399+0000 7f95d5d9b640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95c0038960 0x7f95c003ae20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006010 tx=0x7f95cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:46.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.403+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f95c8021430 con 0x7f95d80fe520 2024-09-13T17:18:46.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.578+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi070=y;smithi100=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f95d80608f0 con 0x7f95c0038960 2024-09-13T17:18:46.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.585+0000 7f95c77fe640 1 -- 172.21.15.100:0/2807299229 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f95d80608f0 con 0x7f95c0038960 2024-09-13T17:18:46.586 INFO:teuthology.orchestra.run.smithi100.stdout:Scheduled mgr update... 2024-09-13T17:18:46.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95c0038960 msgr2=0x7f95c003ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95c0038960 0x7f95c003ae20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006010 tx=0x7f95cc000bc0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 msgr2=0x7f95d8195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 0x7f95d8195d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f95c800e9c0 tx=0x7f95c800ee80 comp rx=0 tx=0).stop 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 shutdown_connections 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95c0038960 0x7f95c003ae20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d8101dd0 0x7f95d819d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95d8101400 0x7f95d8196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 --2- 172.21.15.100:0/2807299229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d80fe520 0x7f95d8195d30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 >> 172.21.15.100:0/2807299229 conn(0x7f95d80f9c10 msgr2=0x7f95d8104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 shutdown_connections 2024-09-13T17:18:46.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:18:46.587+0000 7f95dca23640 1 -- 172.21.15.100:0/2807299229 wait complete. 2024-09-13T17:18:47.199 DEBUG:teuthology.orchestra.run.smithi070:mgr.y> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.y.service 2024-09-13T17:18:47.202 DEBUG:teuthology.orchestra.run.smithi100:mgr.z> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.z.service 2024-09-13T17:18:47.204 INFO:tasks.cephadm:Deploying OSDs... 2024-09-13T17:18:47.205 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:18:47.205 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:18:47.225 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:18:47.225 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 17:17:53.646561690 +0000 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 17:13:29.010355365 +0000 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 17:13:29.010355365 +0000 2024-09-13T17:18:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 17:13:29.010355365 +0000 2024-09-13T17:18:47.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:18:47.355 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T17:18:47.355 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T17:18:47.355 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00028829 s, 1.8 MB/s 2024-09-13T17:18:47.357 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:18:47.416 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 17:17:53.682561133 +0000 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 17:13:29.352350220 +0000 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 17:13:29.352350220 +0000 2024-09-13T17:18:47.475 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 17:13:29.352350220 +0000 2024-09-13T17:18:47.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:18:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T17:18:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T17:18:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00019814 s, 2.6 MB/s 2024-09-13T17:18:47.544 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:18:47.603 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-09-13T17:18:47.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi070=y;smithi100=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:47.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: Saving service mgr spec with placement smithi044=x;smithi070=y;smithi100=z;count:3 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: Deploying daemon mgr.z on smithi100 2024-09-13T17:18:47.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:47 smithi044 ceph-mon[31175]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:18:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:47.623 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-13T17:18:47.624 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:47.624 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:47.624 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 17:17:53.719560561 +0000 2024-09-13T17:18:47.624 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 17:13:29.675345362 +0000 2024-09-13T17:18:47.624 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 17:13:29.675345362 +0000 2024-09-13T17:18:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 17:13:29.675345362 +0000 2024-09-13T17:18:47.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:18:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T17:18:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T17:18:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000229676 s, 2.2 MB/s 2024-09-13T17:18:47.691 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:18:47.748 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi070=y;smithi100=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: Saving service mgr spec with placement smithi044=x;smithi070=y;smithi100=z;count:3 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T17:18:47.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:47.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:47.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: Deploying daemon mgr.z on smithi100 2024-09-13T17:18:47.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:47 smithi100 ceph-mon[35055]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 17:17:53.755560005 +0000 2024-09-13T17:18:47.805 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 17:13:29.997340519 +0000 2024-09-13T17:18:47.806 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 17:13:29.997340519 +0000 2024-09-13T17:18:47.806 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 17:13:29.997340519 +0000 2024-09-13T17:18:47.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:18:47.869 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T17:18:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T17:18:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000212675 s, 2.4 MB/s 2024-09-13T17:18:47.872 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:18:47.929 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T17:18:47.929 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi070=y;smithi100=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: Saving service mgr spec with placement smithi044=x;smithi070=y;smithi100=z;count:3 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: Deploying daemon mgr.z on smithi100 2024-09-13T17:18:47.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:47 smithi070 ceph-mon[34502]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:47.949 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:18:47.949 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 17:18:33.702416785 +0000 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 17:13:29.108960276 +0000 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 17:13:29.108960276 +0000 2024-09-13T17:18:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 17:13:29.108960276 +0000 2024-09-13T17:18:48.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:18:48.075 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T17:18:48.075 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T17:18:48.075 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000126005 s, 4.1 MB/s 2024-09-13T17:18:48.076 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:18:48.135 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 17:18:33.703416756 +0000 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 17:13:29.420951025 +0000 2024-09-13T17:18:48.192 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 17:13:29.420951025 +0000 2024-09-13T17:18:48.193 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 17:13:29.420951025 +0000 2024-09-13T17:18:48.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:18:48.257 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T17:18:48.257 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T17:18:48.257 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000120982 s, 4.2 MB/s 2024-09-13T17:18:48.259 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:18:48.316 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-13T17:18:48.372 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:18:48.372 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:48.372 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-13T17:18:48.372 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:48.372 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:48.372 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 17:18:33.703416756 +0000 2024-09-13T17:18:48.373 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 17:13:29.737941625 +0000 2024-09-13T17:18:48.373 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 17:13:29.737941625 +0000 2024-09-13T17:18:48.373 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 17:13:29.737941625 +0000 2024-09-13T17:18:48.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:18:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T17:18:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T17:18:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.7873e-05 s, 5.8 MB/s 2024-09-13T17:18:48.437 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:18:48.494 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-13T17:18:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:18:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:48.549 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T17:18:48.550 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:48.550 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:48.550 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 17:18:33.704416726 +0000 2024-09-13T17:18:48.550 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 17:13:30.063931959 +0000 2024-09-13T17:18:48.550 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 17:13:30.063931959 +0000 2024-09-13T17:18:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 17:13:30.063931959 +0000 2024-09-13T17:18:48.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:18:48.613 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T17:18:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T17:18:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.00011515 s, 4.4 MB/s 2024-09-13T17:18:48.615 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:18:48.671 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-13T17:18:48.671 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:18:48.687 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:18:48.687 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-13T17:18:48.744 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:18:48.744 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:48.744 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-13T17:18:48.744 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:48.745 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:48.745 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-13 17:18:48.651603642 +0000 2024-09-13T17:18:48.745 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-13 17:13:24.849249713 +0000 2024-09-13T17:18:48.745 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-13 17:13:24.849249713 +0000 2024-09-13T17:18:48.745 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-13 17:13:24.849249713 +0000 2024-09-13T17:18:48.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:18:48.808 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:48 smithi100 systemd[1]: Starting Ceph mgr.z for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:18:48.812 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-13T17:18:48.812 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-13T17:18:48.812 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000196304 s, 2.6 MB/s 2024-09-13T17:18:48.813 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:18:48.870 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-13T17:18:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:18:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:48.925 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-13T17:18:48.925 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:48.925 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:48.926 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-13 17:18:48.651603642 +0000 2024-09-13T17:18:48.926 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-13 17:13:25.182240895 +0000 2024-09-13T17:18:48.926 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-13 17:13:25.182240895 +0000 2024-09-13T17:18:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-13 17:13:25.182240895 +0000 2024-09-13T17:18:48.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:18:48.989 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-13T17:18:48.989 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-13T17:18:48.989 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000204475 s, 2.5 MB/s 2024-09-13T17:18:48.990 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:18:49.053 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-13T17:18:49.108 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:18:49.108 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-13 17:18:48.652603615 +0000 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-13 17:13:25.508232263 +0000 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-13 17:13:25.508232263 +0000 2024-09-13T17:18:49.109 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-13 17:13:25.508232263 +0000 2024-09-13T17:18:49.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:18:49.139 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:48 smithi100 podman[36872]: 2024-09-13 17:18:48.828354003 +0000 UTC m=+0.015903047 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:18:49.139 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:48 smithi100 podman[36872]: 2024-09-13 17:18:48.92740431 +0000 UTC m=+0.114953343 container create f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731) 2024-09-13T17:18:49.139 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 podman[36872]: 2024-09-13 17:18:49.138243807 +0000 UTC m=+0.325792846 container init f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-13T17:18:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-13T17:18:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-13T17:18:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000216828 s, 2.4 MB/s 2024-09-13T17:18:49.165 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:18:49.221 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-13 17:18:48.652603615 +0000 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-13 17:13:25.838223525 +0000 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-13 17:13:25.838223525 +0000 2024-09-13T17:18:49.278 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-13 17:13:25.838223525 +0000 2024-09-13T17:18:49.279 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:18:49.343 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-13T17:18:49.343 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-13T17:18:49.343 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000193352 s, 2.6 MB/s 2024-09-13T17:18:49.345 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:18:49.402 INFO:tasks.cephadm:Deploying osd.0 on smithi044 with /dev/vg_nvme/lv_4... 2024-09-13T17:18:49.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 podman[36872]: 2024-09-13 17:18:49.141609705 +0000 UTC m=+0.329158745 container start f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-mgr[36981]: -- 172.21.15.100:0/240026712 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56436c5fcd00 con 0x56436c5d1400 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:49.262+0000 7f5cea0c5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 bash[36872]: f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 systemd[1]: Started Ceph mgr.z for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:49.325+0000 7f5cea0c5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:18:49.477 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:49.475+0000 7f5cea0c5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:18:49.607 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:49.774 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:49.540+0000 7f5cea0c5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:18:49.774 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:49.681+0000 7f5cea0c5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:18:50.122 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:49.818+0000 7f5cea0c5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:18:50.122 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:50.121+0000 7f5cea0c5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:18:50.418 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:50.180+0000 7f5cea0c5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:18:50.418 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:50.361+0000 7f5cea0c5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: Deploying daemon mgr.y on smithi070 2024-09-13T17:18:50.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:50 smithi100 ceph-mon[35055]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: Deploying daemon mgr.y on smithi070 2024-09-13T17:18:50.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:50 smithi044 ceph-mon[31175]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: Deploying daemon mgr.y on smithi070 2024-09-13T17:18:50.628 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:50 smithi070 ceph-mon[34502]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:51.399 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T17:18:51.498 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T17:18:51.512 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-09-13T17:18:51.512 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-09-13T17:18:51.512 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113546 s, 923 MB/s 2024-09-13T17:18:51.512 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-09-13T17:18:51.556 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:51.276+0000 7f5cea0c5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:18:51.556 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:51.343+0000 7f5cea0c5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:18:51.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-mon[34502]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:51.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.697 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:51.581+0000 7fbbb7754040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:18:51.829 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:51 smithi044 ceph-mon[31175]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:51.829 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:51 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.829 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:51 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.829 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:51 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:51.829 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:51 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-mon[35055]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:51 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:52.112 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:51.737+0000 7fbbb7754040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:18:52.113 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:51.805+0000 7fbbb7754040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:18:52.113 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:51.967+0000 7fbbb7754040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:18:52.113 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:52.111+0000 7fbbb7754040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:18:52.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-09-13T17:18:52.339 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:18:52.381 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.113+0000 7f5cea0c5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:18:52.381 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.235+0000 7f5cea0c5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:18:52.381 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.379+0000 7f5cea0c5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:18:52.433 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:52.431+0000 7fbbb7754040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:18:52.669 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:52.669 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.441+0000 7f5cea0c5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:18:52.669 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.596+0000 7f5cea0c5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:18:52.670 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.668+0000 7f5cea0c5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:18:52.684 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:52.684 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:52.491+0000 7fbbb7754040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:18:52.756 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:52.947 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:52.682+0000 7fbbb7754040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:18:52.953 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.728+0000 7f5cea0c5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:18:52.953 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.887+0000 7f5cea0c5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:18:52.953 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:52.951+0000 7f5cea0c5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:18:53.219 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:53.018+0000 7f5cea0c5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:18:53.219 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:53.101+0000 7f5cea0c5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:18:53.219 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:53.159+0000 7f5cea0c5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:18:53.219 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:53.217+0000 7f5cea0c5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:18:53.737 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.737 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.737 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.737 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.737 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:53.737 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: Reconfiguring daemon mgr.x on smithi044 2024-09-13T17:18:53.738 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:53 smithi044 ceph-mon[31175]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:53.544+0000 7fbbb7754040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:53.610+0000 7fbbb7754040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: Reconfiguring daemon mgr.x on smithi044 2024-09-13T17:18:53.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:53 smithi070 ceph-mon[34502]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: Reconfiguring daemon mgr.x on smithi044 2024-09-13T17:18:53.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-mon[35055]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:54.274 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:18:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:18:53.995+0000 7f5cea0c5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:18:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 -- 172.21.15.44:0/4208763700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0701008a0 msgr2=0x7fc070107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 --2- 172.21.15.44:0/4208763700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0701008a0 0x7fc070107130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802a410 comp rx=0 tx=0).stop 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 -- 172.21.15.44:0/4208763700 shutdown_connections 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 --2- 172.21.15.44:0/4208763700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0701008a0 0x7fc070107130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 --2- 172.21.15.44:0/4208763700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0700ffee0 0x7fc070100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 --2- 172.21.15.44:0/4208763700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 0x7fc0700ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 -- 172.21.15.44:0/4208763700 >> 172.21.15.44:0/4208763700 conn(0x7fc0700fa490 msgr2=0x7fc0700fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 -- 172.21.15.44:0/4208763700 shutdown_connections 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.433+0000 7fc076e85640 1 -- 172.21.15.44:0/4208763700 wait complete. 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 Processor -- start 2024-09-13T17:18:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 -- start start 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 0x7fc070197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 0x7fc0701983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0701008a0 0x7fc07019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07019f9a0 con 0x7fc0700ffee0 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07019fb10 con 0x7fc0701008a0 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc076e85640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07019fcd0 con 0x7fc0700fece0 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 0x7fc0701983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 0x7fc0701983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45928/0 (socket says 172.21.15.44:45928) 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 -- 172.21.15.44:0/675263783 learned_addr learned my addr 172.21.15.44:0/675263783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:18:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc075422640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0701008a0 0x7fc07019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc074c21640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 0x7fc070197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 -- 172.21.15.44:0/675263783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 msgr2=0x7fc070197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 0x7fc070197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 -- 172.21.15.44:0/675263783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0701008a0 msgr2=0x7fc07019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0701008a0 0x7fc07019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc06ffff640 1 -- 172.21.15.44:0/675263783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009840 con 0x7fc0700ffee0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.434+0000 7fc074c21640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 0x7fc070197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.435+0000 7fc06ffff640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 0x7fc0701983e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc060009d50 tx=0x7fc06000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.435+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060003b70 con 0x7fc0700ffee0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.435+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc060003cd0 con 0x7fc0700ffee0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.435+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068009560 con 0x7fc0700ffee0 2024-09-13T17:18:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.435+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600107b0 con 0x7fc0700ffee0 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.435+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0701a0290 con 0x7fc0700ffee0 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.436+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fc060010910 con 0x7fc0700ffee0 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.436+0000 7fc06dffb640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc05c038960 0x7fc05c03ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.436+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc060014070 con 0x7fc0700ffee0 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.436+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0700ff0e0 con 0x7fc0700ffee0 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.436+0000 7fc074c21640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc05c038960 0x7fc05c03ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.436+0000 7fc074c21640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc05c038960 0x7fc05c03ae20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0700ffd40 tx=0x7fc058005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.441+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc06001f330 con 0x7fc0700ffee0 2024-09-13T17:18:54.624 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.313+0000 7fbbb7754040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:18:54.624 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.425+0000 7fbbb7754040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:18:54.624 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.562+0000 7fbbb7754040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:18:54.624 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.623+0000 7fbbb7754040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:18:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.624+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc070106e20 con 0x7fc05c038960 2024-09-13T17:18:54.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:54.660+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7fc060021a90 con 0x7fc0700ffee0 2024-09-13T17:18:54.897 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: Standby manager daemon z started 2024-09-13T17:18:54.897 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.770+0000 7fbbb7754040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.836+0000 7fbbb7754040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:18:54.898 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:54.896+0000 7fbbb7754040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:18:54.986 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: Standby manager daemon z started 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:18:54.987 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: Standby manager daemon z started 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/549168586' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:18:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:55.185 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:55.054+0000 7fbbb7754040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:18:55.185 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:55.118+0000 7fbbb7754040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:18:55.185 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:55.183+0000 7fbbb7754040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:18:55.447 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:55.265+0000 7fbbb7754040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:18:55.447 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:55.323+0000 7fbbb7754040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:18:55.447 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:55.381+0000 7fbbb7754040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: mgrmap e14: x(active, since 73s), standbys: z 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:55 smithi070 ceph-mon[34502]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: mgrmap e14: x(active, since 73s), standbys: z 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:55.985 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:55 smithi044 ceph-mon[31175]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:18:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: mgrmap e14: x(active, since 73s), standbys: z 2024-09-13T17:18:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T17:18:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:18:56.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:18:56.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:18:56.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:56.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:55 smithi100 ceph-mon[35055]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:56.447 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:18:56.143+0000 7fbbb7754040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:18:56.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:18:56.864+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fc060020390 con 0x7fc0700ffee0 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: Standby manager daemon y started 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2563047954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d9259-48ec-4ece-86c1-326e4bc5ab62"}]: dispatch 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2563047954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8d9259-48ec-4ece-86c1-326e4bc5ab62"}]': finished 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T17:18:57.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:18:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: Standby manager daemon y started 2024-09-13T17:18:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T17:18:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:18:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T17:18:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:18:57.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:57.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2563047954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d9259-48ec-4ece-86c1-326e4bc5ab62"}]: dispatch 2024-09-13T17:18:57.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2563047954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8d9259-48ec-4ece-86c1-326e4bc5ab62"}]': finished 2024-09-13T17:18:57.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T17:18:57.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: Standby manager daemon y started 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/605607163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:18:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2563047954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d9259-48ec-4ece-86c1-326e4bc5ab62"}]: dispatch 2024-09-13T17:18:57.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2563047954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8d9259-48ec-4ece-86c1-326e4bc5ab62"}]': finished 2024-09-13T17:18:57.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T17:18:57.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:18:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:57 smithi070 ceph-mon[34502]: mgrmap e15: x(active, since 75s), standbys: y, z 2024-09-13T17:18:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T17:18:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:57 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3862830079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:18:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:57 smithi070 ceph-mon[34502]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:57 smithi100 ceph-mon[35055]: mgrmap e15: x(active, since 75s), standbys: y, z 2024-09-13T17:18:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T17:18:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:57 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3862830079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:18:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:57 smithi100 ceph-mon[35055]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:58.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:57 smithi044 ceph-mon[31175]: mgrmap e15: x(active, since 75s), standbys: y, z 2024-09-13T17:18:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T17:18:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:57 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3862830079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:18:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:57 smithi044 ceph-mon[31175]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:59.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:18:59 smithi044 ceph-mon[31175]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:18:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:18:59 smithi070 ceph-mon[34502]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:18:59 smithi100 ceph-mon[35055]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:01.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:01 smithi044 ceph-mon[31175]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:01 smithi070 ceph-mon[34502]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:02.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:01 smithi100 ceph-mon[35055]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:03 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T17:19:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:03 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:03 smithi100 ceph-mon[35055]: Deploying daemon osd.0 on smithi044 2024-09-13T17:19:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:03 smithi100 ceph-mon[35055]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:03.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T17:19:03.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:03.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:03 smithi044 ceph-mon[31175]: Deploying daemon osd.0 on smithi044 2024-09-13T17:19:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:03 smithi044 ceph-mon[31175]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:03 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T17:19:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:03 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:03 smithi070 ceph-mon[34502]: Deploying daemon osd.0 on smithi044 2024-09-13T17:19:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:03 smithi070 ceph-mon[34502]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:05.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:05 smithi044 ceph-mon[31175]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:05.949 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:05 smithi070 ceph-mon[34502]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:05 smithi100 ceph-mon[35055]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:07 smithi044 ceph-mon[31175]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:07.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:07 smithi070 ceph-mon[34502]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:07 smithi100 ceph-mon[35055]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:08.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:09.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:09 smithi044 ceph-mon[31175]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:09 smithi070 ceph-mon[34502]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:09 smithi100 ceph-mon[35055]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.709+0000 7fc06dffb640 1 -- 172.21.15.44:0/675263783 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc070106e20 con 0x7fc05c038960 2024-09-13T17:19:11.711 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 0 on host 'smithi044' 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc05c038960 msgr2=0x7fc05c03ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc05c038960 0x7fc05c03ae20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0700ffd40 tx=0x7fc058005f90 comp rx=0 tx=0).stop 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 msgr2=0x7fc0701983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 0x7fc0701983e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc060009d50 tx=0x7fc06000c6a0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 shutdown_connections 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc05c038960 0x7fc05c03ae20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0701008a0 0x7fc07019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0700ffee0 0x7fc0701983e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 --2- 172.21.15.44:0/675263783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0700fece0 0x7fc070197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 >> 172.21.15.44:0/675263783 conn(0x7fc0700fa490 msgr2=0x7fc070105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 shutdown_connections 2024-09-13T17:19:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:19:11.713+0000 7fc076e85640 1 -- 172.21.15.44:0/675263783 wait complete. 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:12.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:13.072 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:12 smithi044 ceph-mon[31175]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T17:19:13.100 DEBUG:teuthology.orchestra.run.smithi044:osd.0> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.0.service 2024-09-13T17:19:13.103 INFO:tasks.cephadm:Deploying osd.1 on smithi070 with /dev/vg_nvme/lv_4... 2024-09-13T17:19:13.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T17:19:13.128 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:12 smithi070 ceph-mon[34502]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T17:19:13.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:12 smithi100 ceph-mon[35055]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T17:19:13.296 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:19:14.023 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:13 smithi070 ceph-mon[34502]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T17:19:14.023 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:13 smithi070 ceph-mon[34502]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T17:19:14.023 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:13 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:14.023 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:13 smithi070 ceph-mon[34502]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:19:14.024 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:13 smithi070 ceph-mon[34502]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-mon[31175]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T17:19:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-mon[31175]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T17:19:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-mon[31175]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:19:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-mon[31175]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:14.107 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0[45525]: 2024-09-13T17:19:13.879+0000 7f0783ae2640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T17:19:14.108 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:19:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0[45525]: 2024-09-13T17:19:13.885+0000 7f077f90b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:19:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:13 smithi100 ceph-mon[35055]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T17:19:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:13 smithi100 ceph-mon[35055]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T17:19:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:13 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:13 smithi100 ceph-mon[35055]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:19:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:13 smithi100 ceph-mon[35055]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T17:19:15.002 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T17:19:15.012 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-09-13T17:19:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-09-13T17:19:15.013 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834191 s, 1.3 GB/s 2024-09-13T17:19:15.013 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-09-13T17:19:15.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:14 smithi044 ceph-mon[31175]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:19:15.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:14 smithi044 ceph-mon[31175]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T17:19:15.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:14 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:15.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:14 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:15.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:14 smithi070 ceph-mon[34502]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:19:15.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:14 smithi070 ceph-mon[34502]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T17:19:15.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:14 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:15.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:14 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:15.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:14 smithi100 ceph-mon[35055]: from='osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:19:15.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:14 smithi100 ceph-mon[35055]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T17:19:15.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:14 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:15.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:14 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:15.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2024-09-13T17:19:15.897 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] boot 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi044 2024-09-13T17:19:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi044 to 17092M 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:15 smithi070 ceph-mon[34502]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:19:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:19:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] boot 2024-09-13T17:19:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi044 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi044 to 17092M 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:15 smithi100 ceph-mon[35055]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:19:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:19:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: osd.0 [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] boot 2024-09-13T17:19:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi044 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi044 to 17092M 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:16.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:15 smithi044 ceph-mon[31175]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T17:19:17.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.436+0000 7f00b4df2640 1 -- 172.21.15.70:0/3639978433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 msgr2=0x7f00b0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:17.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.436+0000 7f00b4df2640 1 --2- 172.21.15.70:0/3639978433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0102c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009900 tx=0x7f00a002a410 comp rx=0 tx=0).stop 2024-09-13T17:19:17.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 -- 172.21.15.70:0/3639978433 shutdown_connections 2024-09-13T17:19:17.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 --2- 172.21.15.70:0/3639978433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0105af0 0x7f00b0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:17.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 --2- 172.21.15.70:0/3639978433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 0x7f00b01055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:17.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 --2- 172.21.15.70:0/3639978433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0102c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:17.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 -- 172.21.15.70:0/3639978433 >> 172.21.15.70:0/3639978433 conn(0x7f00b00fa490 msgr2=0x7f00b00fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:17.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 -- 172.21.15.70:0/3639978433 shutdown_connections 2024-09-13T17:19:17.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 -- 172.21.15.70:0/3639978433 wait complete. 2024-09-13T17:19:17.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.437+0000 7f00b4df2640 1 Processor -- start 2024-09-13T17:19:17.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 -- start start 2024-09-13T17:19:17.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:17.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 0x7f00b01961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0105af0 0x7f00b019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b019d760 con 0x7f00b01031c0 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b019d8d0 con 0x7f00b0100890 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00b4df2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b019da90 con 0x7f00b0105af0 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00ae59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00ae59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41492/0 (socket says 172.21.15.70:41492) 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00ae59c640 1 -- 172.21.15.70:0/2705649080 learned_addr learned my addr 172.21.15.70:0/2705649080 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00add9b640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 0x7f00b01961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.438+0000 7f00aed9d640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0105af0 0x7f00b019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00ae59c640 1 -- 172.21.15.70:0/2705649080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0105af0 msgr2=0x7f00b019d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00ae59c640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0105af0 0x7f00b019d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00ae59c640 1 -- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 msgr2=0x7f00b01961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:17.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00ae59c640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 0x7f00b01961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00ae59c640 1 -- 172.21.15.70:0/2705649080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0009560 con 0x7f00b0100890 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00add9b640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 0x7f00b01961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00ae59c640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0195c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009230 tx=0x7f00a0004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0031d70 con 0x7f00b0100890 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00a0004b50 con 0x7f00b0100890 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0003bf0 con 0x7f00b0100890 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b019dd10 con 0x7f00b0100890 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.439+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b019e1b0 con 0x7f00b0100890 2024-09-13T17:19:17.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.440+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b00fbf80 con 0x7f00b0100890 2024-09-13T17:19:17.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.441+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f00a00046c0 con 0x7f00b0100890 2024-09-13T17:19:17.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.442+0000 7f009f7fe640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00980a1070 0x7f00980a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:17.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.442+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f00a00ec320 con 0x7f00b0100890 2024-09-13T17:19:17.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.442+0000 7f00add9b640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00980a1070 0x7f00980a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:17.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.442+0000 7f00add9b640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00980a1070 0x7f00980a3530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00b0197180 tx=0x7f00a400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:17.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.446+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f00a00b7520 con 0x7f00b0100890 2024-09-13T17:19:17.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:17 smithi100 ceph-mon[35055]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T17:19:17.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:17 smithi044 ceph-mon[31175]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T17:19:17.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:17.616+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f00b002c4f0 con 0x7f00980a1070 2024-09-13T17:19:17.618 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:17 smithi070 ceph-mon[34502]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T17:19:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:18 smithi100 ceph-mon[35055]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:19:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:19:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:18.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:18 smithi044 ceph-mon[31175]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:19:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:19:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:18.637 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:18 smithi070 ceph-mon[34502]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:18.637 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:19:18.637 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:19:18.637 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:19.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:19 smithi100 ceph-mon[35055]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:19:19.579 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:19 smithi070 ceph-mon[34502]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:19:19.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:19 smithi044 ceph-mon[31175]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:19:20.438 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:20 smithi070 ceph-mon[34502]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:20.439 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:20 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/301628682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]: dispatch 2024-09-13T17:19:20.439 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:20 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]: dispatch 2024-09-13T17:19:20.439 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:20 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]': finished 2024-09-13T17:19:20.439 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:20 smithi070 ceph-mon[34502]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T17:19:20.439 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:20 smithi100 ceph-mon[35055]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:20 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/301628682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]: dispatch 2024-09-13T17:19:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:20 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]: dispatch 2024-09-13T17:19:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:20 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]': finished 2024-09-13T17:19:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:20 smithi100 ceph-mon[35055]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T17:19:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:20 smithi044 ceph-mon[31175]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:20 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/301628682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]: dispatch 2024-09-13T17:19:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:20 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]: dispatch 2024-09-13T17:19:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:20 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8eb850b3-92e7-40d5-a891-9c21f9f591dd"}]': finished 2024-09-13T17:19:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:20 smithi044 ceph-mon[31175]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T17:19:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:21.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:21 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/2706655481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:19:21.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:21 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/2706655481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:19:21.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:21 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/2706655481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:19:22.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:22 smithi100 ceph-mon[35055]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:22 smithi044 ceph-mon[31175]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:22 smithi070 ceph-mon[34502]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:23 smithi070 ceph-mon[34502]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:23.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:23 smithi044 ceph-mon[31175]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:24.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:23 smithi100 ceph-mon[35055]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:25.772 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:25 smithi070 ceph-mon[34502]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:25.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:25 smithi044 ceph-mon[31175]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:26.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:25 smithi100 ceph-mon[35055]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:26.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:26 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T17:19:26.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:26 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:27.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:26 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T17:19:27.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:26 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:27.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:26 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T17:19:27.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:26 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:27 smithi100 ceph-mon[35055]: Deploying daemon osd.1 on smithi070 2024-09-13T17:19:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:27 smithi100 ceph-mon[35055]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:28.027 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:27 smithi070 ceph-mon[34502]: Deploying daemon osd.1 on smithi070 2024-09-13T17:19:28.027 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:27 smithi070 ceph-mon[34502]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:28.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:27 smithi044 ceph-mon[31175]: Deploying daemon osd.1 on smithi070 2024-09-13T17:19:28.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:27 smithi044 ceph-mon[31175]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:29 smithi070 ceph-mon[34502]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:29.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:29 smithi044 ceph-mon[31175]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:30.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:29 smithi100 ceph-mon[35055]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:31.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:31 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:31.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:31 smithi044 ceph-mon[31175]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:31.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:31 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:31.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:31 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:31.878 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:31 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:31.878 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:31 smithi070 ceph-mon[34502]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:31.878 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:31 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:31.878 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:31 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:31 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:31 smithi100 ceph-mon[35055]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:31 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:31 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:33.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:33 smithi044 ceph-mon[31175]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:33.932 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:33 smithi070 ceph-mon[34502]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:33 smithi100 ceph-mon[35055]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:34.895 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 1 on host 'smithi070' 2024-09-13T17:19:34.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.893+0000 7f009f7fe640 1 -- 172.21.15.70:0/2705649080 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f00b002c4f0 con 0x7f00980a1070 2024-09-13T17:19:34.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00980a1070 msgr2=0x7f00980a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00980a1070 0x7f00980a3530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00b0197180 tx=0x7f00a400a2d0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 msgr2=0x7f00b0195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0195c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009230 tx=0x7f00a0004380 comp rx=0 tx=0).stop 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 shutdown_connections 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00980a1070 0x7f00980a3530 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0105af0 0x7f00b019d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00b01031c0 0x7f00b01961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 --2- 172.21.15.70:0/2705649080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b0100890 0x7f00b0195c70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 >> 172.21.15.70:0/2705649080 conn(0x7f00b00fa490 msgr2=0x7f00b01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 shutdown_connections 2024-09-13T17:19:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:19:34.896+0000 7f00b4df2640 1 -- 172.21.15.70:0/2705649080 wait complete. 2024-09-13T17:19:35.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:35.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:35.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:35.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:35.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:35.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:35.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:35.470 DEBUG:teuthology.orchestra.run.smithi070:osd.1> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.1.service 2024-09-13T17:19:35.472 INFO:tasks.cephadm:Deploying osd.2 on smithi100 with /dev/vg_nvme/lv_4... 2024-09-13T17:19:35.472 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T17:19:35.676 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:19:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:35 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:36.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:35 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:36.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:35 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:36.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:35 smithi100 ceph-mon[35055]: from='osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T17:19:36.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:35 smithi100 ceph-mon[35055]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T17:19:36.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:35 smithi100 ceph-mon[35055]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:36.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:35 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:36.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:35 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:36.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:35 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:36.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:35 smithi044 ceph-mon[31175]: from='osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T17:19:36.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:35 smithi044 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T17:19:36.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:35 smithi044 ceph-mon[31175]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:35 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:35 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:35 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:35 smithi070 ceph-mon[34502]: from='osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T17:19:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:35 smithi070 ceph-mon[34502]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T17:19:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:35 smithi070 ceph-mon[34502]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:36.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:36 smithi100 ceph-mon[35055]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T17:19:36.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:36 smithi100 ceph-mon[35055]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T17:19:36.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:36 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:36.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:36 smithi100 ceph-mon[35055]: from='osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:19:36.997 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:36 smithi100 ceph-mon[35055]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:19:37.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:36 smithi070 ceph-mon[34502]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T17:19:37.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:36 smithi070 ceph-mon[34502]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T17:19:37.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:36 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:37.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:36 smithi070 ceph-mon[34502]: from='osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:19:37.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:36 smithi070 ceph-mon[34502]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:19:37.355 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T17:19:37.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:36 smithi044 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T17:19:37.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:36 smithi044 ceph-mon[31175]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T17:19:37.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:36 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:37.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:36 smithi044 ceph-mon[31175]: from='osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:19:37.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:36 smithi044 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:19:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T17:19:37.460 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-13T17:19:37.460 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-13T17:19:37.460 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113081 s, 927 MB/s 2024-09-13T17:19:37.460 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-13T17:19:37.697 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1[38995]: 2024-09-13T17:19:37.358+0000 7f9baff42640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T17:19:37.697 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1[38995]: 2024-09-13T17:19:37.362+0000 7f9ba7d63640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:19:38.030 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:37 smithi100 ceph-mon[35055]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:19:38.030 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:37 smithi100 ceph-mon[35055]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T17:19:38.030 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:37 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:38.031 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:37 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:38.031 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:37 smithi100 ceph-mon[35055]: from='osd.1 ' entity='osd.1' 2024-09-13T17:19:38.031 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:37 smithi100 ceph-mon[35055]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:38.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-09-13T17:19:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-mon[34502]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:19:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-mon[34502]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T17:19:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-mon[34502]: from='osd.1 ' entity='osd.1' 2024-09-13T17:19:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:37 smithi070 ceph-mon[34502]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:38.229 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:19:38.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:37 smithi044 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:19:38.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:37 smithi044 ceph-mon[31175]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T17:19:38.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:37 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:38.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:37 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:38.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:37 smithi044 ceph-mon[31175]: from='osd.1 ' entity='osd.1' 2024-09-13T17:19:38.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:37 smithi044 ceph-mon[31175]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T17:19:39.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:19:39.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:19:39.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:39.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi070 2024-09-13T17:19:39.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi070 to 17088M 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] boot 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T17:19:39.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi070 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi070 to 17088M 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.267 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:39.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:39.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] boot 2024-09-13T17:19:39.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T17:19:39.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi070 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi070 to 17088M 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: osd.1 [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] boot 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T17:19:39.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:19:39.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 -- 172.21.15.100:0/4248071931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 msgr2=0x7f7d5c0fea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 --2- 172.21.15.100:0/4248071931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c0fea80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009ca0 tx=0x7f7d5802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 -- 172.21.15.100:0/4248071931 shutdown_connections 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 --2- 172.21.15.100:0/4248071931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 0x7f7d5c0ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 --2- 172.21.15.100:0/4248071931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 0x7f7d5c0ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 --2- 172.21.15.100:0/4248071931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c0fea80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 -- 172.21.15.100:0/4248071931 >> 172.21.15.100:0/4248071931 conn(0x7f7d5c0fa490 msgr2=0x7f7d5c0fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 -- 172.21.15.100:0/4248071931 shutdown_connections 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.799+0000 7f7d63540640 1 -- 172.21.15.100:0/4248071931 wait complete. 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 Processor -- start 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 -- start start 2024-09-13T17:19:39.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 0x7f7d5c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 0x7f7d5c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c19f980 con 0x7f7d5c0fefc0 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c19faf0 con 0x7f7d5c0ff960 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d63540640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c19fcb0 con 0x7f7d5c0fe680 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d612dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d612dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52962/0 (socket says 172.21.15.100:52962) 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d612dc640 1 -- 172.21.15.100:0/33373162 learned_addr learned my addr 172.21.15.100:0/33373162 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d60adb640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 0x7f7d5c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.800+0000 7f7d61add640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 0x7f7d5c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d612dc640 1 -- 172.21.15.100:0/33373162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 msgr2=0x7f7d5c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d612dc640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 0x7f7d5c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d612dc640 1 -- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 msgr2=0x7f7d5c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d612dc640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 0x7f7d5c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d612dc640 1 -- 172.21.15.100:0/33373162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58009900 con 0x7f7d5c0fe680 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d61add640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 0x7f7d5c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d60adb640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 0x7f7d5c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d612dc640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c197e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009dc0 tx=0x7f7d58003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580307c0 con 0x7f7d5c0fe680 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d58040070 con 0x7f7d5c0fe680 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c19ff30 con 0x7f7d5c0fe680 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004690 con 0x7f7d5c0fe680 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.801+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1a0340 con 0x7f7d5c0fe680 2024-09-13T17:19:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.802+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c0ff420 con 0x7f7d5c0fe680 2024-09-13T17:19:39.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.803+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f7d58042030 con 0x7f7d5c0fe680 2024-09-13T17:19:39.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.804+0000 7f7d527fc640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7d480a1020 0x7f7d480a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.804+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f7d580ec860 con 0x7f7d5c0fe680 2024-09-13T17:19:39.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.804+0000 7f7d60adb640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7d480a1020 0x7f7d480a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.805+0000 7f7d60adb640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7d480a1020 0x7f7d480a34e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0058c0 tx=0x7f7d4c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:39.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.807+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7d580b7840 con 0x7f7d5c0fe680 2024-09-13T17:19:39.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:39.976+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d5c02c4f0 con 0x7f7d480a1020 2024-09-13T17:19:40.605 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:40 smithi100 ceph-mon[35055]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T17:19:40.605 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:40 smithi100 ceph-mon[35055]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:40.605 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:19:40.605 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:19:40.605 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:40 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:40 smithi070 ceph-mon[34502]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T17:19:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:40 smithi070 ceph-mon[34502]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:40 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:19:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:40 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:19:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:40 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:40.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:40 smithi044 ceph-mon[31175]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T17:19:40.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:40 smithi044 ceph-mon[31175]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:40.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:19:40.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:19:40.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:40 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:41.512 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:41 smithi100 ceph-mon[35055]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:19:41.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:41 smithi070 ceph-mon[34502]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:19:41.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:41 smithi044 ceph-mon[31175]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:19:42.626 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:42 smithi100 ceph-mon[35055]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:42.626 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:42 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/773493334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]: dispatch 2024-09-13T17:19:42.626 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:42 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]: dispatch 2024-09-13T17:19:42.626 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:42 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]': finished 2024-09-13T17:19:42.626 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:42 smithi100 ceph-mon[35055]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T17:19:42.626 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:42 smithi070 ceph-mon[34502]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:42 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/773493334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]: dispatch 2024-09-13T17:19:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:42 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]: dispatch 2024-09-13T17:19:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:42 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]': finished 2024-09-13T17:19:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:42 smithi070 ceph-mon[34502]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T17:19:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:42 smithi044 ceph-mon[31175]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:42 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/773493334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]: dispatch 2024-09-13T17:19:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:42 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]: dispatch 2024-09-13T17:19:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:42 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cfc25ea-477f-4e25-b425-2481547b68c0"}]': finished 2024-09-13T17:19:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:42 smithi044 ceph-mon[31175]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T17:19:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:43.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:43 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/3361066517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:19:43.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:43 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/3361066517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:19:43.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:43 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/3361066517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:19:44.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:44 smithi070 ceph-mon[34502]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:44.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:44 smithi100 ceph-mon[35055]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:44.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:44 smithi044 ceph-mon[31175]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:45.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:45 smithi044 ceph-mon[31175]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:45 smithi070 ceph-mon[34502]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:45 smithi100 ceph-mon[35055]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:47.842 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:47 smithi100 ceph-mon[35055]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:47.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:47 smithi044 ceph-mon[31175]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:47 smithi070 ceph-mon[34502]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:48.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:48 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T17:19:48.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:48 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:48.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:48 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T17:19:48.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:48 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:49.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:48 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T17:19:49.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:48 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:49 smithi070 ceph-mon[34502]: Deploying daemon osd.2 on smithi100 2024-09-13T17:19:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:49 smithi070 ceph-mon[34502]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:49.957 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:49 smithi100 ceph-mon[35055]: Deploying daemon osd.2 on smithi100 2024-09-13T17:19:49.957 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:49 smithi100 ceph-mon[35055]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:50.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:49 smithi044 ceph-mon[31175]: Deploying daemon osd.2 on smithi100 2024-09-13T17:19:50.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:49 smithi044 ceph-mon[31175]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:51.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:51 smithi044 ceph-mon[31175]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:51.863 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:51 smithi100 ceph-mon[35055]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:51 smithi070 ceph-mon[34502]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:53.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:53.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:19:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:54 smithi100 ceph-mon[35055]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:54.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:54 smithi044 ceph-mon[31175]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:54.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:54 smithi070 ceph-mon[34502]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 2 on host 'smithi100' 2024-09-13T17:19:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.719+0000 7f7d527fc640 1 -- 172.21.15.100:0/33373162 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d5c02c4f0 con 0x7f7d480a1020 2024-09-13T17:19:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7d480a1020 msgr2=0x7f7d480a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7d480a1020 0x7f7d480a34e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0058c0 tx=0x7f7d4c005850 comp rx=0 tx=0).stop 2024-09-13T17:19:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 msgr2=0x7f7d5c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c197e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009dc0 tx=0x7f7d58003c20 comp rx=0 tx=0).stop 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 shutdown_connections 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7d480a1020 0x7f7d480a34e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d5c0ff960 0x7f7d5c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d5c0fefc0 0x7f7d5c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 --2- 172.21.15.100:0/33373162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5c0fe680 0x7f7d5c197e80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 >> 172.21.15.100:0/33373162 conn(0x7f7d5c0fa490 msgr2=0x7f7d5c0fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 shutdown_connections 2024-09-13T17:19:55.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:19:55.721+0000 7f7d63540640 1 -- 172.21.15.100:0/33373162 wait complete. 2024-09-13T17:19:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:55 smithi044 ceph-mon[31175]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:55 smithi070 ceph-mon[34502]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:55 smithi100 ceph-mon[35055]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:56.349 DEBUG:teuthology.orchestra.run.smithi100:osd.2> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.2.service 2024-09-13T17:19:56.351 INFO:tasks.cephadm:Deploying osd.3 on smithi044 with /dev/vg_nvme/lv_3... 2024-09-13T17:19:56.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T17:19:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:19:56.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:56.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:57.396 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:19:57 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2[40248]: 2024-09-13T17:19:57.102+0000 7f92807b3640 -1 osd.2 0 log_to_monitors true 2024-09-13T17:19:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:57 smithi100 ceph-mon[35055]: from='osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T17:19:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:57 smithi100 ceph-mon[35055]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T17:19:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:57 smithi100 ceph-mon[35055]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:58.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:58.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:58.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:57 smithi044 ceph-mon[31175]: from='osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T17:19:58.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:57 smithi044 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T17:19:58.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:57 smithi044 ceph-mon[31175]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:57 smithi070 ceph-mon[34502]: from='osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T17:19:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:57 smithi070 ceph-mon[34502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T17:19:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:57 smithi070 ceph-mon[34502]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T17:19:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T17:19:58.693 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-09-13T17:19:58.693 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-09-13T17:19:58.693 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129608 s, 809 MB/s 2024-09-13T17:19:58.693 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-09-13T17:19:58.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:58 smithi044 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T17:19:58.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:58 smithi044 ceph-mon[31175]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T17:19:58.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:58 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:58.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:58 smithi044 ceph-mon[31175]: from='osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:19:58.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:58 smithi044 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:19:59.038 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:58 smithi100 ceph-mon[35055]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T17:19:59.039 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:58 smithi100 ceph-mon[35055]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T17:19:59.039 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:58 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:59.039 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:58 smithi100 ceph-mon[35055]: from='osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:19:59.039 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:58 smithi100 ceph-mon[35055]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:19:59.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:58 smithi070 ceph-mon[34502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T17:19:59.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:58 smithi070 ceph-mon[34502]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T17:19:59.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:58 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:59.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:58 smithi070 ceph-mon[34502]: from='osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:19:59.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:58 smithi070 ceph-mon[34502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:19:59.325 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-09-13T17:19:59.432 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2[40248]: 2024-09-13T17:19:59.228+0000 7f927c734640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T17:19:59.432 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2[40248]: 2024-09-13T17:19:59.232+0000 7f9274555640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:19:59.526 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:19:59.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='osd.2 ' entity='osd.2' 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:19:59.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:19:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='osd.2 ' entity='osd.2' 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:20:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:19:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:20:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T17:20:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='osd.2 ' entity='osd.2' 2024-09-13T17:20:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:00.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:19:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:01.020 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi100 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi100 to 17087M 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: overall HEALTH_OK 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] boot 2024-09-13T17:20:01.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T17:20:01.022 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:01.098 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi100 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi100 to 17087M 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: overall HEALTH_OK 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] boot 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T17:20:01.099 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:01.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 -- 172.21.15.44:0/4151066924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480ffee0 msgr2=0x7ff748100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 --2- 172.21.15.44:0/4151066924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480ffee0 0x7ff748100360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009900 tx=0x7ff73c02a410 comp rx=0 tx=0).stop 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 -- 172.21.15.44:0/4151066924 shutdown_connections 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 --2- 172.21.15.44:0/4151066924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7481008a0 0x7ff748107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 --2- 172.21.15.44:0/4151066924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480ffee0 0x7ff748100360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 --2- 172.21.15.44:0/4151066924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7480fece0 0x7ff7480ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 -- 172.21.15.44:0/4151066924 >> 172.21.15.44:0/4151066924 conn(0x7ff7480fa490 msgr2=0x7ff7480fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 -- 172.21.15.44:0/4151066924 shutdown_connections 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 -- 172.21.15.44:0/4151066924 wait complete. 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.246+0000 7ff74df31640 1 Processor -- start 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 -- start start 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 0x7ff748102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7480ffee0 0x7ff7481012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7481008a0 0x7ff748101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff748101f90 con 0x7ff7480fece0 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff748102100 con 0x7ff7481008a0 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff74df31640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74819fe00 con 0x7ff7480ffee0 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 0x7ff748102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 0x7ff748102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49784/0 (socket says 172.21.15.44:49784) 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff746ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7480ffee0 0x7ff7481012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 -- 172.21.15.44:0/1159091895 learned_addr learned my addr 172.21.15.44:0/1159091895 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff747fff640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7481008a0 0x7ff748101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 -- 172.21.15.44:0/1159091895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7480ffee0 msgr2=0x7ff7481012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7480ffee0 0x7ff7481012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 -- 172.21.15.44:0/1159091895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7481008a0 msgr2=0x7ff748101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7481008a0 0x7ff748101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.247+0000 7ff7477fe640 1 -- 172.21.15.44:0/1159091895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009560 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.248+0000 7ff7477fe640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 0x7ff748102c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff73800b4b0 tx=0x7ff73800b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.248+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738010610 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.248+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7481a00b0 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.248+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff738010c00 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.248+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738017bb0 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.248+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7481a0600 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.249+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74810a550 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.251+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7ff738017d50 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.252+0000 7ff744ff9640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff7300a0ff0 0x7ff7300a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.252+0000 7ff746ffd640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff7300a0ff0 0x7ff7300a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.252+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7ff7380ca860 con 0x7ff7480fece0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.252+0000 7ff746ffd640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff7300a0ff0 0x7ff7300a34b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006010 tx=0x7ff73c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:20:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.256+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff738095630 con 0x7ff7480fece0 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi100 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi100 to 17087M 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: overall HEALTH_OK 2024-09-13T17:20:01.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: osd.2 [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] boot 2024-09-13T17:20:01.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T17:20:01.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:20:01.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:01.437+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff748068610 con 0x7ff7300a0ff0 2024-09-13T17:20:02.082 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:01 smithi044 ceph-mon[31175]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:02.082 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:01 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:02.082 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:01 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:02.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:01 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:02.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:01 smithi044 ceph-mon[31175]: pgmap v78: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:01 smithi100 ceph-mon[35055]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:01 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:01 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:01 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:01 smithi100 ceph-mon[35055]: pgmap v78: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:02.393 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:01 smithi070 ceph-mon[34502]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:02.394 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:01 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:02.394 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:01 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:02.394 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:01 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:02.394 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:01 smithi070 ceph-mon[34502]: pgmap v78: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:03.105 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:20:03.105 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T17:20:03.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:03 smithi044 ceph-mon[31175]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T17:20:03.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:03 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:20:03.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:03 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:20:03.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:03 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T17:20:03.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:03 smithi100 ceph-mon[35055]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T17:20:03.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:03 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:20:03.368 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:03 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:20:03.368 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:03 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T17:20:03.368 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:03 smithi070 ceph-mon[34502]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T17:20:03.368 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:03 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:20:03.857 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T17:20:03.857 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:03.857 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49902]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:03.857 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49902]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:04.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:04.015+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff738095a00 con 0x7ff7480fece0 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2051477974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f620c4-46d1-40d5-9346-56d4f1cb4841"}]: dispatch 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2051477974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65f620c4-46d1-40d5-9346-56d4f1cb4841"}]': finished 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: osdmap e21: 4 total, 3 up, 4 in 2024-09-13T17:20:04.264 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2051477974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f620c4-46d1-40d5-9346-56d4f1cb4841"}]: dispatch 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2051477974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65f620c4-46d1-40d5-9346-56d4f1cb4841"}]': finished 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: osdmap e21: 4 total, 3 up, 4 in 2024-09-13T17:20:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:04.356 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T17:20:04.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:04.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49934]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:04.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:20:03 smithi044 sudo[49934]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2051477974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f620c4-46d1-40d5-9346-56d4f1cb4841"}]: dispatch 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2051477974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65f620c4-46d1-40d5-9346-56d4f1cb4841"}]': finished 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: osdmap e21: 4 total, 3 up, 4 in 2024-09-13T17:20:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:04.522 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42652]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T17:20:04.522 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42652]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:04.522 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42652]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:04.522 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42652]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:04.947 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42655]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T17:20:04.947 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42655]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:04.947 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42655]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:04.947 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:20:04 smithi070 sudo[42655]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:05.042 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:04 smithi100 sudo[43828]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T17:20:05.042 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:04 smithi100 sudo[43828]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:05.042 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:04 smithi100 sudo[43828]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:05.042 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:04 smithi100 sudo[43828]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:05.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:05 smithi100 ceph-mon[35055]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-09-13T17:20:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:05 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/15105583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:05.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 ceph-mon[31175]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-09-13T17:20:05.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:05.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/15105583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:05.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:05.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:05 smithi070 ceph-mon[34502]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-09-13T17:20:05.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:05.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:05 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/15105583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:05.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:05.524 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:05 smithi100 sudo[43831]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T17:20:05.524 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:05 smithi100 sudo[43831]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:05.524 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:05 smithi100 sudo[43831]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:05.524 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:20:05 smithi100 sudo[43831]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:05.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T17:20:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50377]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50377]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:06.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50565]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T17:20:06.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50565]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:05 smithi044 sudo[50565]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:06.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43834]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T17:20:06.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43834]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:06.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43834]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:06.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43834]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43837]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T17:20:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43837]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43837]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 sudo[43837]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:06.962 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T17:20:06.962 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:06.963 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42658]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:06.963 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42658]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:07.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:07.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:07.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:07.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:07.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:07.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:07.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:07.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:07.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:07.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:07.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:07.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T17:20:07.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T17:20:07.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42661]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T17:20:07.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 sudo[42661]: pam_unix(sudo:session): session closed for user root 2024-09-13T17:20:07.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:07.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:07.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:07 smithi100 ceph-mon[35055]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:08.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:07 smithi044 ceph-mon[31175]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T17:20:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:20:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:20:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:20:08.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T17:20:08.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:07 smithi070 ceph-mon[34502]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:09.744 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:09 smithi044 ceph-mon[31175]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 457 KiB/s wr, 31 op/s 2024-09-13T17:20:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:09 smithi070 ceph-mon[34502]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 457 KiB/s wr, 31 op/s 2024-09-13T17:20:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:09 smithi100 ceph-mon[35055]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 457 KiB/s wr, 31 op/s 2024-09-13T17:20:11.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T17:20:11.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:11.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T17:20:11.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:11.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T17:20:11.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:12.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:12 smithi044 ceph-mon[31175]: Deploying daemon osd.3 on smithi044 2024-09-13T17:20:12.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:12 smithi044 ceph-mon[31175]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-13T17:20:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:12 smithi070 ceph-mon[34502]: Deploying daemon osd.3 on smithi044 2024-09-13T17:20:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:12 smithi070 ceph-mon[34502]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-13T17:20:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:12 smithi100 ceph-mon[35055]: Deploying daemon osd.3 on smithi044 2024-09-13T17:20:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:12 smithi100 ceph-mon[35055]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-13T17:20:13.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:13 smithi044 ceph-mon[31175]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-13T17:20:13.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:13 smithi070 ceph-mon[34502]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-13T17:20:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:13 smithi100 ceph-mon[35055]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-13T17:20:15.647 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:15 smithi044 ceph-mon[31175]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-13T17:20:15.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:15 smithi070 ceph-mon[34502]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-13T17:20:16.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:15 smithi100 ceph-mon[35055]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-13T17:20:16.833 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:16.833 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:16.834 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:16.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:16.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:16.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:17.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:17.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:17.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:17.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:17 smithi044 ceph-mon[31175]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-13T17:20:17.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:17 smithi070 ceph-mon[34502]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-13T17:20:18.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:17 smithi100 ceph-mon[35055]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-13T17:20:19.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.682+0000 7ff744ff9640 1 -- 172.21.15.44:0/1159091895 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff748068610 con 0x7ff7300a0ff0 2024-09-13T17:20:19.684 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 3 on host 'smithi044' 2024-09-13T17:20:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff7300a0ff0 msgr2=0x7ff7300a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff7300a0ff0 0x7ff7300a34b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006010 tx=0x7ff73c005a20 comp rx=0 tx=0).stop 2024-09-13T17:20:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 msgr2=0x7ff748102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 0x7ff748102c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff73800b4b0 tx=0x7ff73800b970 comp rx=0 tx=0).stop 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 shutdown_connections 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff7300a0ff0 0x7ff7300a34b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7480ffee0 0x7ff7481012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7481008a0 0x7ff748101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 --2- 172.21.15.44:0/1159091895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7480fece0 0x7ff748102c70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 >> 172.21.15.44:0/1159091895 conn(0x7ff7480fa490 msgr2=0x7ff748067f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.684+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 shutdown_connections 2024-09-13T17:20:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:20:19.685+0000 7ff74df31640 1 -- 172.21.15.44:0/1159091895 wait complete. 2024-09-13T17:20:19.996 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:19.996 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:19.996 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:19.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:19.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:19.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-13T17:20:19.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:19.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:19.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-13T17:20:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:20.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:20.319 DEBUG:teuthology.orchestra.run.smithi044:osd.3> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.3.service 2024-09-13T17:20:20.322 INFO:tasks.cephadm:Deploying osd.4 on smithi070 with /dev/vg_nvme/lv_3... 2024-09-13T17:20:20.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T17:20:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:20:20.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:20 smithi070 ceph-mon[34502]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T17:20:21.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:20 smithi044 ceph-mon[31175]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T17:20:21.176 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:20 smithi100 ceph-mon[35055]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T17:20:22.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:21 smithi044 ceph-mon[31175]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T17:20:22.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:21 smithi044 ceph-mon[31175]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T17:20:22.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:22.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:21 smithi044 ceph-mon[31175]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:20:22.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:21 smithi044 ceph-mon[31175]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:21 smithi070 ceph-mon[34502]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T17:20:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:21 smithi070 ceph-mon[34502]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T17:20:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:21 smithi070 ceph-mon[34502]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:20:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:21 smithi070 ceph-mon[34502]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:22.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:21 smithi100 ceph-mon[35055]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T17:20:22.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:21 smithi100 ceph-mon[35055]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T17:20:22.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:22.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:21 smithi100 ceph-mon[35055]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:20:22.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:21 smithi100 ceph-mon[35055]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-13T17:20:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T17:20:22.318 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T17:20:22.329 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-09-13T17:20:22.329 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-09-13T17:20:22.329 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00890101 s, 1.2 GB/s 2024-09-13T17:20:22.329 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-09-13T17:20:22.630 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3[51955]: 2024-09-13T17:20:22.365+0000 7f3e994cc640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T17:20:22.630 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3[51955]: 2024-09-13T17:20:22.371+0000 7f3e912ed640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:20:22.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2024-09-13T17:20:23.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:20:23.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T17:20:23.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' 2024-09-13T17:20:23.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] boot 2024-09-13T17:20:23.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: osdmap e24: 4 total, 4 up, 4 in 2024-09-13T17:20:23.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.207 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:20:23.221 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:20:23.221 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T17:20:23.221 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.221 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.221 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' 2024-09-13T17:20:23.221 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] boot 2024-09-13T17:20:23.222 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: osdmap e24: 4 total, 4 up, 4 in 2024-09-13T17:20:23.222 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: from='osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337]' entity='osd.3' 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: osd.3 [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] boot 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: osdmap e24: 4 total, 4 up, 4 in 2024-09-13T17:20:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:20:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:20:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:20:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi044 2024-09-13T17:20:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi044 to 8546M 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:23 smithi070 ceph-mon[34502]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi044 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:24.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi044 to 8546M 2024-09-13T17:20:24.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:24.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:24.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:23 smithi100 ceph-mon[35055]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi044 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi044 to 8546M 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:24.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:24.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:24.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:23 smithi044 ceph-mon[31175]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/3865934060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 msgr2=0x7fbf08101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/3865934060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf08101500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009900 tx=0x7fbef802a410 comp rx=0 tx=0).stop 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/3865934060 shutdown_connections 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/3865934060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf08101a40 0x7fbf081082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/3865934060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf08101500 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/3865934060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf080fe0a0 0x7fbf080fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/3865934060 >> 172.21.15.70:0/3865934060 conn(0x7fbf080f9c10 msgr2=0x7fbf080fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/3865934060 shutdown_connections 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.907+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/3865934060 wait complete. 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 Processor -- start 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 -- start start 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf080fe0a0 0x7fbf081009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf080ff030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf08101a40 0x7fbf080ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080ffd00 con 0x7fbf08101a40 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080ffe70 con 0x7fbf08101080 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0f9c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0819fdb0 con 0x7fbf080fe0a0 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf080ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf080ff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43378/0 (socket says 172.21.15.70:43378) 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 -- 172.21.15.70:0/2798969279 learned_addr learned my addr 172.21.15.70:0/2798969279 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0d762640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf080fe0a0 0x7fbf081009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0df63640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf08101a40 0x7fbf080ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 -- 172.21.15.70:0/2798969279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf080fe0a0 msgr2=0x7fbf081009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf080fe0a0 0x7fbf081009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 -- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf08101a40 msgr2=0x7fbf080ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf08101a40 0x7fbf080ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0cf61640 1 -- 172.21.15.70:0/2798969279 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef8009560 con 0x7fbf08101080 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.908+0000 7fbf0df63640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf08101a40 0x7fbf080ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.909+0000 7fbf0cf61640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf080ff030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbef8004970 tx=0x7fbef80049a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.909+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80037f0 con 0x7fbf08101080 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.909+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf081a0000 con 0x7fbf08101080 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.909+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbef80044f0 con 0x7fbf08101080 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.909+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef803caa0 con 0x7fbf08101080 2024-09-13T17:20:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.909+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf081a04a0 con 0x7fbf08101080 2024-09-13T17:20:24.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.910+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf08107310 con 0x7fbf08101080 2024-09-13T17:20:24.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.911+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbef8003950 con 0x7fbf08101080 2024-09-13T17:20:24.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.911+0000 7fbefe7fc640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbef40a1080 0x7fbef40a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:24.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.912+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fbef80ecfa0 con 0x7fbf08101080 2024-09-13T17:20:24.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.912+0000 7fbf0d762640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbef40a1080 0x7fbef40a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:24.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.912+0000 7fbf0d762640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbef40a1080 0x7fbef40a3540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf08068520 tx=0x7fbf0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:20:24.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:24.915+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbef80b7930 con 0x7fbf08101080 2024-09-13T17:20:25.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:25.085+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf080608f0 con 0x7fbef40a1080 2024-09-13T17:20:25.087 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:24 smithi070 ceph-mon[34502]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T17:20:25.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:24 smithi100 ceph-mon[35055]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T17:20:25.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:24 smithi044 ceph-mon[31175]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T17:20:26.132 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:25 smithi070 ceph-mon[34502]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T17:20:26.132 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:25 smithi070 ceph-mon[34502]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:26.132 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:26.132 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:26.132 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:26.132 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:25 smithi070 ceph-mon[34502]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:25 smithi100 ceph-mon[35055]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T17:20:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:25 smithi100 ceph-mon[35055]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:25 smithi100 ceph-mon[35055]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:25 smithi044 ceph-mon[31175]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T17:20:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:25 smithi044 ceph-mon[31175]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:25 smithi044 ceph-mon[31175]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:27.075 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:26 smithi070 ceph-mon[34502]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T17:20:27.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:26 smithi100 ceph-mon[35055]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T17:20:27.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:26 smithi044 ceph-mon[31175]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/2936158009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]: dispatch 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]: dispatch 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]': finished 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:27 smithi070 ceph-mon[34502]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/2936158009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]: dispatch 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]: dispatch 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]': finished 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:27 smithi100 ceph-mon[35055]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/2936158009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]: dispatch 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]: dispatch 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9180bdb-24d0-42c8-9d13-51f4ae9812ba"}]': finished 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:28.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:27 smithi044 ceph-mon[31175]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:29.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:28 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/2737222897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:29.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:28 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/2737222897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:29.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:28 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/2737222897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:30.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:29 smithi070 ceph-mon[34502]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-13T17:20:30.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:29 smithi100 ceph-mon[35055]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-13T17:20:30.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:29 smithi044 ceph-mon[31175]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-13T17:20:31.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:31 smithi044 ceph-mon[31175]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:20:31.915 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:31 smithi070 ceph-mon[34502]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:20:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:31 smithi100 ceph-mon[35055]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:20:33.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:33 smithi100 ceph-mon[35055]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-13T17:20:33.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:33 smithi044 ceph-mon[31175]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-13T17:20:33.888 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:33 smithi070 ceph-mon[34502]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-13T17:20:34.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T17:20:34.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:34.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:34 smithi044 ceph-mon[31175]: Deploying daemon osd.4 on smithi070 2024-09-13T17:20:34.925 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T17:20:34.925 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:34.925 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:34 smithi070 ceph-mon[34502]: Deploying daemon osd.4 on smithi070 2024-09-13T17:20:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T17:20:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:34 smithi100 ceph-mon[35055]: Deploying daemon osd.4 on smithi070 2024-09-13T17:20:35.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:35 smithi044 ceph-mon[31175]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-13T17:20:35.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:35 smithi070 ceph-mon[34502]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-13T17:20:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:35 smithi100 ceph-mon[35055]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-13T17:20:37.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:37 smithi070 ceph-mon[34502]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-13T17:20:37.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:37 smithi100 ceph-mon[35055]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-13T17:20:37.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:37 smithi044 ceph-mon[31175]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-13T17:20:39.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:39.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:39.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:39 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:39.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:39.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:39 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:39.617 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:39 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:39.617 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:39 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:39.618 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:39 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:40.553 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:40 smithi070 ceph-mon[34502]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:20:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:40 smithi044 ceph-mon[31175]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:20:40.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:40 smithi100 ceph-mon[35055]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:20:41.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:41 smithi044 ceph-mon[31175]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:41 smithi070 ceph-mon[34502]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:41 smithi100 ceph-mon[35055]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.867+0000 7fbefe7fc640 1 -- 172.21.15.70:0/2798969279 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbf080608f0 con 0x7fbef40a1080 2024-09-13T17:20:42.869 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 4 on host 'smithi070' 2024-09-13T17:20:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.869+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbef40a1080 msgr2=0x7fbef40a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.869+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbef40a1080 0x7fbef40a3540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf08068520 tx=0x7fbf0400b390 comp rx=0 tx=0).stop 2024-09-13T17:20:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.869+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 msgr2=0x7fbf080ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.869+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf080ff030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbef8004970 tx=0x7fbef80049a0 comp rx=0 tx=0).stop 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 shutdown_connections 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbef40a1080 0x7fbef40a3540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf08101a40 0x7fbf080ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf08101080 0x7fbf080ff030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 --2- 172.21.15.70:0/2798969279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf080fe0a0 0x7fbf081009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 >> 172.21.15.70:0/2798969279 conn(0x7fbf080f9c10 msgr2=0x7fbf081044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 shutdown_connections 2024-09-13T17:20:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:20:42.870+0000 7fbf0f9c6640 1 -- 172.21.15.70:0/2798969279 wait complete. 2024-09-13T17:20:43.094 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.094 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.094 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:43.094 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:43.094 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.094 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:43.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:43.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:43.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:43.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:20:43.514 DEBUG:teuthology.orchestra.run.smithi070:osd.4> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.4.service 2024-09-13T17:20:43.517 INFO:tasks.cephadm:Deploying osd.5 on smithi100 with /dev/vg_nvme/lv_3... 2024-09-13T17:20:43.517 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T17:20:43.721 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:20:44.004 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:44.004 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:44.004 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:43 smithi070 ceph-mon[34502]: from='osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T17:20:44.004 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:43 smithi070 ceph-mon[34502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T17:20:44.005 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:43 smithi070 ceph-mon[34502]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:43 smithi100 ceph-mon[35055]: from='osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T17:20:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:43 smithi100 ceph-mon[35055]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T17:20:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:43 smithi100 ceph-mon[35055]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:44.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:43 smithi044 ceph-mon[31175]: from='osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T17:20:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:43 smithi044 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T17:20:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:43 smithi044 ceph-mon[31175]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:45.077 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:44 smithi070 ceph-mon[34502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T17:20:45.078 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:44 smithi070 ceph-mon[34502]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T17:20:45.078 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:45.078 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:44 smithi070 ceph-mon[34502]: from='osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:20:45.078 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:44 smithi070 ceph-mon[34502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:20:45.173 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:44 smithi100 ceph-mon[35055]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T17:20:45.173 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:44 smithi100 ceph-mon[35055]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T17:20:45.173 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:45.173 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:44 smithi100 ceph-mon[35055]: from='osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:20:45.174 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:44 smithi100 ceph-mon[35055]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:20:45.329 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4[45424]: 2024-09-13T17:20:45.294+0000 7fb30f01d640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T17:20:45.330 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4[45424]: 2024-09-13T17:20:45.300+0000 7fb306e3e640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:20:45.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:44 smithi044 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T17:20:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:44 smithi044 ceph-mon[31175]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T17:20:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:44 smithi044 ceph-mon[31175]: from='osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:20:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:44 smithi044 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:20:45.566 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T17:20:45.660 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T17:20:45.673 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-13T17:20:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-13T17:20:45.674 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115668 s, 907 MB/s 2024-09-13T17:20:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-13T17:20:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-mon[34502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:20:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-mon[34502]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T17:20:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-mon[34502]: from='osd.4 ' entity='osd.4' 2024-09-13T17:20:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:45 smithi070 ceph-mon[34502]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:45 smithi100 ceph-mon[35055]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:20:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:45 smithi100 ceph-mon[35055]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T17:20:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:45 smithi100 ceph-mon[35055]: from='osd.4 ' entity='osd.4' 2024-09-13T17:20:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:45 smithi100 ceph-mon[35055]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:46.299 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-09-13T17:20:46.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:45 smithi044 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:20:46.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:45 smithi044 ceph-mon[31175]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T17:20:46.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:46.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:46.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:45 smithi044 ceph-mon[31175]: from='osd.4 ' entity='osd.4' 2024-09-13T17:20:46.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:45 smithi044 ceph-mon[31175]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-13T17:20:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] boot 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: osdmap e32: 5 total, 5 up, 5 in 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:47.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:47.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:47.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:47.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] boot 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: osdmap e32: 5 total, 5 up, 5 in 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:47.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: osd.4 [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] boot 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: osdmap e32: 5 total, 5 up, 5 in 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:20:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:20:48.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:47 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi070 2024-09-13T17:20:48.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:47 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi070 to 8544M 2024-09-13T17:20:48.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:47 smithi100 ceph-mon[35055]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 -- 172.21.15.100:0/2532419333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff130100fd0 msgr2=0x7ff130103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 --2- 172.21.15.100:0/2532419333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff130100fd0 0x7ff130103440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff124009900 tx=0x7ff12402a410 comp rx=0 tx=0).stop 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 -- 172.21.15.100:0/2532419333 shutdown_connections 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 --2- 172.21.15.100:0/2532419333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff130100fd0 0x7ff130103440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 --2- 172.21.15.100:0/2532419333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130103980 0x7ff130103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 --2- 172.21.15.100:0/2532419333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300fe680 0x7ff130100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.152+0000 7ff135ef9640 1 -- 172.21.15.100:0/2532419333 >> 172.21.15.100:0/2532419333 conn(0x7ff1300fa490 msgr2=0x7ff1300fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:20:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 -- 172.21.15.100:0/2532419333 shutdown_connections 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 -- 172.21.15.100:0/2532419333 wait complete. 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 Processor -- start 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 -- start start 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 0x7ff13019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130100fd0 0x7ff13019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130103980 0x7ff1301a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.153+0000 7ff135ef9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301a3e40 con 0x7ff130100fd0 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff135ef9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301a3fb0 con 0x7ff130103980 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff135ef9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301a4170 con 0x7ff1300fe680 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 0x7ff13019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 0x7ff13019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36304/0 (socket says 172.21.15.100:36304) 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 -- 172.21.15.100:0/1096857790 learned_addr learned my addr 172.21.15.100:0/1096857790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12effd640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130100fd0 0x7ff13019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12ffff640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130103980 0x7ff1301a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 -- 172.21.15.100:0/1096857790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130103980 msgr2=0x7ff1301a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130103980 0x7ff1301a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 -- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130100fd0 msgr2=0x7ff13019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130100fd0 0x7ff13019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 -- 172.21.15.100:0/1096857790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124009560 con 0x7ff1300fe680 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12f7fe640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 0x7ff13019c340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff12000ba50 tx=0x7ff12000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000cbe0 con 0x7ff1300fe680 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff12000cd40 con 0x7ff1300fe680 2024-09-13T17:20:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1301a4450 con 0x7ff1300fe680 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120007610 con 0x7ff1300fe680 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.154+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1301a49a0 con 0x7ff1300fe680 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.155+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1300fbf80 con 0x7ff1300fe680 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.156+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff120014440 con 0x7ff1300fe680 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.157+0000 7ff12cff9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff1180a1130 0x7ff1180a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.157+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7ff1200cb5c0 con 0x7ff1300fe680 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.157+0000 7ff12effd640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff1180a1130 0x7ff1180a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.158+0000 7ff12effd640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff1180a1130 0x7ff1180a35f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff124006010 tx=0x7ff124005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:20:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.160+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff120095e70 con 0x7ff1300fe680 2024-09-13T17:20:48.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:47 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi070 2024-09-13T17:20:48.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:47 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi070 to 8544M 2024-09-13T17:20:48.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:47 smithi070 ceph-mon[34502]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:20:48.332+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff13002c450 con 0x7ff1180a1130 2024-09-13T17:20:48.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:47 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi070 2024-09-13T17:20:48.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:47 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi070 to 8544M 2024-09-13T17:20:48.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:47 smithi044 ceph-mon[31175]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:48.958 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:48 smithi100 ceph-mon[35055]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T17:20:48.958 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:48 smithi100 ceph-mon[35055]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:48.958 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:48 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:48.958 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:48 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:48.959 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:48 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:49.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:48 smithi070 ceph-mon[34502]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T17:20:49.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:48 smithi070 ceph-mon[34502]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:49.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:48 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:49.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:48 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:49.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:48 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:49.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:48 smithi044 ceph-mon[31175]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T17:20:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:48 smithi044 ceph-mon[31175]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:20:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:48 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:20:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:48 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:20:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:48 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:50.015 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:49 smithi100 ceph-mon[35055]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:50.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:49 smithi070 ceph-mon[34502]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:50.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:49 smithi044 ceph-mon[31175]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/1369531196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]: dispatch 2024-09-13T17:20:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:50 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]: dispatch 2024-09-13T17:20:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:50 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]': finished 2024-09-13T17:20:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:50 smithi100 ceph-mon[35055]: osdmap e34: 6 total, 5 up, 6 in 2024-09-13T17:20:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:20:51.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/1369531196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]: dispatch 2024-09-13T17:20:51.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:50 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]: dispatch 2024-09-13T17:20:51.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:50 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]': finished 2024-09-13T17:20:51.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:50 smithi070 ceph-mon[34502]: osdmap e34: 6 total, 5 up, 6 in 2024-09-13T17:20:51.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:20:51.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/1369531196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]: dispatch 2024-09-13T17:20:51.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:50 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]: dispatch 2024-09-13T17:20:51.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:50 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55e04db8-8d0a-42bc-a507-aa94db69fa62"}]': finished 2024-09-13T17:20:51.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:50 smithi044 ceph-mon[31175]: osdmap e34: 6 total, 5 up, 6 in 2024-09-13T17:20:51.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:20:52.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/4068833811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:52.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:51 smithi070 ceph-mon[34502]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:52.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/4068833811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:52.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:51 smithi100 ceph-mon[35055]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:52.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/4068833811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:20:52.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:51 smithi044 ceph-mon[31175]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:53 smithi044 ceph-mon[31175]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:53 smithi070 ceph-mon[34502]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:53.959 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:53 smithi100 ceph-mon[35055]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:55 smithi044 ceph-mon[31175]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:55 smithi070 ceph-mon[34502]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:55.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:55 smithi100 ceph-mon[35055]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:57.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T17:20:57.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:57.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T17:20:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:57.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T17:20:57.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:20:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:58 smithi100 ceph-mon[35055]: Deploying daemon osd.5 on smithi100 2024-09-13T17:20:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:58 smithi100 ceph-mon[35055]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:58 smithi044 ceph-mon[31175]: Deploying daemon osd.5 on smithi100 2024-09-13T17:20:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:58 smithi044 ceph-mon[31175]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:58 smithi070 ceph-mon[34502]: Deploying daemon osd.5 on smithi100 2024-09-13T17:20:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:58 smithi070 ceph-mon[34502]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:59.833 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:20:59 smithi100 ceph-mon[35055]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:59.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:20:59 smithi044 ceph-mon[31175]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:20:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:20:59 smithi070 ceph-mon[34502]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:01.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:01 smithi044 ceph-mon[31175]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:01.866 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:01 smithi100 ceph-mon[35055]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:01 smithi070 ceph-mon[34502]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:02.842 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:02.842 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:02.842 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:02.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:02.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:02.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:02.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:02.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:02.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:03 smithi100 ceph-mon[35055]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:03.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:03 smithi044 ceph-mon[31175]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:03 smithi070 ceph-mon[34502]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:05.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.404+0000 7ff12cff9640 1 -- 172.21.15.100:0/1096857790 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff13002c450 con 0x7ff1180a1130 2024-09-13T17:21:05.407 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 5 on host 'smithi100' 2024-09-13T17:21:05.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff1180a1130 msgr2=0x7ff1180a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff1180a1130 0x7ff1180a35f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff124006010 tx=0x7ff124005a20 comp rx=0 tx=0).stop 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 msgr2=0x7ff13019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 0x7ff13019c340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff12000ba50 tx=0x7ff12000bf10 comp rx=0 tx=0).stop 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 shutdown_connections 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff1180a1130 0x7ff1180a35f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130103980 0x7ff1301a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130100fd0 0x7ff13019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 --2- 172.21.15.100:0/1096857790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1300fe680 0x7ff13019c340 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 >> 172.21.15.100:0/1096857790 conn(0x7ff1300fa490 msgr2=0x7ff1300ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 shutdown_connections 2024-09-13T17:21:05.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:05.406+0000 7ff135ef9640 1 -- 172.21.15.100:0/1096857790 wait complete. 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:05 smithi070 ceph-mon[34502]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:05.961 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.961 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.961 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:05.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:05.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:05.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:05.962 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:05 smithi100 ceph-mon[35055]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:05.984 DEBUG:teuthology.orchestra.run.smithi100:osd.5> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.5.service 2024-09-13T17:21:05.986 INFO:tasks.cephadm:Deploying osd.6 on smithi044 with /dev/vg_nvme/lv_2... 2024-09-13T17:21:05.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T17:21:06.016 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:06.017 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:05 smithi044 ceph-mon[31175]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:21:06.274 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:21:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:21:06.030+0000 7fb59fd39640 -1 osd.5 0 log_to_monitors true 2024-09-13T17:21:06.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:06 smithi044 ceph-mon[31175]: from='osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T17:21:06.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:06 smithi044 ceph-mon[31175]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T17:21:06.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:06 smithi070 ceph-mon[34502]: from='osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T17:21:06.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:06 smithi070 ceph-mon[34502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T17:21:07.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:06 smithi100 ceph-mon[35055]: from='osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T17:21:07.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:06 smithi100 ceph-mon[35055]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T17:21:07.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:07 smithi044 ceph-mon[31175]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T17:21:07.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:07 smithi044 ceph-mon[31175]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T17:21:07.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:07.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:07 smithi044 ceph-mon[31175]: from='osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:21:07.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:07 smithi044 ceph-mon[31175]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:21:07.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:07 smithi044 ceph-mon[31175]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-mon[35055]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T17:21:08.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-mon[35055]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T17:21:08.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:08.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-mon[35055]: from='osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:21:08.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-mon[35055]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:21:08.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-mon[35055]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:08.025 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:21:07.701+0000 7fb59bcba640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T17:21:08.025 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:21:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:21:07.709+0000 7fb597ae3640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:21:08.104 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T17:21:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T17:21:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:07 smithi070 ceph-mon[34502]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T17:21:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:07 smithi070 ceph-mon[34502]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T17:21:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:07 smithi070 ceph-mon[34502]: from='osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:21:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:07 smithi070 ceph-mon[34502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:21:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:07 smithi070 ceph-mon[34502]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 157 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:08.210 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-09-13T17:21:08.210 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-09-13T17:21:08.210 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126794 s, 827 MB/s 2024-09-13T17:21:08.210 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-09-13T17:21:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:08 smithi100 ceph-mon[35055]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:21:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:08 smithi100 ceph-mon[35055]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T17:21:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:08 smithi044 ceph-mon[31175]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:21:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:08 smithi044 ceph-mon[31175]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T17:21:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:08.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-09-13T17:21:09.142 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:21:09.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:08 smithi070 ceph-mon[34502]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:21:09.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:08 smithi070 ceph-mon[34502]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T17:21:09.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:09.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] boot 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: osdmap e37: 6 total, 6 up, 6 in 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi100 2024-09-13T17:21:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi100 to 8543M 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:09.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:09 smithi044 ceph-mon[31175]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] boot 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: osdmap e37: 6 total, 6 up, 6 in 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi100 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi100 to 8543M 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:10.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:10.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:09 smithi100 ceph-mon[35055]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: osd.5 [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] boot 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: osdmap e37: 6 total, 6 up, 6 in 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:21:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi100 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi100 to 8543M 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:09 smithi070 ceph-mon[34502]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-09-13T17:21:10.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 -- 172.21.15.44:0/1371579273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 msgr2=0x7ff91c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:10.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 --2- 172.21.15.44:0/1371579273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c1013e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009900 tx=0x7ff90c02a410 comp rx=0 tx=0).stop 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 -- 172.21.15.44:0/1371579273 shutdown_connections 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 --2- 172.21.15.44:0/1371579273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91c101920 0x7ff91c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 --2- 172.21.15.44:0/1371579273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c1013e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 --2- 172.21.15.44:0/1371579273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91c0ffe00 0x7ff91c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 -- 172.21.15.44:0/1371579273 >> 172.21.15.44:0/1371579273 conn(0x7ff91c0fb5d0 msgr2=0x7ff91c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 -- 172.21.15.44:0/1371579273 shutdown_connections 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 -- 172.21.15.44:0/1371579273 wait complete. 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 Processor -- start 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.685+0000 7ff923a99640 1 -- start start 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91c0ffe00 0x7ff91c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:10.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91c101920 0x7ff91c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c073570 con 0x7ff91c100f60 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0736e0 con 0x7ff91c0ffe00 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c19fdd0 con 0x7ff91c101920 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55470/0 (socket says 172.21.15.44:55470) 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 -- 172.21.15.44:0/481849112 learned_addr learned my addr 172.21.15.44:0/481849112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff922036640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91c101920 0x7ff91c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921835640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91c0ffe00 0x7ff91c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 -- 172.21.15.44:0/481849112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91c101920 msgr2=0x7ff91c072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91c101920 0x7ff91c072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 -- 172.21.15.44:0/481849112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91c0ffe00 msgr2=0x7ff91c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91c0ffe00 0x7ff91c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:10.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 -- 172.21.15.44:0/481849112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c009560 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff921034640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c0728a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0048f0 tx=0x7ff90c004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0035f0 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c1a0020 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff90c003be0 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.686+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c03cbc0 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.687+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c1a0510 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.688+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c107330 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.689+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff90c003750 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.689+0000 7ff912ffd640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff9080a1080 0x7ff9080a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.689+0000 7ff921835640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff9080a1080 0x7ff9080a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.689+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7ff90c0eccf0 con 0x7ff91c100f60 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.690+0000 7ff921835640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff9080a1080 0x7ff9080a3540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9180058c0 tx=0x7ff91800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:21:10.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.694+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff90c0b7320 con 0x7ff91c100f60 2024-09-13T17:21:10.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:10 smithi044 ceph-mon[31175]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T17:21:10.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:10.879+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff91c0608f0 con 0x7ff9080a1080 2024-09-13T17:21:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:10 smithi100 ceph-mon[35055]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T17:21:11.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:10 smithi070 ceph-mon[34502]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T17:21:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:11 smithi044 ceph-mon[31175]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T17:21:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:11 smithi044 ceph-mon[31175]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:11 smithi044 ceph-mon[31175]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:11 smithi100 ceph-mon[35055]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T17:21:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:11 smithi100 ceph-mon[35055]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:11 smithi100 ceph-mon[35055]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:11 smithi070 ceph-mon[34502]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T17:21:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:11 smithi070 ceph-mon[34502]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:11 smithi070 ceph-mon[34502]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:13.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:12 smithi100 ceph-mon[35055]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T17:21:13.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:12 smithi044 ceph-mon[31175]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T17:21:13.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:12 smithi070 ceph-mon[34502]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2290993309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a8c0b4-0801-40db-88a0-ee66c59f28bf"}]: dispatch 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2290993309' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18a8c0b4-0801-40db-88a0-ee66c59f28bf"}]': finished 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3236917783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2290993309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a8c0b4-0801-40db-88a0-ee66c59f28bf"}]: dispatch 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2290993309' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18a8c0b4-0801-40db-88a0-ee66c59f28bf"}]': finished 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3236917783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:21:14.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T17:21:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2290993309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a8c0b4-0801-40db-88a0-ee66c59f28bf"}]: dispatch 2024-09-13T17:21:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2290993309' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18a8c0b4-0801-40db-88a0-ee66c59f28bf"}]': finished 2024-09-13T17:21:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T17:21:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3236917783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:21:15.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:15 smithi044 ceph-mon[31175]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-13T17:21:15.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:15 smithi070 ceph-mon[34502]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-13T17:21:16.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:15 smithi100 ceph-mon[35055]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-13T17:21:17.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:17 smithi044 ceph-mon[31175]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:21:17.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:17 smithi070 ceph-mon[34502]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:21:18.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:17 smithi100 ceph-mon[35055]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:21:19.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:19 smithi044 ceph-mon[31175]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-13T17:21:19.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:19 smithi070 ceph-mon[34502]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-13T17:21:20.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:19 smithi100 ceph-mon[35055]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-13T17:21:20.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T17:21:20.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:20.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:20 smithi044 ceph-mon[31175]: Deploying daemon osd.6 on smithi044 2024-09-13T17:21:20.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T17:21:20.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:20.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:20 smithi070 ceph-mon[34502]: Deploying daemon osd.6 on smithi044 2024-09-13T17:21:21.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T17:21:21.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:21.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:20 smithi100 ceph-mon[35055]: Deploying daemon osd.6 on smithi044 2024-09-13T17:21:21.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:21 smithi044 ceph-mon[31175]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-13T17:21:21.925 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:21 smithi100 ceph-mon[35055]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-13T17:21:21.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:21 smithi070 ceph-mon[34502]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-13T17:21:23.747 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:23 smithi044 ceph-mon[31175]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T17:21:23.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:23 smithi070 ceph-mon[34502]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T17:21:24.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:23 smithi100 ceph-mon[35055]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T17:21:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:25.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:25.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:25.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:26 smithi100 ceph-mon[35055]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:21:26.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:26 smithi044 ceph-mon[31175]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:21:26.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:26 smithi070 ceph-mon[34502]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:21:27.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:27 smithi044 ceph-mon[31175]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:27.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:27 smithi070 ceph-mon[34502]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:27 smithi100 ceph-mon[35055]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:29.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.041+0000 7ff912ffd640 1 -- 172.21.15.44:0/481849112 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff91c0608f0 con 0x7ff9080a1080 2024-09-13T17:21:29.043 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 6 on host 'smithi044' 2024-09-13T17:21:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff9080a1080 msgr2=0x7ff9080a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff9080a1080 0x7ff9080a3540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9180058c0 tx=0x7ff91800a4c0 comp rx=0 tx=0).stop 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 msgr2=0x7ff91c0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c0728a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0048f0 tx=0x7ff90c004920 comp rx=0 tx=0).stop 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 shutdown_connections 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff9080a1080 0x7ff9080a3540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91c101920 0x7ff91c072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff91c100f60 0x7ff91c0728a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 --2- 172.21.15.44:0/481849112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91c0ffe00 0x7ff91c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 >> 172.21.15.44:0/481849112 conn(0x7ff91c0fb5d0 msgr2=0x7ff91c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 shutdown_connections 2024-09-13T17:21:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:21:29.044+0000 7ff923a99640 1 -- 172.21.15.44:0/481849112 wait complete. 2024-09-13T17:21:29.347 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.347 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:29.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:29.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:29.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:29 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:29 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:29.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:29 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:29.753 DEBUG:teuthology.orchestra.run.smithi044:osd.6> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.6.service 2024-09-13T17:21:29.755 INFO:tasks.cephadm:Deploying osd.7 on smithi070 with /dev/vg_nvme/lv_2... 2024-09-13T17:21:29.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T17:21:29.961 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:21:30.355 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:30 smithi070 ceph-mon[34502]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T17:21:30.355 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:30 smithi070 ceph-mon[34502]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:30.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:30 smithi044 ceph-mon[31175]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T17:21:30.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:30 smithi044 ceph-mon[31175]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:30 smithi100 ceph-mon[35055]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T17:21:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:30 smithi100 ceph-mon[35055]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:31.213 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:31 smithi044 ceph-mon[31175]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T17:21:31.214 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:31 smithi044 ceph-mon[31175]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T17:21:31.214 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:31 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:31.214 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:31 smithi044 ceph-mon[31175]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:21:31.214 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:21:31 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:21:31.102+0000 7f53fb2bc640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T17:21:31.214 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:21:31 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:21:31.111+0000 7f53f70e5640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:21:31.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:31 smithi070 ceph-mon[34502]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T17:21:31.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:31 smithi070 ceph-mon[34502]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T17:21:31.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:31 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:31.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:31 smithi070 ceph-mon[34502]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:21:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:31 smithi100 ceph-mon[35055]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T17:21:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:31 smithi100 ceph-mon[35055]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T17:21:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:31 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:31 smithi100 ceph-mon[35055]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:21:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T17:21:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T17:21:31.917 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-09-13T17:21:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-09-13T17:21:31.917 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00859146 s, 1.2 GB/s 2024-09-13T17:21:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-09-13T17:21:32.323 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:32 smithi044 ceph-mon[31175]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:21:32.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:32 smithi044 ceph-mon[31175]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T17:21:32.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:32 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:32.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:32 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:32.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:32 smithi044 ceph-mon[31175]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:32.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:32 smithi070 ceph-mon[34502]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:21:32.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:32 smithi070 ceph-mon[34502]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T17:21:32.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:32 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:32.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:32 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:32.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:32 smithi070 ceph-mon[34502]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:32.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:32 smithi100 ceph-mon[35055]: from='osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:21:32.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:32 smithi100 ceph-mon[35055]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T17:21:32.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:32 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:32.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:32 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:32.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:32 smithi100 ceph-mon[35055]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-13T17:21:32.639 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2024-09-13T17:21:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:21:33.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:21:33.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:21:33.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:33.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] boot 2024-09-13T17:21:33.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: osdmap e45: 7 total, 7 up, 7 in 2024-09-13T17:21:33.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:33.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] boot 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: osdmap e45: 7 total, 7 up, 7 in 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:33.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:33.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:21:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:21:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: osd.6 [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] boot 2024-09-13T17:21:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: osdmap e45: 7 total, 7 up, 7 in 2024-09-13T17:21:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:33.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:34.217 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:34 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi044 2024-09-13T17:21:34.217 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:34 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi044 to 5697M 2024-09-13T17:21:34.217 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:34 smithi070 ceph-mon[34502]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T17:21:34.217 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:34 smithi070 ceph-mon[34502]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:34.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:34 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi044 2024-09-13T17:21:34.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:34 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi044 to 5697M 2024-09-13T17:21:34.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:34 smithi044 ceph-mon[31175]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T17:21:34.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:34 smithi044 ceph-mon[31175]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:34 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi044 2024-09-13T17:21:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:34 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi044 to 5697M 2024-09-13T17:21:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:34 smithi100 ceph-mon[35055]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T17:21:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:34 smithi100 ceph-mon[35055]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:34.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 -- 172.21.15.70:0/2942451607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 msgr2=0x7f06c8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:34.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 --2- 172.21.15.70:0/2942451607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c8108240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009ca0 tx=0x7f06c002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:21:34.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 -- 172.21.15.70:0/2942451607 shutdown_connections 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 --2- 172.21.15.70:0/2942451607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c8100f60 0x7f06c81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 --2- 172.21.15.70:0/2942451607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c8108240 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 --2- 172.21.15.70:0/2942451607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c80ffe00 0x7f06c8100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 -- 172.21.15.70:0/2942451607 >> 172.21.15.70:0/2942451607 conn(0x7f06c80fb5d0 msgr2=0x7f06c80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 -- 172.21.15.70:0/2942451607 shutdown_connections 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 -- 172.21.15.70:0/2942451607 wait complete. 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.549+0000 7f06ce75c640 1 Processor -- start 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 -- start start 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c80ffe00 0x7f06c8195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c8100f60 0x7f06c81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c819d770 con 0x7f06c8100f60 2024-09-13T17:21:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c819d8e0 con 0x7f06c8101920 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06ce75c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c819daa0 con 0x7f06c80ffe00 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c819d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39008/0 (socket says 172.21.15.70:39008) 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 -- 172.21.15.70:0/1879786318 learned_addr learned my addr 172.21.15.70:0/1879786318 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06bffff640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c8100f60 0x7f06c81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06c7fff640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c80ffe00 0x7f06c8195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 -- 172.21.15.70:0/1879786318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c80ffe00 msgr2=0x7f06c8195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c80ffe00 0x7f06c8195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 -- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c8100f60 msgr2=0x7f06c81961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c8100f60 0x7f06c81961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 -- 172.21.15.70:0/1879786318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009900 con 0x7f06c8101920 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06c7fff640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c80ffe00 0x7f06c8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06cccf9640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c819d230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06c002aeb0 tx=0x7f06c0003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.550+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c00307c0 con 0x7f06c8101920 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.551+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06c0040070 con 0x7f06c8101920 2024-09-13T17:21:34.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.551+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c819dd20 con 0x7f06c8101920 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.551+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c00043c0 con 0x7f06c8101920 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.551+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c819e290 con 0x7f06c8101920 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.552+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c8107280 con 0x7f06c8101920 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.553+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f06c0042030 con 0x7f06c8101920 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.553+0000 7f06c5ffb640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06b40a1100 0x7f06b40a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.553+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f06c00ed2b0 con 0x7f06c8101920 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.553+0000 7f06c7fff640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06b40a1100 0x7f06b40a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.554+0000 7f06c7fff640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06b40a1100 0x7f06b40a35c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005fd0 tx=0x7f06b8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:21:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.557+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f06c00b7750 con 0x7f06c8101920 2024-09-13T17:21:34.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:34.729+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f06c802c450 con 0x7f06b40a1100 2024-09-13T17:21:35.347 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:35 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:35.347 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:35 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:35.347 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:35 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:35.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:35 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:35.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:35 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:35.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:35 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:35.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:35 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:35.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:35 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:35.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:35 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:36.424 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:36 smithi070 ceph-mon[34502]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:36.424 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:36 smithi070 ceph-mon[34502]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:36 smithi100 ceph-mon[35055]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:36 smithi100 ceph-mon[35055]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:36 smithi044 ceph-mon[31175]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:36 smithi044 ceph-mon[31175]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:37.436 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/1881722349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]: dispatch 2024-09-13T17:21:37.436 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:37 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]: dispatch 2024-09-13T17:21:37.436 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:37 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]': finished 2024-09-13T17:21:37.436 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:37 smithi070 ceph-mon[34502]: osdmap e47: 8 total, 7 up, 8 in 2024-09-13T17:21:37.437 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:37 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/1881722349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]: dispatch 2024-09-13T17:21:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:37 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]: dispatch 2024-09-13T17:21:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:37 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]': finished 2024-09-13T17:21:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:37 smithi100 ceph-mon[35055]: osdmap e47: 8 total, 7 up, 8 in 2024-09-13T17:21:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:37 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:37.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/1881722349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]: dispatch 2024-09-13T17:21:37.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:37 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]: dispatch 2024-09-13T17:21:37.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:37 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c62e54-ff2e-4578-8eb7-93a848d283c7"}]': finished 2024-09-13T17:21:37.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:37 smithi044 ceph-mon[31175]: osdmap e47: 8 total, 7 up, 8 in 2024-09-13T17:21:37.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:37 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:38.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:38 smithi070 ceph-mon[34502]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:38.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:38 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/2484816780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:21:38.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:38 smithi100 ceph-mon[35055]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:38.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:38 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/2484816780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:21:38.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:38 smithi044 ceph-mon[31175]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:38.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:38 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/2484816780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:21:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:39 smithi044 ceph-mon[31175]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:39 smithi070 ceph-mon[34502]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:39 smithi100 ceph-mon[35055]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:41.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:41 smithi070 ceph-mon[34502]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:41.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:41 smithi044 ceph-mon[31175]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:41 smithi100 ceph-mon[35055]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:43.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:43 smithi070 ceph-mon[34502]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:43.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:43 smithi044 ceph-mon[31175]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:43 smithi100 ceph-mon[35055]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:44.779 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T17:21:44.779 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:44.779 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:44 smithi070 ceph-mon[34502]: Deploying daemon osd.7 on smithi070 2024-09-13T17:21:44.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T17:21:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:44 smithi044 ceph-mon[31175]: Deploying daemon osd.7 on smithi070 2024-09-13T17:21:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T17:21:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:44 smithi100 ceph-mon[35055]: Deploying daemon osd.7 on smithi070 2024-09-13T17:21:45.800 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:45 smithi070 ceph-mon[34502]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:45.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:45 smithi044 ceph-mon[31175]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:45 smithi100 ceph-mon[35055]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:47.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:47 smithi044 ceph-mon[31175]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:47.904 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:47 smithi070 ceph-mon[34502]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:47 smithi100 ceph-mon[35055]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:49.692 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:49.693 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:49.693 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:50.755 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:50 smithi070 ceph-mon[34502]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:50.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:50 smithi100 ceph-mon[35055]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:50.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:50 smithi044 ceph-mon[31175]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:51.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:51 smithi044 ceph-mon[31175]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:51.910 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:51 smithi070 ceph-mon[34502]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:51.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:51 smithi100 ceph-mon[35055]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:52.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.871+0000 7f06c5ffb640 1 -- 172.21.15.70:0/1879786318 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f06c802c450 con 0x7f06b40a1100 2024-09-13T17:21:52.873 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 7 on host 'smithi070' 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06b40a1100 msgr2=0x7f06b40a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06b40a1100 0x7f06b40a35c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005fd0 tx=0x7f06b8005850 comp rx=0 tx=0).stop 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 msgr2=0x7f06c819d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c819d230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06c002aeb0 tx=0x7f06c0003c20 comp rx=0 tx=0).stop 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 shutdown_connections 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06b40a1100 0x7f06b40a35c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c8101920 0x7f06c819d230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c8100f60 0x7f06c81961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 --2- 172.21.15.70:0/1879786318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c80ffe00 0x7f06c8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 >> 172.21.15.70:0/1879786318 conn(0x7f06c80fb5d0 msgr2=0x7f06c8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:21:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 shutdown_connections 2024-09-13T17:21:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:21:52.873+0000 7f06ce75c640 1 -- 172.21.15.70:0/1879786318 wait complete. 2024-09-13T17:21:53.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:53.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:53.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:53.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:53 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:53 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.513 DEBUG:teuthology.orchestra.run.smithi070:osd.7> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.7.service 2024-09-13T17:21:53.515 INFO:tasks.cephadm:Deploying osd.8 on smithi100 with /dev/vg_nvme/lv_2... 2024-09-13T17:21:53.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:53 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:53.711 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:21:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:54 smithi070 ceph-mon[34502]: from='osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T17:21:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:54 smithi070 ceph-mon[34502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T17:21:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:54 smithi070 ceph-mon[34502]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:54 smithi100 ceph-mon[35055]: from='osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T17:21:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:54 smithi100 ceph-mon[35055]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T17:21:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:54 smithi100 ceph-mon[35055]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:54.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:54 smithi044 ceph-mon[31175]: from='osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T17:21:54.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:54 smithi044 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T17:21:54.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:54 smithi044 ceph-mon[31175]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-mon[34502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T17:21:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-mon[34502]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T17:21:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-mon[34502]: from='osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:21:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-mon[34502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:21:55.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:55 smithi100 ceph-mon[35055]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T17:21:55.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:55 smithi100 ceph-mon[35055]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T17:21:55.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:55.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:55 smithi100 ceph-mon[35055]: from='osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:21:55.461 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:55 smithi100 ceph-mon[35055]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:21:55.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:55 smithi044 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T17:21:55.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:55 smithi044 ceph-mon[31175]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T17:21:55.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:55.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:55 smithi044 ceph-mon[31175]: from='osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:21:55.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:55 smithi044 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:21:55.703 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T17:21:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T17:21:55.800 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-13T17:21:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-13T17:21:55.801 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126845 s, 827 MB/s 2024-09-13T17:21:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-13T17:21:55.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:21:55.586+0000 7f490b68c640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T17:21:55.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:21:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:21:55.598+0000 7f49034ad640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: from='osd.7 ' entity='osd.7' 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] boot 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: osdmap e50: 8 total, 8 up, 8 in 2024-09-13T17:21:56.293 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:21:56.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T17:21:56.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:56.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: from='osd.7 ' entity='osd.7' 2024-09-13T17:21:56.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] boot 2024-09-13T17:21:56.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: osdmap e50: 8 total, 8 up, 8 in 2024-09-13T17:21:56.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-09-13T17:21:56.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: from='osd.7 ' entity='osd.7' 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: osd.7 [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] boot 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: osdmap e50: 8 total, 8 up, 8 in 2024-09-13T17:21:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:21:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:21:57.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:21:57.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi070 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi070 to 5696M 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:57 smithi100 ceph-mon[35055]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi070 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi070 to 5696M 2024-09-13T17:21:57.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:57.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:57.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:57 smithi070 ceph-mon[34502]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi070 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi070 to 5696M 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:57.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:21:57.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:21:57.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:57 smithi044 ceph-mon[31175]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 -- 172.21.15.100:0/1901299473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4100f60 msgr2=0x7fe1d41013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/1901299473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4100f60 0x7fe1d41013e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009900 tx=0x7fe1c402a410 comp rx=0 tx=0).stop 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 -- 172.21.15.100:0/1901299473 shutdown_connections 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/1901299473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4100f60 0x7fe1d41013e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/1901299473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1d4101920 0x7fe1d4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/1901299473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1d40ffe00 0x7fe1d4100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 -- 172.21.15.100:0/1901299473 >> 172.21.15.100:0/1901299473 conn(0x7fe1d40fb5d0 msgr2=0x7fe1d40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.249+0000 7fe1db1ee640 1 -- 172.21.15.100:0/1901299473 shutdown_connections 2024-09-13T17:21:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 -- 172.21.15.100:0/1901299473 wait complete. 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 Processor -- start 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 -- start start 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1d40ffe00 0x7fe1d4074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1d4100f60 0x7fe1d40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 0x7fe1d4072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40734e0 con 0x7fe1d40ffe00 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4073650 con 0x7fe1d4100f60 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1db1ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d419fda0 con 0x7fe1d4101920 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1d978b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 0x7fe1d4072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1d978b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 0x7fe1d4072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53718/0 (socket says 172.21.15.100:53718) 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1d978b640 1 -- 172.21.15.100:0/2951074692 learned_addr learned my addr 172.21.15.100:0/2951074692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:21:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.250+0000 7fe1d8f8a640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1d40ffe00 0x7fe1d4074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1d978b640 1 -- 172.21.15.100:0/2951074692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1d4100f60 msgr2=0x7fe1d40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1cbfff640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1d4100f60 0x7fe1d40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1d978b640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1d4100f60 0x7fe1d40728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1d978b640 1 -- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1d40ffe00 msgr2=0x7fe1d4074200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1d978b640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1d40ffe00 0x7fe1d4074200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1d978b640 1 -- 172.21.15.100:0/2951074692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4009560 con 0x7fe1d4101920 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1d978b640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 0x7fe1d4072de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00eaa0 tx=0x7fe1cc00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc004d20 con 0x7fe1d4101920 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1cc004e80 con 0x7fe1d4101920 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d419ffa0 con 0x7fe1d4101920 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc011470 con 0x7fe1d4101920 2024-09-13T17:21:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.251+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d41a0470 con 0x7fe1d4101920 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.252+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d4107350 con 0x7fe1d4101920 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.253+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe1cc00caf0 con 0x7fe1d4101920 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.254+0000 7fe1c9ffb640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1bc0a1100 0x7fe1bc0a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.254+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fe1cc0cbfd0 con 0x7fe1d4101920 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.254+0000 7fe1d8f8a640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1bc0a1100 0x7fe1bc0a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.255+0000 7fe1d8f8a640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1bc0a1100 0x7fe1bc0a35c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009990 tx=0x7fe1d0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:21:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.258+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe1cc0962a0 con 0x7fe1d4101920 2024-09-13T17:21:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:21:58.429+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1d402c4f0 con 0x7fe1bc0a1100 2024-09-13T17:21:58.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:58 smithi100 ceph-mon[35055]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:21:58.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:58 smithi100 ceph-mon[35055]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T17:21:58.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:58 smithi070 ceph-mon[34502]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:21:58.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:58 smithi070 ceph-mon[34502]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T17:21:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:58 smithi044 ceph-mon[31175]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:21:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:58 smithi044 ceph-mon[31175]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T17:21:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:59 smithi070 ceph-mon[34502]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:59 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:21:59 smithi070 ceph-mon[34502]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T17:21:59.468 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:59 smithi100 ceph-mon[35055]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:59.468 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:59.468 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:59.468 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:59 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:59.468 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:21:59 smithi100 ceph-mon[35055]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T17:21:59.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:59 smithi044 ceph-mon[31175]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:21:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:21:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:21:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:59 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:21:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:21:59 smithi044 ceph-mon[31175]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T17:22:00.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:00 smithi100 ceph-mon[35055]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:00.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:00 smithi100 ceph-mon[35055]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T17:22:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:00 smithi070 ceph-mon[34502]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:00 smithi070 ceph-mon[34502]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T17:22:00.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:00 smithi044 ceph-mon[31175]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:00 smithi044 ceph-mon[31175]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T17:22:01.266 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:01 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/3348251238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]: dispatch 2024-09-13T17:22:01.266 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:01 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]: dispatch 2024-09-13T17:22:01.266 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:01 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]': finished 2024-09-13T17:22:01.266 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:01 smithi100 ceph-mon[35055]: osdmap e55: 9 total, 8 up, 9 in 2024-09-13T17:22:01.266 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:01 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:01.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:01 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/3348251238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]: dispatch 2024-09-13T17:22:01.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:01 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]: dispatch 2024-09-13T17:22:01.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:01 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]': finished 2024-09-13T17:22:01.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:01 smithi070 ceph-mon[34502]: osdmap e55: 9 total, 8 up, 9 in 2024-09-13T17:22:01.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:01 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:01.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:01 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/3348251238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]: dispatch 2024-09-13T17:22:01.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:01 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]: dispatch 2024-09-13T17:22:01.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:01 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07c1084-7ab4-4531-bb14-893e93b0e1b8"}]': finished 2024-09-13T17:22:01.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:01 smithi044 ceph-mon[31175]: osdmap e55: 9 total, 8 up, 9 in 2024-09-13T17:22:01.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:01 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:02.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:02 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/397866923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:02.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:02 smithi070 ceph-mon[34502]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:02 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/397866923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:02 smithi100 ceph-mon[35055]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:02.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/397866923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:02 smithi044 ceph-mon[31175]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:03.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:03 smithi044 ceph-mon[31175]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:03 smithi070 ceph-mon[34502]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:04.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:03 smithi100 ceph-mon[35055]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:05.826 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:05 smithi100 ceph-mon[35055]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:05.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:05 smithi044 ceph-mon[31175]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:05 smithi070 ceph-mon[34502]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:07.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T17:22:07.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:07.744 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T17:22:07.744 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:07.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T17:22:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:08.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:08 smithi100 ceph-mon[35055]: Deploying daemon osd.8 on smithi100 2024-09-13T17:22:08.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:08 smithi100 ceph-mon[35055]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:08.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:08 smithi044 ceph-mon[31175]: Deploying daemon osd.8 on smithi100 2024-09-13T17:22:08.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:08 smithi044 ceph-mon[31175]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:08 smithi070 ceph-mon[34502]: Deploying daemon osd.8 on smithi100 2024-09-13T17:22:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:08 smithi070 ceph-mon[34502]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:09.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:09 smithi044 ceph-mon[31175]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:09 smithi070 ceph-mon[34502]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:09.974 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:09 smithi100 ceph-mon[35055]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:11.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:11 smithi100 ceph-mon[35055]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:11.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:11 smithi044 ceph-mon[31175]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:11.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:11 smithi070 ceph-mon[34502]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:12.845 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:12.845 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:12.845 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:12.846 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:12.846 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:12.846 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:12.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:12.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:12.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:13.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:13 smithi100 ceph-mon[35055]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:13.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:13 smithi044 ceph-mon[31175]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:13 smithi070 ceph-mon[34502]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.850+0000 7fe1c9ffb640 1 -- 172.21.15.100:0/2951074692 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe1d402c4f0 con 0x7fe1bc0a1100 2024-09-13T17:22:15.851 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 8 on host 'smithi100' 2024-09-13T17:22:15.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1bc0a1100 msgr2=0x7fe1bc0a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:15.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1bc0a1100 0x7fe1bc0a35c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009990 tx=0x7fe1d0009110 comp rx=0 tx=0).stop 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 msgr2=0x7fe1d4072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 0x7fe1d4072de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00eaa0 tx=0x7fe1cc00eea0 comp rx=0 tx=0).stop 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 shutdown_connections 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1bc0a1100 0x7fe1bc0a35c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d4101920 0x7fe1d4072de0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1d4100f60 0x7fe1d40728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 --2- 172.21.15.100:0/2951074692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1d40ffe00 0x7fe1d4074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 >> 172.21.15.100:0/2951074692 conn(0x7fe1d40fb5d0 msgr2=0x7fe1d4104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 shutdown_connections 2024-09-13T17:22:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:22:15.852+0000 7fe1db1ee640 1 -- 172.21.15.100:0/2951074692 wait complete. 2024-09-13T17:22:16.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:16 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:16.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:16 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:16.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:16.476 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.476 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:16 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:16.496 DEBUG:teuthology.orchestra.run.smithi100:osd.8> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.8.service 2024-09-13T17:22:16.498 INFO:tasks.cephadm:Deploying osd.9 on smithi044 with /dev/vg_nvme/lv_1... 2024-09-13T17:22:16.499 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T17:22:16.713 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:22:17.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:17 smithi044 ceph-mon[31175]: from='osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T17:22:17.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:17 smithi044 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T17:22:17.381 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:17 smithi100 ceph-mon[35055]: from='osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T17:22:17.381 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:17 smithi100 ceph-mon[35055]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T17:22:17.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:17 smithi070 ceph-mon[34502]: from='osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T17:22:17.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:17 smithi070 ceph-mon[34502]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T17:22:18.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-mon[35055]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T17:22:18.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-mon[35055]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T17:22:18.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:18.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-mon[35055]: from='osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:22:18.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-mon[35055]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:22:18.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-mon[35055]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:18.275 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:22:18.121+0000 7f3dd89da640 -1 osd.8 0 waiting for initial osdmap 2024-09-13T17:22:18.275 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:22:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:22:18.132+0000 7f3dd4803640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:22:18.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:18 smithi044 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T17:22:18.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:18 smithi044 ceph-mon[31175]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T17:22:18.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:18.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:18 smithi044 ceph-mon[31175]: from='osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:22:18.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:18 smithi044 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:22:18.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:18 smithi044 ceph-mon[31175]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:18 smithi070 ceph-mon[34502]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T17:22:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:18 smithi070 ceph-mon[34502]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T17:22:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:18 smithi070 ceph-mon[34502]: from='osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:22:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:18 smithi070 ceph-mon[34502]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:22:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:18 smithi070 ceph-mon[34502]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-13T17:22:18.633 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T17:22:18.739 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T17:22:18.752 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-09-13T17:22:18.752 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-09-13T17:22:18.752 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011489 s, 913 MB/s 2024-09-13T17:22:18.752 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] boot 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: osdmap e58: 9 total, 9 up, 9 in 2024-09-13T17:22:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] boot 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: osdmap e58: 9 total, 9 up, 9 in 2024-09-13T17:22:19.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.541 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:22:19.541 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T17:22:19.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: osd.8 [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] boot 2024-09-13T17:22:19.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: osdmap e58: 9 total, 9 up, 9 in 2024-09-13T17:22:19.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:22:19.563 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-09-13T17:22:19.769 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:22:20.387 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:22:20.387 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:22:20.387 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi100 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi100 to 5695M 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:20.388 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:20 smithi044 ceph-mon[31175]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T17:22:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:22:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:22:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi100 2024-09-13T17:22:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi100 to 5695M 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:20.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:20 smithi070 ceph-mon[34502]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T17:22:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:22:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:22:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi100 2024-09-13T17:22:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi100 to 5695M 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:20 smithi100 ceph-mon[35055]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T17:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 -- 172.21.15.44:0/1004280608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 msgr2=0x7f719c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1004280608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c107130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f719402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 -- 172.21.15.44:0/1004280608 shutdown_connections 2024-09-13T17:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1004280608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c107130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1004280608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f719c0ffee0 0x7f719c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1004280608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f719c0fece0 0x7f719c0ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 -- 172.21.15.44:0/1004280608 >> 172.21.15.44:0/1004280608 conn(0x7f719c0fa490 msgr2=0x7f719c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 -- 172.21.15.44:0/1004280608 shutdown_connections 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.672+0000 7f71a1d12640 1 -- 172.21.15.44:0/1004280608 wait complete. 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 Processor -- start 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 -- start start 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f719c0fece0 0x7f719c109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f719c0ffee0 0x7f719c107bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c108830 con 0x7f719c1008a0 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1089a0 con 0x7f719c0ffee0 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f71a1d12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19fcd0 con 0x7f719c0fece0 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51528/0 (socket says 172.21.15.44:51528) 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 -- 172.21.15.44:0/1870143058 learned_addr learned my addr 172.21.15.44:0/1870143058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719affd640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f719c0ffee0 0x7f719c107bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719b7fe640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f719c0fece0 0x7f719c109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 -- 172.21.15.44:0/1870143058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f719c0fece0 msgr2=0x7f719c109550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f719c0fece0 0x7f719c109550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 -- 172.21.15.44:0/1870143058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f719c0ffee0 msgr2=0x7f719c107bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f719c0ffee0 0x7f719c107bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.673+0000 7f719bfff640 1 -- 172.21.15.44:0/1870143058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194009900 con 0x7f719c1008a0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.674+0000 7f719bfff640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c108130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f719402aeb0 tx=0x7f7194003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.674+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940307c0 con 0x7f719c1008a0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.674+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7194040070 con 0x7f719c1008a0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.674+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c19fed0 con 0x7f719c1008a0 2024-09-13T17:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.674+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940044e0 con 0x7f719c1008a0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.674+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c1a0390 con 0x7f719c1008a0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.675+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c106180 con 0x7f719c1008a0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.676+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7194042030 con 0x7f719c1008a0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.677+0000 7f7198ff9640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71840a1100 0x7f71840a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.677+0000 7f719b7fe640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71840a1100 0x7f71840a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.677+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f71940ed6a0 con 0x7f719c1008a0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.677+0000 7f719b7fe640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71840a1100 0x7f71840a35c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:22:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.681+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f71940b7720 con 0x7f719c1008a0 2024-09-13T17:22:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:21.861+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f719c02c4f0 con 0x7f71840a1100 2024-09-13T17:22:22.441 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:22 smithi044 ceph-mon[31175]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T17:22:22.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:22 smithi044 ceph-mon[31175]: pgmap v190: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:22.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:22:22.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:22:22.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:22.446 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:22 smithi070 ceph-mon[34502]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T17:22:22.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:22 smithi070 ceph-mon[34502]: pgmap v190: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:22.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:22:22.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:22:22.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:22 smithi100 ceph-mon[35055]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T17:22:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:22 smithi100 ceph-mon[35055]: pgmap v190: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:22:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:22:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:23 smithi070 ceph-mon[34502]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:22:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:23 smithi070 ceph-mon[34502]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-13T17:22:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:23 smithi070 ceph-mon[34502]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T17:22:23.455 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:23 smithi044 ceph-mon[31175]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:22:23.455 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:23 smithi044 ceph-mon[31175]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-13T17:22:23.455 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:23 smithi044 ceph-mon[31175]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T17:22:23.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:23 smithi100 ceph-mon[35055]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:22:23.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:23 smithi100 ceph-mon[35055]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-13T17:22:23.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:23 smithi100 ceph-mon[35055]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T17:22:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:24 smithi070 ceph-mon[34502]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T17:22:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:24 smithi070 ceph-mon[34502]: pgmap v193: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:24 smithi100 ceph-mon[35055]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T17:22:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:24 smithi100 ceph-mon[35055]: pgmap v193: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:24 smithi044 ceph-mon[31175]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T17:22:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:24 smithi044 ceph-mon[31175]: pgmap v193: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:25.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:25 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3616581545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b38cbad2-3c65-4acc-bc76-68184079cc48"}]: dispatch 2024-09-13T17:22:25.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:25 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3616581545' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b38cbad2-3c65-4acc-bc76-68184079cc48"}]': finished 2024-09-13T17:22:25.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:25 smithi070 ceph-mon[34502]: osdmap e63: 10 total, 9 up, 10 in 2024-09-13T17:22:25.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:25 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:25.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:25 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3616581545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b38cbad2-3c65-4acc-bc76-68184079cc48"}]: dispatch 2024-09-13T17:22:25.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:25 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3616581545' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b38cbad2-3c65-4acc-bc76-68184079cc48"}]': finished 2024-09-13T17:22:25.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:25 smithi100 ceph-mon[35055]: osdmap e63: 10 total, 9 up, 10 in 2024-09-13T17:22:25.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:25 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:25.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:25 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3616581545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b38cbad2-3c65-4acc-bc76-68184079cc48"}]: dispatch 2024-09-13T17:22:25.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:25 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3616581545' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b38cbad2-3c65-4acc-bc76-68184079cc48"}]': finished 2024-09-13T17:22:25.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:25 smithi044 ceph-mon[31175]: osdmap e63: 10 total, 9 up, 10 in 2024-09-13T17:22:25.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:25 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:26.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:26 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3483348739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:26.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:26 smithi070 ceph-mon[34502]: pgmap v195: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:26 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3483348739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:26 smithi100 ceph-mon[35055]: pgmap v195: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:26 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3483348739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:26.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:26 smithi044 ceph-mon[31175]: pgmap v195: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-09-13T17:22:27.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:27 smithi044 ceph-mon[31175]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:22:27.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:27 smithi044 ceph-mon[31175]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-13T17:22:27.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:27 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:22:27.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:27 smithi070 ceph-mon[34502]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:22:27.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:27 smithi070 ceph-mon[34502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-13T17:22:27.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:27 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:22:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:27 smithi100 ceph-mon[35055]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T17:22:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:27 smithi100 ceph-mon[35055]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-13T17:22:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:27 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:22:29.629 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:29 smithi044 ceph-mon[31175]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-13T17:22:29.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:29 smithi070 ceph-mon[34502]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-13T17:22:30.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:29 smithi100 ceph-mon[35055]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-13T17:22:31.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:31 smithi044 ceph-mon[31175]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-13T17:22:31.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:31 smithi070 ceph-mon[34502]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-13T17:22:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:31 smithi100 ceph-mon[35055]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-13T17:22:32.718 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:32 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T17:22:32.718 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:32 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:32.718 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:32 smithi044 ceph-mon[31175]: Deploying daemon osd.9 on smithi044 2024-09-13T17:22:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:32 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T17:22:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:32 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:32 smithi070 ceph-mon[34502]: Deploying daemon osd.9 on smithi044 2024-09-13T17:22:33.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:32 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T17:22:33.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:32 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:33.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:32 smithi100 ceph-mon[35055]: Deploying daemon osd.9 on smithi044 2024-09-13T17:22:33.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:33 smithi044 ceph-mon[31175]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T17:22:33.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:33 smithi070 ceph-mon[34502]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T17:22:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:33 smithi100 ceph-mon[35055]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T17:22:35.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:35 smithi044 ceph-mon[31175]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T17:22:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:35 smithi070 ceph-mon[34502]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T17:22:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:35 smithi100 ceph-mon[35055]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T17:22:37.749 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:37 smithi044 ceph-mon[31175]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:22:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:37 smithi070 ceph-mon[34502]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:22:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:37 smithi100 ceph-mon[35055]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T17:22:38.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:38.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:38.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:38 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:38.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:38.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:38.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:38 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:39.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:39.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:39.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:38 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:39 smithi044 ceph-mon[31175]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:39 smithi070 ceph-mon[34502]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:39 smithi100 ceph-mon[35055]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:41.523 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 9 on host 'smithi044' 2024-09-13T17:22:41.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.522+0000 7f7198ff9640 1 -- 172.21.15.44:0/1870143058 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f719c02c4f0 con 0x7f71840a1100 2024-09-13T17:22:41.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71840a1100 msgr2=0x7f71840a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:41.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71840a1100 0x7f71840a35c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c00b390 comp rx=0 tx=0).stop 2024-09-13T17:22:41.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 msgr2=0x7f719c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:41.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c108130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f719402aeb0 tx=0x7f7194003c20 comp rx=0 tx=0).stop 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 shutdown_connections 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71840a1100 0x7f71840a35c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719c1008a0 0x7f719c108130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f719c0ffee0 0x7f719c107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 --2- 172.21.15.44:0/1870143058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f719c0fece0 0x7f719c109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 >> 172.21.15.44:0/1870143058 conn(0x7f719c0fa490 msgr2=0x7f719c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 shutdown_connections 2024-09-13T17:22:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:22:41.524+0000 7f71a1d12640 1 -- 172.21.15.44:0/1870143058 wait complete. 2024-09-13T17:22:42.200 DEBUG:teuthology.orchestra.run.smithi044:osd.9> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.9.service 2024-09-13T17:22:42.202 INFO:tasks.cephadm:Deploying osd.10 on smithi070 with /dev/vg_nvme/lv_1... 2024-09-13T17:22:42.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T17:22:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:22:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:42 smithi100 ceph-mon[35055]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:42 smithi100 ceph-mon[35055]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T17:22:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:42 smithi044 ceph-mon[31175]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:42 smithi044 ceph-mon[31175]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T17:22:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.896 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.896 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.896 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:42 smithi070 ceph-mon[34502]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:42.896 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:42 smithi070 ceph-mon[34502]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T17:22:42.897 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:42.897 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:43.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:43 smithi044 ceph-mon[31175]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T17:22:43.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:43 smithi044 ceph-mon[31175]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T17:22:43.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:43.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:43 smithi044 ceph-mon[31175]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:22:43.847 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:22:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:22:43.533+0000 7fb919fd1640 -1 osd.9 0 waiting for initial osdmap 2024-09-13T17:22:43.848 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:22:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:22:43.547+0000 7fb915dfa640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:22:43.852 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:43 smithi070 ceph-mon[34502]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T17:22:43.852 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:43 smithi070 ceph-mon[34502]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T17:22:43.852 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:43.852 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:43 smithi070 ceph-mon[34502]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:22:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:43 smithi100 ceph-mon[35055]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T17:22:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:43 smithi100 ceph-mon[35055]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T17:22:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:43 smithi100 ceph-mon[35055]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-09-13T17:22:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T17:22:44.645 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T17:22:44.655 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-09-13T17:22:44.655 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-09-13T17:22:44.656 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00850993 s, 1.2 GB/s 2024-09-13T17:22:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-09-13T17:22:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:44 smithi044 ceph-mon[31175]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:22:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:44 smithi044 ceph-mon[31175]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T17:22:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:44 smithi044 ceph-mon[31175]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:44.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:44 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:44 smithi070 ceph-mon[34502]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:22:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:44 smithi070 ceph-mon[34502]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T17:22:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:44 smithi070 ceph-mon[34502]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:44 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:44 smithi100 ceph-mon[35055]: from='osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-09-13T17:22:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:44 smithi100 ceph-mon[35055]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T17:22:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:44 smithi100 ceph-mon[35055]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:44 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:45.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_1 2024-09-13T17:22:45.508 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:22:45.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:45 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:22:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:45 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:22:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:45 smithi044 ceph-mon[31175]: osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] boot 2024-09-13T17:22:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:45 smithi044 ceph-mon[31175]: osdmap e66: 10 total, 10 up, 10 in 2024-09-13T17:22:45.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:45 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:45.629 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:45 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:22:45.629 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:45 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:22:45.629 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:45.629 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:45 smithi070 ceph-mon[34502]: osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] boot 2024-09-13T17:22:45.629 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:45 smithi070 ceph-mon[34502]: osdmap e66: 10 total, 10 up, 10 in 2024-09-13T17:22:45.629 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:45 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:45 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:22:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:45 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:22:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:45 smithi100 ceph-mon[35055]: osd.9 [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] boot 2024-09-13T17:22:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:45 smithi100 ceph-mon[35055]: osdmap e66: 10 total, 10 up, 10 in 2024-09-13T17:22:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:45 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi044 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi044 to 4273M 2024-09-13T17:22:46.677 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:46.678 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi044 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi044 to 4273M 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:47.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-13T17:22:47.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T17:22:47.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi044 2024-09-13T17:22:47.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi044 to 4273M 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:22:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:22:47.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- 172.21.15.70:0/239468919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 msgr2=0x7fe534107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:47.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- 172.21.15.70:0/239468919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe534107130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009ca0 tx=0x7fe52c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- 172.21.15.70:0/239468919 shutdown_connections 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- 172.21.15.70:0/239468919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe534107130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- 172.21.15.70:0/239468919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340ffee0 0x7fe534100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- 172.21.15.70:0/239468919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5340fece0 0x7fe5340ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- 172.21.15.70:0/239468919 >> 172.21.15.70:0/239468919 conn(0x7fe5340fa490 msgr2=0x7fe5340fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- 172.21.15.70:0/239468919 shutdown_connections 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- 172.21.15.70:0/239468919 wait complete. 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 Processor -- start 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- start start 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340fece0 0x7fe534195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5340ffee0 0x7fe5341960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe53419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419d690 con 0x7fe5340fece0 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.210+0000 7fe53beb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419d800 con 0x7fe5341008a0 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53beb4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419d9c0 con 0x7fe5340ffee0 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe53419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe53419d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60016/0 (socket says 172.21.15.70:60016) 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 -- 172.21.15.70:0/2401352309 learned_addr learned my addr 172.21.15.70:0/2401352309 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53944f640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5340ffee0 0x7fe5341960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe539c50640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340fece0 0x7fe534195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 -- 172.21.15.70:0/2401352309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5340ffee0 msgr2=0x7fe5341960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5340ffee0 0x7fe5341960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 -- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340fece0 msgr2=0x7fe534195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340fece0 0x7fe534195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:22:47.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 -- 172.21.15.70:0/2401352309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009900 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe539c50640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340fece0 0x7fe534195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53a451640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe53419d150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe52c003ba0 tx=0x7fe52c003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0317c0 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52c040070 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53419dc40 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c004720 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.211+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53419e140 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.212+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5341060d0 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.213+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe52c042030 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.214+0000 7fe526ffd640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe5200a10b0 0x7fe5200a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.214+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fe52c0edb20 con 0x7fe5341008a0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.214+0000 7fe539c50640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe5200a10b0 0x7fe5200a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:22:47.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.215+0000 7fe539c50640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe5200a10b0 0x7fe5200a3570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5340ffd40 tx=0x7fe528005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:22:47.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.218+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe52c0b7ab0 con 0x7fe5341008a0 2024-09-13T17:22:47.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:22:47.390+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe53402c4f0 con 0x7fe5200a10b0 2024-09-13T17:22:47.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:47 smithi044 ceph-mon[31175]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:22:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:22:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:22:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:47 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:47 smithi044 ceph-mon[31175]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:47.909 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:47 smithi070 ceph-mon[34502]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:22:47.909 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:22:47.909 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:22:47.909 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:47 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:47.910 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:47 smithi070 ceph-mon[34502]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:47 smithi100 ceph-mon[35055]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:22:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:22:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:22:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:47 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:47 smithi100 ceph-mon[35055]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:49 smithi044 ceph-mon[31175]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:49 smithi070 ceph-mon[34502]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:49 smithi100 ceph-mon[35055]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:50.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/3905109879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]: dispatch 2024-09-13T17:22:50.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:50 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]: dispatch 2024-09-13T17:22:50.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:50 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]': finished 2024-09-13T17:22:50.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:50 smithi044 ceph-mon[31175]: osdmap e68: 11 total, 10 up, 11 in 2024-09-13T17:22:50.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:22:50.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.70:0/3028094403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/3905109879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]: dispatch 2024-09-13T17:22:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:50 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]: dispatch 2024-09-13T17:22:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:50 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]': finished 2024-09-13T17:22:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:50 smithi070 ceph-mon[34502]: osdmap e68: 11 total, 10 up, 11 in 2024-09-13T17:22:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:22:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.70:0/3028094403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/3905109879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]: dispatch 2024-09-13T17:22:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:50 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]: dispatch 2024-09-13T17:22:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:50 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27a2977e-4fde-4a5f-9be6-9811159c13d4"}]': finished 2024-09-13T17:22:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:50 smithi100 ceph-mon[35055]: osdmap e68: 11 total, 10 up, 11 in 2024-09-13T17:22:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:22:51.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.70:0/3028094403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:22:51.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:51 smithi044 ceph-mon[31175]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:51 smithi070 ceph-mon[34502]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:51 smithi100 ceph-mon[35055]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:53 smithi044 ceph-mon[31175]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:53 smithi070 ceph-mon[34502]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:53 smithi100 ceph-mon[35055]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:55.833 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:55 smithi070 ceph-mon[34502]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:55 smithi044 ceph-mon[31175]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:55 smithi100 ceph-mon[35055]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T17:22:57.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:56 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T17:22:57.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:56 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:57.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T17:22:57.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:56 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:22:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:57 smithi070 ceph-mon[34502]: Deploying daemon osd.10 on smithi070 2024-09-13T17:22:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:57 smithi070 ceph-mon[34502]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:57 smithi100 ceph-mon[35055]: Deploying daemon osd.10 on smithi070 2024-09-13T17:22:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:57 smithi100 ceph-mon[35055]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:57 smithi044 ceph-mon[31175]: Deploying daemon osd.10 on smithi070 2024-09-13T17:22:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:57 smithi044 ceph-mon[31175]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:59.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:22:59 smithi044 ceph-mon[31175]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:22:59.941 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:22:59 smithi070 ceph-mon[34502]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:22:59 smithi100 ceph-mon[35055]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:01.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:01 smithi044 ceph-mon[31175]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:01.938 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:01 smithi070 ceph-mon[34502]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:02.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:01 smithi100 ceph-mon[35055]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:02.766 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:02.766 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:02.767 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:02.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:02.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:02.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:03.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:03.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:03.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:03.719 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:03 smithi070 ceph-mon[34502]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:03.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:03 smithi044 ceph-mon[31175]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:04.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:03 smithi100 ceph-mon[35055]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:05.951 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 10 on host 'smithi070' 2024-09-13T17:23:05.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.950+0000 7fe526ffd640 1 -- 172.21.15.70:0/2401352309 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe53402c4f0 con 0x7fe5200a10b0 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe5200a10b0 msgr2=0x7fe5200a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe5200a10b0 0x7fe5200a3570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5340ffd40 tx=0x7fe528005d30 comp rx=0 tx=0).stop 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 msgr2=0x7fe53419d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe53419d150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe52c003ba0 tx=0x7fe52c003bd0 comp rx=0 tx=0).stop 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 shutdown_connections 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe5200a10b0 0x7fe5200a3570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5340ffee0 0x7fe5341960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5341008a0 0x7fe53419d150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 --2- 172.21.15.70:0/2401352309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5340fece0 0x7fe534195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 >> 172.21.15.70:0/2401352309 conn(0x7fe5340fa490 msgr2=0x7fe5341032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.952+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 shutdown_connections 2024-09-13T17:23:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T17:23:05.953+0000 7fe53beb4640 1 -- 172.21.15.70:0/2401352309 wait complete. 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T17:23:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:06 smithi070 ceph-mon[34502]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T17:23:06.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:06 smithi044 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:06.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T17:23:06.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:06 smithi100 ceph-mon[35055]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T17:23:06.751 DEBUG:teuthology.orchestra.run.smithi070:osd.10> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.10.service 2024-09-13T17:23:06.753 INFO:tasks.cephadm:Deploying osd.11 on smithi100 with /dev/vg_nvme/lv_1... 2024-09-13T17:23:06.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T17:23:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:23:08.213 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:07 smithi100 ceph-mon[35055]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T17:23:08.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:07 smithi100 ceph-mon[35055]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T17:23:08.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:08.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:07 smithi100 ceph-mon[35055]: from='osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:23:08.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:07 smithi100 ceph-mon[35055]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:23:08.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:07 smithi100 ceph-mon[35055]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:08.289 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:07 smithi070 ceph-mon[34502]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T17:23:08.289 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:07 smithi070 ceph-mon[34502]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T17:23:08.289 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:08.289 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:07 smithi070 ceph-mon[34502]: from='osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:23:08.289 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:07 smithi070 ceph-mon[34502]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:23:08.289 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:07 smithi070 ceph-mon[34502]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:08.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:07 smithi044 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T17:23:08.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:07 smithi044 ceph-mon[31175]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T17:23:08.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:08.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:07 smithi044 ceph-mon[31175]: from='osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:23:08.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:07 smithi044 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-13T17:23:08.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:07 smithi044 ceph-mon[31175]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:08.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:23:08.413+0000 7f169ada2640 -1 osd.10 0 waiting for initial osdmap 2024-09-13T17:23:08.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:23:08.428+0000 7f1692bc3640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:23:08.712 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T17:23:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T17:23:08.816 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-13T17:23:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-13T17:23:08.816 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115281 s, 910 MB/s 2024-09-13T17:23:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-13T17:23:09.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:08 smithi100 ceph-mon[35055]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:23:09.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:08 smithi100 ceph-mon[35055]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T17:23:09.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:09.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:08 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:09.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:08 smithi100 ceph-mon[35055]: from='osd.10 ' entity='osd.10' 2024-09-13T17:23:09.329 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-mon[34502]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:23:09.329 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-mon[34502]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T17:23:09.329 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:09.329 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:09.329 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:08 smithi070 ceph-mon[34502]: from='osd.10 ' entity='osd.10' 2024-09-13T17:23:09.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:08 smithi044 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-13T17:23:09.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:08 smithi044 ceph-mon[31175]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T17:23:09.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:09.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:08 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:09.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:08 smithi044 ceph-mon[31175]: from='osd.10 ' entity='osd.10' 2024-09-13T17:23:09.477 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-09-13T17:23:09.679 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:23:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:23:10.081 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:23:10.081 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:10.081 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] boot 2024-09-13T17:23:10.081 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: osdmap e71: 11 total, 11 up, 11 in 2024-09-13T17:23:10.081 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:10.081 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:10.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] boot 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: osdmap e71: 11 total, 11 up, 11 in 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:10.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:10.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:23:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:23:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: osd.10 [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] boot 2024-09-13T17:23:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: osdmap e71: 11 total, 11 up, 11 in 2024-09-13T17:23:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:10.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:11.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:10 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi070 2024-09-13T17:23:11.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:10 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi070 to 4272M 2024-09-13T17:23:11.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:10 smithi100 ceph-mon[35055]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T17:23:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 -- 172.21.15.100:0/2024654168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c480fff10 msgr2=0x7f0c48100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/2024654168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c480fff10 0x7f0c48100390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009900 tx=0x7f0c4402a410 comp rx=0 tx=0).stop 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 -- 172.21.15.100:0/2024654168 shutdown_connections 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/2024654168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c480fff10 0x7f0c48100390 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/2024654168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c481008d0 0x7f0c48107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/2024654168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fed10 0x7f0c480ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 -- 172.21.15.100:0/2024654168 >> 172.21.15.100:0/2024654168 conn(0x7f0c480fa4c0 msgr2=0x7f0c480fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 -- 172.21.15.100:0/2024654168 shutdown_connections 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.255+0000 7f0c4ffda640 1 -- 172.21.15.100:0/2024654168 wait complete. 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 Processor -- start 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 -- start start 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c480fed10 0x7f0c48069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fff10 0x7f0c4819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 0x7f0c4819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4819d8f0 con 0x7f0c480fed10 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4819da60 con 0x7f0c480fff10 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4ffda640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4819dc20 con 0x7f0c481008d0 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 0x7f0c4819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 0x7f0c4819d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56532/0 (socket says 172.21.15.100:56532) 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 -- 172.21.15.100:0/3459761727 learned_addr learned my addr 172.21.15.100:0/3459761727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4dd76640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c480fed10 0x7f0c48069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4d575640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fff10 0x7f0c4819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 -- 172.21.15.100:0/3459761727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fff10 msgr2=0x7f0c4819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fff10 0x7f0c4819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 -- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c480fed10 msgr2=0x7f0c48069280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c480fed10 0x7f0c48069280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4e577640 1 -- 172.21.15.100:0/3459761727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009560 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.256+0000 7f0c4d575640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fff10 0x7f0c4819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.257+0000 7f0c4e577640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 0x7f0c4819d110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b970 tx=0x7f0c4000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.257+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000caa0 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.257+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c4000cc00 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.257+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c400075d0 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.257+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4819dea0 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.257+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4819e340 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.258+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c48106250 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.260+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0c40014440 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.260+0000 7f0c3affd640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c340a1100 0x7f0c340a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.260+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f0c400cc2a0 con 0x7f0c481008d0 2024-09-13T17:23:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.261+0000 7f0c4dd76640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c340a1100 0x7f0c340a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.261+0000 7f0c4dd76640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c340a1100 0x7f0c340a35c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0c480ffd70 tx=0x7f0c3c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.263+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0c400960f0 con 0x7f0c481008d0 2024-09-13T17:23:11.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:10 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi070 2024-09-13T17:23:11.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:10 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi070 to 4272M 2024-09-13T17:23:11.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:10 smithi044 ceph-mon[31175]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T17:23:11.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:11.436+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c4802c450 con 0x7f0c340a1100 2024-09-13T17:23:11.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:10 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi070 2024-09-13T17:23:11.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:10 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi070 to 4272M 2024-09-13T17:23:11.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:10 smithi070 ceph-mon[34502]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T17:23:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:11 smithi100 ceph-mon[35055]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T17:23:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:11 smithi100 ceph-mon[35055]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:23:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:23:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:23:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:11 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:11 smithi100 ceph-mon[35055]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T17:23:12.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:11 smithi044 ceph-mon[31175]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T17:23:12.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:11 smithi044 ceph-mon[31175]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:23:12.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:23:12.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:23:12.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:11 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:12.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:11 smithi044 ceph-mon[31175]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T17:23:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:11 smithi070 ceph-mon[34502]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T17:23:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:11 smithi070 ceph-mon[34502]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:23:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T17:23:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T17:23:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:11 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:11 smithi070 ceph-mon[34502]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T17:23:13.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:12 smithi100 ceph-mon[35055]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T17:23:13.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:12 smithi044 ceph-mon[31175]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T17:23:13.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:12 smithi070 ceph-mon[34502]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T17:23:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T17:23:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T17:23:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/3966950356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]: dispatch 2024-09-13T17:23:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]: dispatch 2024-09-13T17:23:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]': finished 2024-09-13T17:23:14.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: osdmap e76: 12 total, 11 up, 12 in 2024-09-13T17:23:14.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/3966950356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]: dispatch 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]: dispatch 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]': finished 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: osdmap e76: 12 total, 11 up, 12 in 2024-09-13T17:23:14.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:13 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/3966950356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]: dispatch 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]: dispatch 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fdeaf9a-fb39-48a9-a42c-76de2ae09f34"}]': finished 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: osdmap e76: 12 total, 11 up, 12 in 2024-09-13T17:23:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:13 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:15.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:14 smithi100 ceph-mon[35055]: from='client.? 172.21.15.100:0/2116639896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:23:15.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:14 smithi044 ceph-mon[31175]: from='client.? 172.21.15.100:0/2116639896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:23:15.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:14 smithi070 ceph-mon[34502]: from='client.? 172.21.15.100:0/2116639896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T17:23:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:16 smithi100 ceph-mon[35055]: pgmap v233: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-13T17:23:16.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:16 smithi044 ceph-mon[31175]: pgmap v233: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-13T17:23:16.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:16 smithi070 ceph-mon[34502]: pgmap v233: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-13T17:23:17.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:17 smithi044 ceph-mon[31175]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:17.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:17 smithi070 ceph-mon[34502]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:18.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:17 smithi100 ceph-mon[35055]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:19.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:19 smithi044 ceph-mon[31175]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:19.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:19 smithi100 ceph-mon[35055]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:19.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:19 smithi070 ceph-mon[34502]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:21.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T17:23:21.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:21.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T17:23:21.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:21.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T17:23:21.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:22 smithi044 ceph-mon[31175]: Deploying daemon osd.11 on smithi100 2024-09-13T17:23:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:22 smithi044 ceph-mon[31175]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:22 smithi070 ceph-mon[34502]: Deploying daemon osd.11 on smithi100 2024-09-13T17:23:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:22 smithi070 ceph-mon[34502]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:22.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:22 smithi100 ceph-mon[35055]: Deploying daemon osd.11 on smithi100 2024-09-13T17:23:22.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:22 smithi100 ceph-mon[35055]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:23.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:23 smithi044 ceph-mon[31175]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:23.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:23 smithi070 ceph-mon[34502]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:24.014 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:23 smithi100 ceph-mon[35055]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:25.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:25 smithi044 ceph-mon[31175]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:25.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:25 smithi100 ceph-mon[35055]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:25 smithi070 ceph-mon[34502]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:26 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:26 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:26 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:26.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:26 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:26.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:26 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:26.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:26 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:26.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:26 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:26.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:26 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:26.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:26 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:27.912 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:27 smithi100 ceph-mon[35055]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:27.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:27 smithi070 ceph-mon[34502]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:28.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:27 smithi044 ceph-mon[31175]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:29.901 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 11 on host 'smithi100' 2024-09-13T17:23:29.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.899+0000 7f0c3affd640 1 -- 172.21.15.100:0/3459761727 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f0c4802c450 con 0x7f0c340a1100 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c340a1100 msgr2=0x7f0c340a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c340a1100 0x7f0c340a35c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0c480ffd70 tx=0x7f0c3c009110 comp rx=0 tx=0).stop 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 msgr2=0x7f0c4819d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 0x7f0c4819d110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b970 tx=0x7f0c4000bd70 comp rx=0 tx=0).stop 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 shutdown_connections 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c340a1100 0x7f0c340a35c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c481008d0 0x7f0c4819d110 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:29.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c480fff10 0x7f0c4819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:29.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 --2- 172.21.15.100:0/3459761727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c480fed10 0x7f0c48069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:29.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 >> 172.21.15.100:0/3459761727 conn(0x7f0c480fa4c0 msgr2=0x7f0c48103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:29.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 shutdown_connections 2024-09-13T17:23:29.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:29.902+0000 7f0c4ffda640 1 -- 172.21.15.100:0/3459761727 wait complete. 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:30.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:30 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:30 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:30 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:30.627 DEBUG:teuthology.orchestra.run.smithi100:osd.11> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.11.service 2024-09-13T17:23:30.630 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-13T17:23:30.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T17:23:30.844 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:23:31.243 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:31 smithi044 ceph-mon[31175]: from='osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T17:23:31.243 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:31 smithi044 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T17:23:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:31 smithi100 ceph-mon[35055]: from='osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T17:23:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:31 smithi100 ceph-mon[35055]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T17:23:31.616 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:31 smithi070 ceph-mon[34502]: from='osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T17:23:31.616 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:31 smithi070 ceph-mon[34502]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T17:23:32.261 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:32 smithi044 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T17:23:32.261 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:32 smithi044 ceph-mon[31175]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T17:23:32.261 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:32 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:32.261 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:32 smithi044 ceph-mon[31175]: from='osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:23:32.261 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:32 smithi044 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:23:32.261 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:32 smithi044 ceph-mon[31175]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:32.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-mon[35055]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T17:23:32.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-mon[35055]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T17:23:32.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:32.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-mon[35055]: from='osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:23:32.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-mon[35055]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:23:32.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-mon[35055]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:32.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:23:32.217+0000 7fd6758bd640 -1 osd.11 0 waiting for initial osdmap 2024-09-13T17:23:32.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:23:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:23:32.231+0000 7fd6716e6640 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:23:32.575 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:32 smithi070 ceph-mon[34502]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T17:23:32.575 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:32 smithi070 ceph-mon[34502]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T17:23:32.575 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:32 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:32.575 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:32 smithi070 ceph-mon[34502]: from='osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:23:32.575 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:32 smithi070 ceph-mon[34502]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-13T17:23:32.576 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:32 smithi070 ceph-mon[34502]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:32.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 -- 172.21.15.44:0/671904855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 msgr2=0x7f7bfc0ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 --2- 172.21.15.44:0/671904855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc0ffb00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009900 tx=0x7f7be402a3f0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 -- 172.21.15.44:0/671904855 shutdown_connections 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 --2- 172.21.15.44:0/671904855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc100040 0x7f7bfc1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 --2- 172.21.15.44:0/671904855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc0ffb00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 --2- 172.21.15.44:0/671904855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc0fe480 0x7f7bfc0fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 -- 172.21.15.44:0/671904855 >> 172.21.15.44:0/671904855 conn(0x7f7bfc0f9c10 msgr2=0x7f7bfc0fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 -- 172.21.15.44:0/671904855 shutdown_connections 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.771+0000 7f7c01218640 1 -- 172.21.15.44:0/671904855 wait complete. 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 Processor -- start 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 -- start start 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc0fe480 0x7f7bfc195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:32.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc100040 0x7f7bfc19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc19d690 con 0x7f7bfc0ff680 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc19d800 con 0x7f7bfc100040 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7c01218640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc19d9c0 con 0x7f7bfc0fe480 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7bfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7bfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc1960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55176/0 (socket says 172.21.15.44:55176) 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7bfa59c640 1 -- 172.21.15.44:0/3535260989 learned_addr learned my addr 172.21.15.44:0/3535260989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7bfad9d640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc0fe480 0x7f7bfc195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.772+0000 7f7bfb59e640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc100040 0x7f7bfc19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bfa59c640 1 -- 172.21.15.44:0/3535260989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc0fe480 msgr2=0x7f7bfc195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bfa59c640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc0fe480 0x7f7bfc195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bfa59c640 1 -- 172.21.15.44:0/3535260989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc100040 msgr2=0x7f7bfc19d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:32.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bfa59c640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc100040 0x7f7bfc19d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bfa59c640 1 -- 172.21.15.44:0/3535260989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be4009560 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bfa59c640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc1960d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009230 tx=0x7f7be4004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4031d70 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7be4004b50 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc19dc40 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4003bf0 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.773+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc19e0e0 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.774+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc1061d0 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.775+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7be4003d50 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.776+0000 7f7bf3fff640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7be80a10b0 0x7f7be80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.776+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f7be40ed810 con 0x7f7bfc0ff680 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.776+0000 7f7bfad9d640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7be80a10b0 0x7f7be80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.777+0000 7f7bfad9d640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7be80a10b0 0x7f7be80a3570 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0ff4e0 tx=0x7f7bec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.779+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7be40b75f0 con 0x7f7bfc0ff680 2024-09-13T17:23:32.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.952+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7bfc109ea0 con 0x7f7bfc0ff680 2024-09-13T17:23:32.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.953+0000 7f7bf3fff640 1 -- 172.21.15.44:0/3535260989 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7be40bac10 con 0x7f7bfc0ff680 2024-09-13T17:23:32.954 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.954+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7be80a10b0 msgr2=0x7f7be80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.954+0000 7f7c01218640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7be80a10b0 0x7f7be80a3570 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0ff4e0 tx=0x7f7bec005f90 comp rx=0 tx=0).stop 2024-09-13T17:23:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.954+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 msgr2=0x7f7bfc1960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.954+0000 7f7c01218640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc1960d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009230 tx=0x7f7be4004380 comp rx=0 tx=0).stop 2024-09-13T17:23:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 shutdown_connections 2024-09-13T17:23:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7be80a10b0 0x7f7be80a3570 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc100040 0x7f7bfc19d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0ff680 0x7f7bfc1960d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 --2- 172.21.15.44:0/3535260989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc0fe480 0x7f7bfc195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 >> 172.21.15.44:0/3535260989 conn(0x7f7bfc0f9c10 msgr2=0x7f7bfc10a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 shutdown_connections 2024-09-13T17:23:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:32.955+0000 7f7c01218640 1 -- 172.21.15.44:0/3535260989 wait complete. 2024-09-13T17:23:33.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:33 smithi100 ceph-mon[35055]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:23:33.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:33 smithi100 ceph-mon[35055]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T17:23:33.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:33.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:33 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:33.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:33 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3535260989' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T17:23:33.479 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:33 smithi044 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:23:33.479 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:33 smithi044 ceph-mon[31175]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T17:23:33.479 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:33.479 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:33 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:33.480 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:33 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3535260989' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T17:23:33.614 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1726248188,"num_in_osds":12,"osd_in_since":1726248193,"num_remapped_pgs":0} 2024-09-13T17:23:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:33 smithi070 ceph-mon[34502]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-13T17:23:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:33 smithi070 ceph-mon[34502]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T17:23:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:33 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:33 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3535260989' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: purged_snaps scrub starts 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: purged_snaps scrub ok 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] boot 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: osdmap e79: 12 total, 12 up, 12 in 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: Detected new or changed devices on smithi100 2024-09-13T17:23:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi100 to 4271M 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:34 smithi100 ceph-mon[35055]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:34.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: purged_snaps scrub starts 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: purged_snaps scrub ok 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] boot 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: osdmap e79: 12 total, 12 up, 12 in 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: Detected new or changed devices on smithi100 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi100 to 4271M 2024-09-13T17:23:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:34.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:34.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:34 smithi044 ceph-mon[31175]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:34.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: purged_snaps scrub starts 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: purged_snaps scrub ok 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: osd.11 [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] boot 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: osdmap e79: 12 total, 12 up, 12 in 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: Detected new or changed devices on smithi100 2024-09-13T17:23:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi100 to 4271M 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:34.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:34.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:34.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:34 smithi070 ceph-mon[34502]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-13T17:23:34.819 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:23:35.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:35 smithi044 ceph-mon[31175]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T17:23:35.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:35 smithi070 ceph-mon[34502]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T17:23:35.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:35 smithi100 ceph-mon[35055]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T17:23:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 -- 172.21.15.44:0/2887270954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 msgr2=0x7fa998106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/2887270954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa998106f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa990009ca0 tx=0x7fa99002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 -- 172.21.15.44:0/2887270954 shutdown_connections 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/2887270954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9980ffd20 0x7fa9981001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/2887270954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa998106f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/2887270954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 0x7fa9980fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 -- 172.21.15.44:0/2887270954 >> 172.21.15.44:0/2887270954 conn(0x7fa9980fa2d0 msgr2=0x7fa9980fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 -- 172.21.15.44:0/2887270954 shutdown_connections 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.526+0000 7fa99d5a1640 1 -- 172.21.15.44:0/2887270954 wait complete. 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 Processor -- start 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 -- start start 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 0x7fa998195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9980ffd20 0x7fa998195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa99819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99819d580 con 0x7fa9981006e0 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99819d6f0 con 0x7fa9980feb20 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99d5a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99819d8b0 con 0x7fa9980ffd20 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99cda0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa99819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99cda0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa99819d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55194/0 (socket says 172.21.15.44:55194) 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa99cda0640 1 -- 172.21.15.44:0/499277035 learned_addr learned my addr 172.21.15.44:0/499277035 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa98ffff640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9980ffd20 0x7fa998195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.527+0000 7fa997fff640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 0x7fa998195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99cda0640 1 -- 172.21.15.44:0/499277035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9980ffd20 msgr2=0x7fa998195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99cda0640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9980ffd20 0x7fa998195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99cda0640 1 -- 172.21.15.44:0/499277035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 msgr2=0x7fa998195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99cda0640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 0x7fa998195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99cda0640 1 -- 172.21.15.44:0/499277035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa990009900 con 0x7fa9981006e0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa997fff640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 0x7fa998195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99cda0640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa99819d040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa99002aeb0 tx=0x7fa990003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900307c0 con 0x7fa9981006e0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa990040070 con 0x7fa9981006e0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99819db30 con 0x7fa9981006e0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900043c0 con 0x7fa9981006e0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.528+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99819e080 con 0x7fa9981006e0 2024-09-13T17:23:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.529+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99810a290 con 0x7fa9981006e0 2024-09-13T17:23:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.531+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa990042030 con 0x7fa9981006e0 2024-09-13T17:23:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.532+0000 7fa995ffb640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa9840a1120 0x7fa9840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.532+0000 7fa997fff640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa9840a1120 0x7fa9840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.532+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9900edb60 con 0x7fa9981006e0 2024-09-13T17:23:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.532+0000 7fa997fff640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa9840a1120 0x7fa9840a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa9980ffb80 tx=0x7fa988005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:36.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.535+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa9900b76f0 con 0x7fa9981006e0 2024-09-13T17:23:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:36 smithi044 ceph-mon[31175]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.710+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa99810a470 con 0x7fa9981006e0 2024-09-13T17:23:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.710+0000 7fa995ffb640 1 -- 172.21.15.44:0/499277035 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa9900bad10 con 0x7fa9981006e0 2024-09-13T17:23:36.712 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa9840a1120 msgr2=0x7fa9840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa9840a1120 0x7fa9840a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa9980ffb80 tx=0x7fa988005f90 comp rx=0 tx=0).stop 2024-09-13T17:23:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 msgr2=0x7fa99819d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa99819d040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa99002aeb0 tx=0x7fa990003c20 comp rx=0 tx=0).stop 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 shutdown_connections 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa9840a1120 0x7fa9840a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9980ffd20 0x7fa998195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9981006e0 0x7fa99819d040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 --2- 172.21.15.44:0/499277035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9980feb20 0x7fa998195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 >> 172.21.15.44:0/499277035 conn(0x7fa9980fa2d0 msgr2=0x7fa998067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 shutdown_connections 2024-09-13T17:23:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:36.712+0000 7fa99d5a1640 1 -- 172.21.15.44:0/499277035 wait complete. 2024-09-13T17:23:36.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:36 smithi100 ceph-mon[35055]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:36.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:36 smithi070 ceph-mon[34502]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:37.410 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1726248213,"num_in_osds":12,"osd_in_since":1726248193,"num_remapped_pgs":0} 2024-09-13T17:23:37.411 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T17:23:37.617 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:23:37.739 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/499277035' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T17:23:37.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/499277035' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T17:23:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/499277035' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T17:23:38.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:38 smithi100 ceph-mon[35055]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:38.855 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:38 smithi044 ceph-mon[31175]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:38.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:38 smithi070 ceph-mon[34502]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:39.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.530+0000 7f183ef73640 1 -- 172.21.15.44:0/2730318550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380ff880 msgr2=0x7f18381038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:39.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.530+0000 7f183ef73640 1 --2- 172.21.15.44:0/2730318550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380ff880 0x7f18381038d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f182c009930 tx=0x7f182c02a420 comp rx=0 tx=0).stop 2024-09-13T17:23:39.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.530+0000 7f183ef73640 1 -- 172.21.15.44:0/2730318550 shutdown_connections 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 --2- 172.21.15.44:0/2730318550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1838104000 0x7f183810bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 --2- 172.21.15.44:0/2730318550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380ff880 0x7f18381038d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 --2- 172.21.15.44:0/2730318550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18380feed0 0x7f18380ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 -- 172.21.15.44:0/2730318550 >> 172.21.15.44:0/2730318550 conn(0x7f18380fa9b0 msgr2=0x7f18380fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 -- 172.21.15.44:0/2730318550 shutdown_connections 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 -- 172.21.15.44:0/2730318550 wait complete. 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 Processor -- start 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 -- start start 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.531+0000 7f183ef73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 0x7f1838100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183ef73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18380ff880 0x7f1838100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183ef73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1838104000 0x7f18381013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183ef73640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a4510 con 0x7f18380feed0 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183ef73640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a4680 con 0x7f18380ff880 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183ef73640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a4840 con 0x7f1838104000 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 0x7f1838100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 0x7f1838100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52384/0 (socket says 172.21.15.44:52384) 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 -- 172.21.15.44:0/2864197965 learned_addr learned my addr 172.21.15.44:0/2864197965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183d510640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1838104000 0x7f18381013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f1837fff640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18380ff880 0x7f1838100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 -- 172.21.15.44:0/2864197965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1838104000 msgr2=0x7f18381013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1838104000 0x7f18381013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 -- 172.21.15.44:0/2864197965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18380ff880 msgr2=0x7f1838100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18380ff880 0x7f1838100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 -- 172.21.15.44:0/2864197965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c009560 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183cd0f640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 0x7f1838100970 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f182800c970 tx=0x7f182800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828007bb0 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1828007d10 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828004d70 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.532+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18381a4b20 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.533+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18381a4fa0 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.534+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18381003b0 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.535+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1828004f50 con 0x7f18380feed0 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.535+0000 7f1835ffb640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f18200a1100 0x7f18200a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.536+0000 7f1837fff640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f18200a1100 0x7f18200a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:39.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.536+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18280cc130 con 0x7f18380feed0 2024-09-13T17:23:39.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.536+0000 7f1837fff640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f18200a1100 0x7f18200a35c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f182c006010 tx=0x7f182c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:39.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.540+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1828095d60 con 0x7f18380feed0 2024-09-13T17:23:39.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.712+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18380ff2b0 con 0x7f18380feed0 2024-09-13T17:23:39.713 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:39 smithi044 ceph-mon[31175]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:39.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.713+0000 7f1835ffb640 1 -- 172.21.15.44:0/2864197965 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f1828099380 con 0x7f18380feed0 2024-09-13T17:23:39.714 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:39.715 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":80,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:23:34.441805+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:23:39.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f18200a1100 msgr2=0x7f18200a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:39.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f18200a1100 0x7f18200a35c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f182c006010 tx=0x7f182c005a20 comp rx=0 tx=0).stop 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 msgr2=0x7f1838100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 0x7f1838100970 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f182800c970 tx=0x7f182800ce30 comp rx=0 tx=0).stop 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 shutdown_connections 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f18200a1100 0x7f18200a35c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1838104000 0x7f18381013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18380ff880 0x7f1838100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 --2- 172.21.15.44:0/2864197965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18380feed0 0x7f1838100970 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 >> 172.21.15.44:0/2864197965 conn(0x7f18380fa9b0 msgr2=0x7f18380682f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 shutdown_connections 2024-09-13T17:23:39.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:39.715+0000 7f183ef73640 1 -- 172.21.15.44:0/2864197965 wait complete. 2024-09-13T17:23:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:39 smithi070 ceph-mon[34502]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:39 smithi100 ceph-mon[35055]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:40.390 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:23:40.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-13T17:23:40.596 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:23:40.718 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:40 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2864197965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:23:40.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:40 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2864197965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:23:41.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:40 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2864197965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:23:41.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:41 smithi044 ceph-mon[31175]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:41 smithi070 ceph-mon[34502]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:41 smithi100 ceph-mon[35055]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 -- 172.21.15.44:0/2467262388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 msgr2=0x7f2cc81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/2467262388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc81038d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009930 tx=0x7f2cbc02a420 comp rx=0 tx=0).stop 2024-09-13T17:23:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 -- 172.21.15.44:0/2467262388 shutdown_connections 2024-09-13T17:23:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/2467262388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc8104000 0x7f2cc810bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/2467262388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc81038d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/2467262388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc80feed0 0x7f2cc80ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 -- 172.21.15.44:0/2467262388 >> 172.21.15.44:0/2467262388 conn(0x7f2cc80fa9b0 msgr2=0x7f2cc80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 -- 172.21.15.44:0/2467262388 shutdown_connections 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.379+0000 7f2cce7d5640 1 -- 172.21.15.44:0/2467262388 wait complete. 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 Processor -- start 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 -- start start 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc80feed0 0x7f2cc81961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc8104000 0x7f2cc819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819b0d0 con 0x7f2cc80ff880 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819b240 con 0x7f2cc80feed0 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cce7d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819b3b0 con 0x7f2cc8104000 2024-09-13T17:23:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc8196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52402/0 (socket says 172.21.15.44:52402) 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cc77fe640 1 -- 172.21.15.44:0/767964032 learned_addr learned my addr 172.21.15.44:0/767964032 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cccd72640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc8104000 0x7f2cc819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.380+0000 7f2cc7fff640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc80feed0 0x7f2cc81961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc77fe640 1 -- 172.21.15.44:0/767964032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc8104000 msgr2=0x7f2cc819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc77fe640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc8104000 0x7f2cc819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc77fe640 1 -- 172.21.15.44:0/767964032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc80feed0 msgr2=0x7f2cc81961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc77fe640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc80feed0 0x7f2cc81961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc77fe640 1 -- 172.21.15.44:0/767964032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc009560 con 0x7f2cc80ff880 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc7fff640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc80feed0 0x7f2cc81961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc77fe640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc8196720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc005ab0 tx=0x7f2cbc032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc032e10 con 0x7f2cc80ff880 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cbc0046c0 con 0x7f2cc80ff880 2024-09-13T17:23:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc819b630 con 0x7f2cc80ff880 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc003850 con 0x7f2cc80ff880 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.381+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc81a2d80 con 0x7f2cc80ff880 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.382+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc81003b0 con 0x7f2cc80ff880 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.383+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2cbc004830 con 0x7f2cc80ff880 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.384+0000 7f2cc57fa640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2cb00a1080 0x7f2cb00a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.384+0000 7f2cc7fff640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2cb00a1080 0x7f2cb00a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.384+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2cbc0ed050 con 0x7f2cc80ff880 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.385+0000 7f2cc7fff640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2cb00a1080 0x7f2cb00a3540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009b40 tx=0x7f2cb8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.388+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2cbc0b6c80 con 0x7f2cc80ff880 2024-09-13T17:23:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.559+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cc81974f0 con 0x7f2cc80ff880 2024-09-13T17:23:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.560+0000 7f2cc57fa640 1 -- 172.21.15.44:0/767964032 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cbc0ba2a0 con 0x7f2cc80ff880 2024-09-13T17:23:42.561 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:23:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2cb00a1080 msgr2=0x7f2cb00a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2cb00a1080 0x7f2cb00a3540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009b40 tx=0x7f2cb8005e50 comp rx=0 tx=0).stop 2024-09-13T17:23:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 msgr2=0x7f2cc8196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc8196720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc005ab0 tx=0x7f2cbc032c20 comp rx=0 tx=0).stop 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 shutdown_connections 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2cb00a1080 0x7f2cb00a3540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc8104000 0x7f2cc819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc80ff880 0x7f2cc8196720 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 --2- 172.21.15.44:0/767964032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cc80feed0 0x7f2cc81961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 >> 172.21.15.44:0/767964032 conn(0x7f2cc80fa9b0 msgr2=0x7f2cc80fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 shutdown_connections 2024-09-13T17:23:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:42.562+0000 7f2cce7d5640 1 -- 172.21.15.44:0/767964032 wait complete. 2024-09-13T17:23:42.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:42 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/767964032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:23:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:42 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/767964032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:23:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:42 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/767964032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.a on smithi044 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.d on smithi044 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.g on smithi044 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.j on smithi044 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.b on smithi070 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.e on smithi070 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.h on smithi070 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.k on smithi070 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.c on smithi100 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.f on smithi100 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.i on smithi100 2024-09-13T17:23:43.284 INFO:tasks.cephadm:Adding mds.l on smithi100 2024-09-13T17:23:43.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l' 2024-09-13T17:23:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:23:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:43 smithi070 ceph-mon[34502]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:43.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:43 smithi100 ceph-mon[35055]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:44.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:43 smithi044 ceph-mon[31175]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 -- 172.21.15.100:0/2482202111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 msgr2=0x7fee5c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 --2- 172.21.15.100:0/2482202111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c100200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee58009900 tx=0x7fee5802a410 comp rx=0 tx=0).stop 2024-09-13T17:23:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 -- 172.21.15.100:0/2482202111 shutdown_connections 2024-09-13T17:23:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 --2- 172.21.15.100:0/2482202111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee5c101920 0x7fee5c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 --2- 172.21.15.100:0/2482202111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee5c100f60 0x7fee5c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 --2- 172.21.15.100:0/2482202111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c100200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 -- 172.21.15.100:0/2482202111 >> 172.21.15.100:0/2482202111 conn(0x7fee5c0fb5d0 msgr2=0x7fee5c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 -- 172.21.15.100:0/2482202111 shutdown_connections 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.192+0000 7fee63673640 1 -- 172.21.15.100:0/2482202111 wait complete. 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 Processor -- start 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 -- start start 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee5c100f60 0x7fee5c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee5c101920 0x7fee5c19d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c19d790 con 0x7fee5c100f60 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c19d900 con 0x7fee5c101920 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee63673640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c19dac0 con 0x7fee5c0ffe00 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee6140f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee6140f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58030/0 (socket says 172.21.15.100:58030) 2024-09-13T17:23:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee6140f640 1 -- 172.21.15.100:0/1076462260 learned_addr learned my addr 172.21.15.100:0/1076462260 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.193+0000 7fee60c0e640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee5c100f60 0x7fee5c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee6140f640 1 -- 172.21.15.100:0/1076462260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee5c101920 msgr2=0x7fee5c19d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee6140f640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee5c101920 0x7fee5c19d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee6140f640 1 -- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee5c100f60 msgr2=0x7fee5c1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee6140f640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee5c100f60 0x7fee5c1961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee6140f640 1 -- 172.21.15.100:0/1076462260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58009560 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee6140f640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c195ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee58006010 tx=0x7fee58004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee580037f0 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee580044f0 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5803ca80 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c19dd40 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.194+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c19e190 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.195+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c1072b0 con 0x7fee5c0ffe00 2024-09-13T17:23:45.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.196+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fee58003950 con 0x7fee5c0ffe00 2024-09-13T17:23:45.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.197+0000 7fee527fc640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fee480a1080 0x7fee480a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:45.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.197+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fee580ed630 con 0x7fee5c0ffe00 2024-09-13T17:23:45.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.197+0000 7fee60c0e640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fee480a1080 0x7fee480a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:45.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.197+0000 7fee60c0e640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fee480a1080 0x7fee480a3540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005db0 tx=0x7fee4c005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:45.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.201+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fee580b71c0 con 0x7fee5c0ffe00 2024-09-13T17:23:45.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.377+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fee5c02c620 con 0x7fee480a1080 2024-09-13T17:23:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.385+0000 7fee527fc640 1 -- 172.21.15.100:0/1076462260 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fee5c02c620 con 0x7fee480a1080 2024-09-13T17:23:45.386 INFO:teuthology.orchestra.run.smithi100.stdout:Scheduled mds.all update... 2024-09-13T17:23:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fee480a1080 msgr2=0x7fee480a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fee480a1080 0x7fee480a3540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005db0 tx=0x7fee4c005d40 comp rx=0 tx=0).stop 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 msgr2=0x7fee5c195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c195ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee58006010 tx=0x7fee58004970 comp rx=0 tx=0).stop 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 shutdown_connections 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fee480a1080 0x7fee480a3540 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee5c101920 0x7fee5c19d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee5c100f60 0x7fee5c1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 --2- 172.21.15.100:0/1076462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c0ffe00 0x7fee5c195ca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 >> 172.21.15.100:0/1076462260 conn(0x7fee5c0fb5d0 msgr2=0x7fee5c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 shutdown_connections 2024-09-13T17:23:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-13T17:23:45.387+0000 7fee63673640 1 -- 172.21.15.100:0/1076462260 wait complete. 2024-09-13T17:23:46.125 DEBUG:teuthology.orchestra.run.smithi044:mds.a> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.a.service 2024-09-13T17:23:46.128 DEBUG:teuthology.orchestra.run.smithi044:mds.d> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.d.service 2024-09-13T17:23:46.173 DEBUG:teuthology.orchestra.run.smithi044:mds.g> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.g.service 2024-09-13T17:23:46.217 DEBUG:teuthology.orchestra.run.smithi044:mds.j> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.j.service 2024-09-13T17:23:46.260 DEBUG:teuthology.orchestra.run.smithi070:mds.b> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.b.service 2024-09-13T17:23:46.263 DEBUG:teuthology.orchestra.run.smithi070:mds.e> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.e.service 2024-09-13T17:23:46.306 DEBUG:teuthology.orchestra.run.smithi070:mds.h> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.h.service 2024-09-13T17:23:46.350 DEBUG:teuthology.orchestra.run.smithi070:mds.k> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.k.service 2024-09-13T17:23:46.393 DEBUG:teuthology.orchestra.run.smithi100:mds.c> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.c.service 2024-09-13T17:23:46.395 DEBUG:teuthology.orchestra.run.smithi100:mds.f> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.f.service 2024-09-13T17:23:46.438 DEBUG:teuthology.orchestra.run.smithi100:mds.i> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.i.service 2024-09-13T17:23:46.483 DEBUG:teuthology.orchestra.run.smithi100:mds.l> sudo journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.l.service 2024-09-13T17:23:46.526 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-13T17:23:46.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:23:46.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:23:46.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l;count:12 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:46.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:46 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 -- 172.21.15.44:0/3667849202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0fe440 msgr2=0x7fda2c0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 --2- 172.21.15.44:0/3667849202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0fe440 0x7fda2c0fe840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009900 tx=0x7fda1c02a410 comp rx=0 tx=0).stop 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 -- 172.21.15.44:0/3667849202 shutdown_connections 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 --2- 172.21.15.44:0/3667849202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ffff0 0x7fda2c106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 --2- 172.21.15.44:0/3667849202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0ff630 0x7fda2c0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 --2- 172.21.15.44:0/3667849202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0fe440 0x7fda2c0fe840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 -- 172.21.15.44:0/3667849202 >> 172.21.15.44:0/3667849202 conn(0x7fda2c0f9bc0 msgr2=0x7fda2c0fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 -- 172.21.15.44:0/3667849202 shutdown_connections 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 -- 172.21.15.44:0/3667849202 wait complete. 2024-09-13T17:23:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.644+0000 7fda31eb7640 1 Processor -- start 2024-09-13T17:23:46.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 -- start start 2024-09-13T17:23:46.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0fe440 0x7fda2c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ff630 0x7fda2c100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 0x7fda2c100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1016b0 con 0x7fda2c0ffff0 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c101820 con 0x7fda2c0fe440 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda31eb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19fd10 con 0x7fda2c0ff630 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 0x7fda2c100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 0x7fda2c100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52426/0 (socket says 172.21.15.44:52426) 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 -- 172.21.15.44:0/431134345 learned_addr learned my addr 172.21.15.44:0/431134345 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2affd640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ff630 0x7fda2c100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2b7fe640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0fe440 0x7fda2c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 -- 172.21.15.44:0/431134345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ff630 msgr2=0x7fda2c100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ff630 0x7fda2c100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 -- 172.21.15.44:0/431134345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0fe440 msgr2=0x7fda2c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0fe440 0x7fda2c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2bfff640 1 -- 172.21.15.44:0/431134345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009560 con 0x7fda2c0ffff0 2024-09-13T17:23:46.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2affd640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ff630 0x7fda2c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.645+0000 7fda2b7fe640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0fe440 0x7fda2c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.646+0000 7fda2bfff640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 0x7fda2c100f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda2400ec90 tx=0x7fda2400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.646+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24004d20 con 0x7fda2c0ffff0 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.646+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda24004e80 con 0x7fda2c0ffff0 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.646+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c19ffc0 con 0x7fda2c0ffff0 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.646+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240054f0 con 0x7fda2c0ffff0 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.646+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c1a0440 con 0x7fda2c0ffff0 2024-09-13T17:23:46.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.647+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c109380 con 0x7fda2c0ffff0 2024-09-13T17:23:46.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.648+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fda24005650 con 0x7fda2c0ffff0 2024-09-13T17:23:46.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.648+0000 7fda28ff9640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda140a1090 0x7fda140a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.649+0000 7fda2b7fe640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda140a1090 0x7fda140a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.649+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fda240cc960 con 0x7fda2c0ffff0 2024-09-13T17:23:46.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.649+0000 7fda2b7fe640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda140a1090 0x7fda140a3550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005ab0 tx=0x7fda1c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:46.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.653+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fda24096590 con 0x7fda2c0ffff0 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l;count:12 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:46.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:46 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:23:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l;count:12 2024-09-13T17:23:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:23:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:46.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:46.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:23:46.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:46.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:46.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:46.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:46 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.825+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda2c109560 con 0x7fda2c0ffff0 2024-09-13T17:23:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.826+0000 7fda28ff9640 1 -- 172.21.15.44:0/431134345 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18362 (secure 0 0 0) 0x7fda24094e60 con 0x7fda2c0ffff0 2024-09-13T17:23:46.827 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:46.827 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":80,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:23:34.441805+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.827+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda140a1090 msgr2=0x7fda140a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.827+0000 7fda31eb7640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda140a1090 0x7fda140a3550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005ab0 tx=0x7fda1c005a20 comp rx=0 tx=0).stop 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.827+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 msgr2=0x7fda2c100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.827+0000 7fda31eb7640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 0x7fda2c100f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda2400ec90 tx=0x7fda2400c7a0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 shutdown_connections 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda140a1090 0x7fda140a3550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda2c0ff630 0x7fda2c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda2c0ffff0 0x7fda2c100f50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 --2- 172.21.15.44:0/431134345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda2c0fe440 0x7fda2c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 >> 172.21.15.44:0/431134345 conn(0x7fda2c0f9bc0 msgr2=0x7fda2c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 shutdown_connections 2024-09-13T17:23:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.828+0000 7fda31eb7640 1 -- 172.21.15.44:0/431134345 wait complete. 2024-09-13T17:23:46.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:23:46.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:23:46.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 -- 172.21.15.44:0/1268593303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2564100810 msgr2=0x7f25641070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 --2- 172.21.15.44:0/1268593303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2564100810 0x7f25641070a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f255c009ca0 tx=0x7f255c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 -- 172.21.15.44:0/1268593303 shutdown_connections 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 --2- 172.21.15.44:0/1268593303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25640ffe50 0x7f25641002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 --2- 172.21.15.44:0/1268593303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2564100810 0x7f25641070a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 --2- 172.21.15.44:0/1268593303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25640fec60 0x7f25640ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 -- 172.21.15.44:0/1268593303 >> 172.21.15.44:0/1268593303 conn(0x7f25640fa3e0 msgr2=0x7f25640fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 -- 172.21.15.44:0/1268593303 shutdown_connections 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 -- 172.21.15.44:0/1268593303 wait complete. 2024-09-13T17:23:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.987+0000 7f2568f9c640 1 Processor -- start 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 -- start start 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 0x7f2564194af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25640ffe50 0x7f2564195030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2564100810 0x7f256419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256419c5e0 con 0x7f25640fec60 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256419c750 con 0x7f2564100810 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2568f9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256419c910 con 0x7f25640ffe50 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 0x7f2564194af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 0x7f2564194af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52456/0 (socket says 172.21.15.44:52456) 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 -- 172.21.15.44:0/3530428305 learned_addr learned my addr 172.21.15.44:0/3530428305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2561d9b640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25640ffe50 0x7f2564195030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f2562d9d640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2564100810 0x7f256419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 -- 172.21.15.44:0/3530428305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25640ffe50 msgr2=0x7f2564195030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25640ffe50 0x7f2564195030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 -- 172.21.15.44:0/3530428305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2564100810 msgr2=0x7f256419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2564100810 0x7f256419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 -- 172.21.15.44:0/3530428305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009900 con 0x7f25640fec60 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f256259c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 0x7f2564194af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f255400c970 tx=0x7f255400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554004220 con 0x7f25640fec60 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2554007e40 con 0x7f25640fec60 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.988+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554003e30 con 0x7f25640fec60 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.989+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256419cbf0 con 0x7f25640fec60 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.989+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256419d0b0 con 0x7f25640fec60 2024-09-13T17:23:46.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.990+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2564106140 con 0x7f25640fec60 2024-09-13T17:23:46.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.991+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f25540079b0 con 0x7f25640fec60 2024-09-13T17:23:46.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.992+0000 7f25537fe640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f254c0a1090 0x7f254c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:46.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.992+0000 7f2561d9b640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f254c0a1090 0x7f254c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:46.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.992+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25540cc530 con 0x7f25640fec60 2024-09-13T17:23:46.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.993+0000 7f2561d9b640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f254c0a1090 0x7f254c0a3550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2564196000 tx=0x7f255800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:46.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:46.995+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2554096160 con 0x7f25640fec60 2024-09-13T17:23:47.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.165+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f256410a3b0 con 0x7f25640fec60 2024-09-13T17:23:47.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.166+0000 7f25537fe640 1 -- 172.21.15.44:0/3530428305 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f255401d0a0 con 0x7f25640fec60 2024-09-13T17:23:47.167 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:23:47.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f254c0a1090 msgr2=0x7f254c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f254c0a1090 0x7f254c0a3550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2564196000 tx=0x7f255800b410 comp rx=0 tx=0).stop 2024-09-13T17:23:47.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 msgr2=0x7f2564194af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 0x7f2564194af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f255400c970 tx=0x7f255400ce30 comp rx=0 tx=0).stop 2024-09-13T17:23:47.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 shutdown_connections 2024-09-13T17:23:47.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f254c0a1090 0x7f254c0a3550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25640ffe50 0x7f2564195030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2564100810 0x7f256419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 --2- 172.21.15.44:0/3530428305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25640fec60 0x7f2564194af0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 >> 172.21.15.44:0/3530428305 conn(0x7f25640fa3e0 msgr2=0x7f2564103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 shutdown_connections 2024-09-13T17:23:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.168+0000 7f2568f9c640 1 -- 172.21.15.44:0/3530428305 wait complete. 2024-09-13T17:23:47.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- 172.21.15.44:0/3928712209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340ffc80 msgr2=0x7f3234100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- 172.21.15.44:0/3928712209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340ffc80 0x7f3234100100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3220009900 tx=0x7f322002a410 comp rx=0 tx=0).stop 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- 172.21.15.44:0/3928712209 shutdown_connections 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- 172.21.15.44:0/3928712209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3234100640 0x7f3234106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- 172.21.15.44:0/3928712209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340ffc80 0x7f3234100100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- 172.21.15.44:0/3928712209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32340fea90 0x7f32340fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- 172.21.15.44:0/3928712209 >> 172.21.15.44:0/3928712209 conn(0x7f32340fa1f0 msgr2=0x7f32340fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- 172.21.15.44:0/3928712209 shutdown_connections 2024-09-13T17:23:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- 172.21.15.44:0/3928712209 wait complete. 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 Processor -- start 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- start start 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 0x7f3234195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32340ffc80 0x7f3234196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3234100640 0x7f323419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419d5d0 con 0x7f32340fea90 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419d740 con 0x7f3234100640 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.282+0000 7f3239da0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419d900 con 0x7f32340ffc80 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 0x7f3234195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 0x7f3234195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52478/0 (socket says 172.21.15.44:52478) 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 -- 172.21.15.44:0/97327908 learned_addr learned my addr 172.21.15.44:0/97327908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3233fff640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32340ffc80 0x7f3234196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f323959f640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3234100640 0x7f323419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 -- 172.21.15.44:0/97327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32340ffc80 msgr2=0x7f3234196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32340ffc80 0x7f3234196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 -- 172.21.15.44:0/97327908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3234100640 msgr2=0x7f323419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3234100640 0x7f323419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 -- 172.21.15.44:0/97327908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220009560 con 0x7f32340fea90 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3238d9e640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 0x7f3234195ae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f322800d770 tx=0x7f322800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228009c30 con 0x7f32340fea90 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f322800de80 con 0x7f32340fea90 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323419dbe0 con 0x7f32340fea90 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.283+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228003d40 con 0x7f32340fea90 2024-09-13T17:23:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.284+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323419e110 con 0x7f32340fea90 2024-09-13T17:23:47.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.284+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323410a1e0 con 0x7f32340fea90 2024-09-13T17:23:47.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.285+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3228003ea0 con 0x7f32340fea90 2024-09-13T17:23:47.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.286+0000 7f3231ffb640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f32240a1110 0x7f32240a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:47.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.287+0000 7f3233fff640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f32240a1110 0x7f32240a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:47.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.287+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32280cbca0 con 0x7f32340fea90 2024-09-13T17:23:47.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.287+0000 7f3233fff640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f32240a1110 0x7f32240a35d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f322002aa00 tx=0x7f3220005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:47.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.291+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3228095980 con 0x7f32340fea90 2024-09-13T17:23:47.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.500+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f323410a3f0 con 0x7f32340fea90 2024-09-13T17:23:47.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.500+0000 7f3231ffb640 1 -- 172.21.15.44:0/97327908 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f3228098fa0 con 0x7f32340fea90 2024-09-13T17:23:47.501 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:47.501 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-13T17:23:47.502 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 1 2024-09-13T17:23:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.502+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f32240a1110 msgr2=0x7f32240a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.502+0000 7f3239da0640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f32240a1110 0x7f32240a35d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f322002aa00 tx=0x7f3220005a20 comp rx=0 tx=0).stop 2024-09-13T17:23:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.502+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 msgr2=0x7f3234195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.502+0000 7f3239da0640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 0x7f3234195ae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f322800d770 tx=0x7f322800dc30 comp rx=0 tx=0).stop 2024-09-13T17:23:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 shutdown_connections 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f32240a1110 0x7f32240a35d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32340ffc80 0x7f3234196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3234100640 0x7f323419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 --2- 172.21.15.44:0/97327908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32340fea90 0x7f3234195ae0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 >> 172.21.15.44:0/97327908 conn(0x7f32340fa1f0 msgr2=0x7f3234067650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 shutdown_connections 2024-09-13T17:23:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:47.503+0000 7f3239da0640 1 -- 172.21.15.44:0/97327908 wait complete. 2024-09-13T17:23:47.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:47 smithi044 ceph-mon[31175]: Deploying daemon mds.l on smithi100 2024-09-13T17:23:47.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:47 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/431134345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:23:47.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:47 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3530428305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:23:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:47 smithi100 ceph-mon[35055]: Deploying daemon mds.l on smithi100 2024-09-13T17:23:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:47 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/431134345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:23:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:47 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3530428305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:23:47.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:47 smithi070 ceph-mon[34502]: Deploying daemon mds.l on smithi100 2024-09-13T17:23:47.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:47 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/431134345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:23:47.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:47 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3530428305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:23:48.024 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:47 smithi100 systemd[1]: Starting Ceph mds.l for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:23:48.334 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 podman[64969]: 2024-09-13 17:23:48.071802586 +0000 UTC m=+0.107917086 container create feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:23:48.334 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 podman[64969]: 2024-09-13 17:23:47.979443047 +0000 UTC m=+0.015557547 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:23:48.334 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 podman[64969]: 2024-09-13 17:23:48.33318387 +0000 UTC m=+0.369298371 container init feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-13T17:23:48.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:48 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/97327908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:48.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:48 smithi070 ceph-mon[34502]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:48.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:48 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6832/459553980,v1:172.21.15.100:6833/459553980] up:boot 2024-09-13T17:23:48.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:48 smithi070 ceph-mon[34502]: fsmap 1 up:standby 2024-09-13T17:23:48.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:48 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:23:48.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/97327908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:48.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-mon[35055]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:48.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6832/459553980,v1:172.21.15.100:6833/459553980] up:boot 2024-09-13T17:23:48.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-mon[35055]: fsmap 1 up:standby 2024-09-13T17:23:48.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:23:48.775 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 podman[64969]: 2024-09-13 17:23:48.336080725 +0000 UTC m=+0.372195232 container start feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-13T17:23:48.775 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-mds[64988]: -- 172.21.15.100:0/720829884 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621ebdaed00 con 0x5621ebd83400 2024-09-13T17:23:48.775 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l[64984]: starting mds.l at 2024-09-13T17:23:48.775 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 bash[64969]: feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 2024-09-13T17:23:48.775 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:23:48 smithi100 systemd[1]: Started Ceph mds.l for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:23:48.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:48 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/97327908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:48.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:48 smithi044 ceph-mon[31175]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:48.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:48 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6832/459553980,v1:172.21.15.100:6833/459553980] up:boot 2024-09-13T17:23:48.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:48 smithi044 ceph-mon[31175]: fsmap 1 up:standby 2024-09-13T17:23:48.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:48 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:23:49.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:23:49.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 -- 172.21.15.44:0/870088895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8103180 msgr2=0x7f00e8105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/870088895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8103180 0x7f00e8105570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d802a410 comp rx=0 tx=0).stop 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 -- 172.21.15.44:0/870088895 shutdown_connections 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/870088895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e8105ab0 0x7f00e8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/870088895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8103180 0x7f00e8105570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/870088895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8100850 0x7f00e8102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 -- 172.21.15.44:0/870088895 >> 172.21.15.44:0/870088895 conn(0x7f00e80fa430 msgr2=0x7f00e80fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 -- 172.21.15.44:0/870088895 shutdown_connections 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.630+0000 7f00ef4f7640 1 -- 172.21.15.44:0/870088895 wait complete. 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 Processor -- start 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 -- start start 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e819d890 con 0x7f00e8105ab0 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e819da00 con 0x7f00e8100850 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ef4f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e819dbc0 con 0x7f00e8103180 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54238/0 (socket says 172.21.15.44:54238) 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 -- 172.21.15.44:0/2815702190 learned_addr learned my addr 172.21.15.44:0/2815702190 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00ed293640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eca92640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 -- 172.21.15.44:0/2815702190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103180 msgr2=0x7f00e81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 -- 172.21.15.44:0/2815702190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e8100850 msgr2=0x7f00e8195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eda94640 1 -- 172.21.15.44:0/2815702190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8009560 con 0x7f00e8105ab0 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.631+0000 7f00eca92640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.632+0000 7f00eda94640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eb40 tx=0x7f00e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.632+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0004b20 con 0x7f00e8105ab0 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.632+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00e0004c80 con 0x7f00e8105ab0 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.632+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e819dea0 con 0x7f00e8105ab0 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.632+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0010690 con 0x7f00e8105ab0 2024-09-13T17:23:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.632+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e819e2c0 con 0x7f00e8105ab0 2024-09-13T17:23:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.633+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e8103560 con 0x7f00e8105ab0 2024-09-13T17:23:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.634+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f00e0005000 con 0x7f00e8105ab0 2024-09-13T17:23:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.635+0000 7f00de7fc640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00d40a1110 0x7f00d40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.635+0000 7f00ed293640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00d40a1110 0x7f00d40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.635+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f00e0014070 con 0x7f00e8105ab0 2024-09-13T17:23:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.636+0000 7f00ed293640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00d40a1110 0x7f00d40a35d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00e8197280 tx=0x7f00e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.638+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f00e0096750 con 0x7f00e8105ab0 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:49 smithi100 ceph-mon[35055]: Deploying daemon mds.k on smithi070 2024-09-13T17:23:49.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.850+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00e80fbf20 con 0x7f00e8105ab0 2024-09-13T17:23:49.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.850+0000 7f00de7fc640 1 -- 172.21.15.44:0/2815702190 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f00e0098680 con 0x7f00e8105ab0 2024-09-13T17:23:49.851 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:49.851 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:23:49.852 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-09-13T17:23:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00d40a1110 msgr2=0x7f00d40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00d40a1110 0x7f00d40a35d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00e8197280 tx=0x7f00e400b410 comp rx=0 tx=0).stop 2024-09-13T17:23:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 msgr2=0x7f00e819d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eb40 tx=0x7f00e000c2d0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 shutdown_connections 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00d40a1110 0x7f00d40a35d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 --2- 172.21.15.44:0/2815702190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 >> 172.21.15.44:0/2815702190 conn(0x7f00e80fa430 msgr2=0x7f00e8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 shutdown_connections 2024-09-13T17:23:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:49.852+0000 7f00ef4f7640 1 -- 172.21.15.44:0/2815702190 wait complete. 2024-09-13T17:23:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:49 smithi044 ceph-mon[31175]: Deploying daemon mds.k on smithi070 2024-09-13T17:23:49.942 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.942 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.943 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:49.943 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:49.943 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:49.943 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:49.943 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:49 smithi070 ceph-mon[34502]: Deploying daemon mds.k on smithi070 2024-09-13T17:23:50.482 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 systemd[1]: Starting Ceph mds.k for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:23:50.767 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:50 smithi070 ceph-mon[34502]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:50.767 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2815702190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:50.768 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 podman[63187]: 2024-09-13 17:23:50.521212274 +0000 UTC m=+0.016233973 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:23:50.768 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 podman[63187]: 2024-09-13 17:23:50.643933075 +0000 UTC m=+0.138954764 container create fb48a9aeda9f4c589e5ad9d6c6ac61f97e8f045dd896b2ff7fa36ac7edf8ea33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T17:23:50.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:50 smithi100 ceph-mon[35055]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:50.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2815702190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:50.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:50 smithi044 ceph-mon[31175]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:50.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2815702190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:51.106 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 podman[63187]: 2024-09-13 17:23:50.88915363 +0000 UTC m=+0.384175316 container init fb48a9aeda9f4c589e5ad9d6c6ac61f97e8f045dd896b2ff7fa36ac7edf8ea33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:23:51.106 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 podman[63187]: 2024-09-13 17:23:50.892000693 +0000 UTC m=+0.387022379 container start fb48a9aeda9f4c589e5ad9d6c6ac61f97e8f045dd896b2ff7fa36ac7edf8ea33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-13T17:23:51.106 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 ceph-mds[63206]: -- 172.21.15.70:0/3452375007 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c16a32d00 con 0x560c16a07400 2024-09-13T17:23:51.107 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:50 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k[63202]: starting mds.k at 2024-09-13T17:23:51.447 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:51 smithi070 bash[63187]: fb48a9aeda9f4c589e5ad9d6c6ac61f97e8f045dd896b2ff7fa36ac7edf8ea33 2024-09-13T17:23:51.447 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:23:51 smithi070 systemd[1]: Started Ceph mds.k for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:23:51.868 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:23:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 -- 172.21.15.44:0/4164288101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c103180 msgr2=0x7f860c105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 --2- 172.21.15.44:0/4164288101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c103180 0x7f860c105570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8600009900 tx=0x7f860002a410 comp rx=0 tx=0).stop 2024-09-13T17:23:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 -- 172.21.15.44:0/4164288101 shutdown_connections 2024-09-13T17:23:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 --2- 172.21.15.44:0/4164288101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c105ab0 0x7f860c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 --2- 172.21.15.44:0/4164288101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c103180 0x7f860c105570 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 --2- 172.21.15.44:0/4164288101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 0x7f860c102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 -- 172.21.15.44:0/4164288101 >> 172.21.15.44:0/4164288101 conn(0x7f860c0fa430 msgr2=0x7f860c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 -- 172.21.15.44:0/4164288101 shutdown_connections 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.971+0000 7f861166c640 1 -- 172.21.15.44:0/4164288101 wait complete. 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 Processor -- start 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 -- start start 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 0x7f860c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c103180 0x7f860c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 0x7f860c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c19d8a0 con 0x7f860c105ab0 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c19da10 con 0x7f860c100850 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f861166c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c19dbd0 con 0x7f860c103180 2024-09-13T17:23:51.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 0x7f860c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 0x7f860c19d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54258/0 (socket says 172.21.15.44:54258) 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 -- 172.21.15.44:0/2055978363 learned_addr learned my addr 172.21.15.44:0/2055978363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860affd640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 0x7f860c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 -- 172.21.15.44:0/2055978363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c103180 msgr2=0x7f860c1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c103180 0x7f860c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 -- 172.21.15.44:0/2055978363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 msgr2=0x7f860c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 0x7f860c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860b7fe640 1 -- 172.21.15.44:0/2055978363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8600009560 con 0x7f860c105ab0 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.972+0000 7f860affd640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 0x7f860c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.973+0000 7f860b7fe640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 0x7f860c19d360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f860400e9d0 tx=0x7f860400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.973+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400cd80 con 0x7f860c105ab0 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.973+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8604005020 con 0x7f860c105ab0 2024-09-13T17:23:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.973+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c19deb0 con 0x7f860c105ab0 2024-09-13T17:23:51.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.973+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86040116d0 con 0x7f860c105ab0 2024-09-13T17:23:51.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.973+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c19e3e0 con 0x7f860c105ab0 2024-09-13T17:23:51.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.974+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860c0fbf20 con 0x7f860c105ab0 2024-09-13T17:23:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.975+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8604005190 con 0x7f860c105ab0 2024-09-13T17:23:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.976+0000 7f85f3fff640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85f40a1110 0x7f85f40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.976+0000 7f860affd640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85f40a1110 0x7f85f40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.976+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f86040cc8b0 con 0x7f860c105ab0 2024-09-13T17:23:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.976+0000 7f860affd640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85f40a1110 0x7f85f40a35d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005d00 tx=0x7f85fc005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:51.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:51.979+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8604096560 con 0x7f860c105ab0 2024-09-13T17:23:52.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.189+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f860c10b580 con 0x7f860c105ab0 2024-09-13T17:23:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.190+0000 7f85f3fff640 1 -- 172.21.15.44:0/2055978363 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2011 (secure 0 0 0) 0x7f8604099b80 con 0x7f860c105ab0 2024-09-13T17:23:52.191 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:52.191 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:23:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85f40a1110 msgr2=0x7f85f40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85f40a1110 0x7f85f40a35d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005d00 tx=0x7f85fc005ed0 comp rx=0 tx=0).stop 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 msgr2=0x7f860c19d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 0x7f860c19d360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f860400e9d0 tx=0x7f860400ee90 comp rx=0 tx=0).stop 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 shutdown_connections 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85f40a1110 0x7f85f40a35d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c103180 0x7f860c1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860c105ab0 0x7f860c19d360 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 --2- 172.21.15.44:0/2055978363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c100850 0x7f860c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 >> 172.21.15.44:0/2055978363 conn(0x7f860c0fa430 msgr2=0x7f860c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 shutdown_connections 2024-09-13T17:23:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:52.192+0000 7f861166c640 1 -- 172.21.15.44:0/2055978363 wait complete. 2024-09-13T17:23:52.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: Deploying daemon mds.j on smithi044 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6832/982414748,v1:172.21.15.70:6833/982414748] up:boot 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: fsmap 2 up:standby 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T17:23:52.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:52 smithi070 ceph-mon[34502]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: Deploying daemon mds.j on smithi044 2024-09-13T17:23:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6832/982414748,v1:172.21.15.70:6833/982414748] up:boot 2024-09-13T17:23:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: fsmap 2 up:standby 2024-09-13T17:23:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T17:23:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:52 smithi100 ceph-mon[35055]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:52.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: Deploying daemon mds.j on smithi044 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6832/982414748,v1:172.21.15.70:6833/982414748] up:boot 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: fsmap 2 up:standby 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T17:23:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:52 smithi044 ceph-mon[31175]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:53.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:53 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2055978363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:53.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:53 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2055978363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:53 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2055978363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:54.106 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:23:53 smithi044 systemd[1]: Started Ceph mds.j for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:23:54.205 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:23:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 -- 172.21.15.44:0/1203622692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 msgr2=0x7f8700100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 --2- 172.21.15.44:0/1203622692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700100310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002a410 comp rx=0 tx=0).stop 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 -- 172.21.15.44:0/1203622692 shutdown_connections 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 --2- 172.21.15.44:0/1203622692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 0x7f87001070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 --2- 172.21.15.44:0/1203622692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700100310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 --2- 172.21.15.44:0/1203622692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000feca0 0x7f87000ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 -- 172.21.15.44:0/1203622692 >> 172.21.15.44:0/1203622692 conn(0x7f87000fa440 msgr2=0x7f87000fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 -- 172.21.15.44:0/1203622692 shutdown_connections 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.311+0000 7f8706b76640 1 -- 172.21.15.44:0/1203622692 wait complete. 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 Processor -- start 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 -- start start 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000feca0 0x7f8700102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 0x7f87001017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8700101f10 con 0x7f87000ffe90 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8700102080 con 0x7f8700100850 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8706b76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019fe60 con 0x7f87000feca0 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54286/0 (socket says 172.21.15.44:54286) 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 -- 172.21.15.44:0/883557948 learned_addr learned my addr 172.21.15.44:0/883557948 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8705113640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 0x7f87001017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 -- 172.21.15.44:0/883557948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000feca0 msgr2=0x7f8700102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8704912640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000feca0 0x7f8700102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000feca0 0x7f8700102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 -- 172.21.15.44:0/883557948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 msgr2=0x7f87001017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 0x7f87001017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 -- 172.21.15.44:0/883557948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0009560 con 0x7f87000ffe90 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f8705113640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 0x7f87001017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.312+0000 7f86f7fff640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700101270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86f002aa00 tx=0x7f86f0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.313+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f00044f0 con 0x7f87000ffe90 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.313+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87001a00b0 con 0x7f87000ffe90 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.313+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86f0003a80 con 0x7f87000ffe90 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.313+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f003caf0 con 0x7f87000ffe90 2024-09-13T17:23:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.313+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a05b0 con 0x7f87000ffe90 2024-09-13T17:23:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.314+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870010a460 con 0x7f87000ffe90 2024-09-13T17:23:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.315+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f86f0003bf0 con 0x7f87000ffe90 2024-09-13T17:23:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.316+0000 7f86f5ffb640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86e80a1090 0x7f86e80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.316+0000 7f8704912640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86e80a1090 0x7f86e80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.316+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f86f00edcf0 con 0x7f87000ffe90 2024-09-13T17:23:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.316+0000 7f8704912640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86e80a1090 0x7f86e80a3550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005fd0 tx=0x7f86fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.319+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f86f00b7880 con 0x7f87000ffe90 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:54.422 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:54 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:54 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:54 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:54.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.528+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f870010a5f0 con 0x7f87000ffe90 2024-09-13T17:23:54.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.528+0000 7f86f5ffb640 1 -- 172.21.15.44:0/883557948 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f86f00baea0 con 0x7f87000ffe90 2024-09-13T17:23:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:54.531 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:23:54.531 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 4 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86e80a1090 msgr2=0x7f86e80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86e80a1090 0x7f86e80a3550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005fd0 tx=0x7f86fc005850 comp rx=0 tx=0).stop 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 msgr2=0x7f8700101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700101270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86f002aa00 tx=0x7f86f0004970 comp rx=0 tx=0).stop 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 shutdown_connections 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86e80a1090 0x7f86e80a3550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700100850 0x7f87001017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87000ffe90 0x7f8700101270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 --2- 172.21.15.44:0/883557948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000feca0 0x7f8700102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.530+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 >> 172.21.15.44:0/883557948 conn(0x7f87000fa440 msgr2=0x7f8700067630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.531+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 shutdown_connections 2024-09-13T17:23:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:54.531+0000 7f8706b76640 1 -- 172.21.15.44:0/883557948 wait complete. 2024-09-13T17:23:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:55 smithi044 ceph-mon[31175]: Deploying daemon mds.d on smithi044 2024-09-13T17:23:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:55 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.44:6834/673823135,v1:172.21.15.44:6835/673823135] up:boot 2024-09-13T17:23:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:55 smithi044 ceph-mon[31175]: fsmap 3 up:standby 2024-09-13T17:23:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:55 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T17:23:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:55 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/883557948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:55 smithi070 ceph-mon[34502]: Deploying daemon mds.d on smithi044 2024-09-13T17:23:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:55 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.44:6834/673823135,v1:172.21.15.44:6835/673823135] up:boot 2024-09-13T17:23:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:55 smithi070 ceph-mon[34502]: fsmap 3 up:standby 2024-09-13T17:23:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:55 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T17:23:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:55 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/883557948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:55.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:55 smithi100 ceph-mon[35055]: Deploying daemon mds.d on smithi044 2024-09-13T17:23:55.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:55 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.44:6834/673823135,v1:172.21.15.44:6835/673823135] up:boot 2024-09-13T17:23:55.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:55 smithi100 ceph-mon[35055]: fsmap 3 up:standby 2024-09-13T17:23:55.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:55 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T17:23:55.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:55 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/883557948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:56.107 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:55 smithi044 systemd[1]: Starting Ceph mds.d for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:23:56.420 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:56 smithi044 ceph-mon[31175]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:56.421 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 podman[71387]: 2024-09-13 17:23:56.064729536 +0000 UTC m=+0.016492775 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:23:56.421 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 podman[71387]: 2024-09-13 17:23:56.182691783 +0000 UTC m=+0.134455011 container create 94e9650c1045d50fea442074a2a529219516f92a3343427c84415e08263e599d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:23:56.421 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 podman[71387]: 2024-09-13 17:23:56.393606576 +0000 UTC m=+0.345369810 container init 94e9650c1045d50fea442074a2a529219516f92a3343427c84415e08263e599d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-13T17:23:56.421 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 podman[71387]: 2024-09-13 17:23:56.396602697 +0000 UTC m=+0.348365926 container start 94e9650c1045d50fea442074a2a529219516f92a3343427c84415e08263e599d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T17:23:56.421 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 ceph-mds[71406]: -- 172.21.15.44:0/246341659 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f9f36ad00 con 0x558f9f33f000 2024-09-13T17:23:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:56 smithi070 ceph-mon[34502]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:56 smithi100 ceph-mon[35055]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:56.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:23:56.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 -- 172.21.15.44:0/931644467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068100850 msgr2=0x7f10681070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:56.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 --2- 172.21.15.44:0/931644467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068100850 0x7f10681070e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1060009ca0 tx=0x7f106002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 -- 172.21.15.44:0/931644467 shutdown_connections 2024-09-13T17:23:56.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 --2- 172.21.15.44:0/931644467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10680ffe90 0x7f1068100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 --2- 172.21.15.44:0/931644467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068100850 0x7f10681070e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 --2- 172.21.15.44:0/931644467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10680feca0 0x7f10680ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 -- 172.21.15.44:0/931644467 >> 172.21.15.44:0/931644467 conn(0x7f10680fa440 msgr2=0x7f10680fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 -- 172.21.15.44:0/931644467 shutdown_connections 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.650+0000 7f106d29c640 1 -- 172.21.15.44:0/931644467 wait complete. 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 Processor -- start 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 -- start start 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 0x7f1068068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10680ffe90 0x7f106819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068100850 0x7f106819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106819d740 con 0x7f10680feca0 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106819d8b0 con 0x7f10680ffe90 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f106d29c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106819da70 con 0x7f1068100850 2024-09-13T17:23:56.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 0x7f1068068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 0x7f1068068150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54330/0 (socket says 172.21.15.44:54330) 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 -- 172.21.15.44:0/2464332750 learned_addr learned my addr 172.21.15.44:0/2464332750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f10667fc640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10680ffe90 0x7f106819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 -- 172.21.15.44:0/2464332750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068100850 msgr2=0x7f106819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068100850 0x7f106819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 -- 172.21.15.44:0/2464332750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10680ffe90 msgr2=0x7f106819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10680ffe90 0x7f106819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 -- 172.21.15.44:0/2464332750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1060009900 con 0x7f10680feca0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.651+0000 7f1066ffd640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 0x7f1068068150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f105800d770 tx=0x7f105800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.652+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058009c30 con 0x7f10680feca0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.652+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f105800de80 con 0x7f10680feca0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.652+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106819dd50 con 0x7f10680feca0 2024-09-13T17:23:56.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.652+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058003d40 con 0x7f10680feca0 2024-09-13T17:23:56.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.652+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106819e1f0 con 0x7f10680feca0 2024-09-13T17:23:56.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.653+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10681061a0 con 0x7f10680feca0 2024-09-13T17:23:56.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.654+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1058003ea0 con 0x7f10680feca0 2024-09-13T17:23:56.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.654+0000 7f104ffff640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f10500a1110 0x7f10500a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:56.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.655+0000 7f10667fc640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f10500a1110 0x7f10500a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:56.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.655+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f10580cbe00 con 0x7f10680feca0 2024-09-13T17:23:56.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.655+0000 7f10667fc640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f10500a1110 0x7f10500a35d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1068067ec0 tx=0x7f105c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:56.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.659+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1058095990 con 0x7f10680feca0 2024-09-13T17:23:56.856 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d[71402]: starting mds.d at 2024-09-13T17:23:56.856 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 bash[71387]: 94e9650c1045d50fea442074a2a529219516f92a3343427c84415e08263e599d 2024-09-13T17:23:56.857 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:23:56 smithi044 systemd[1]: Started Ceph mds.d for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:23:56.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.866+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f106810a410 con 0x7f10680feca0 2024-09-13T17:23:56.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.867+0000 7f104ffff640 1 -- 172.21.15.44:0/2464332750 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f1058098fb0 con 0x7f10680feca0 2024-09-13T17:23:56.869 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:56.869 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:23:56.869 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 4 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f10500a1110 msgr2=0x7f10500a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f10500a1110 0x7f10500a35d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1068067ec0 tx=0x7f105c00b390 comp rx=0 tx=0).stop 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 msgr2=0x7f1068068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 0x7f1068068150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f105800d770 tx=0x7f105800dc30 comp rx=0 tx=0).stop 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 shutdown_connections 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f10500a1110 0x7f10500a35d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068100850 0x7f106819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10680ffe90 0x7f106819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 --2- 172.21.15.44:0/2464332750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10680feca0 0x7f1068068150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:56.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 >> 172.21.15.44:0/2464332750 conn(0x7f10680fa440 msgr2=0x7f1068103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:56.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 shutdown_connections 2024-09-13T17:23:56.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:56.869+0000 7f106d29c640 1 -- 172.21.15.44:0/2464332750 wait complete. 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: Deploying daemon mds.e on smithi070 2024-09-13T17:23:57.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2464332750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:57.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.44:6836/3952931988,v1:172.21.15.44:6837/3952931988] up:boot 2024-09-13T17:23:57.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: fsmap 4 up:standby 2024-09-13T17:23:57.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:57 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: Deploying daemon mds.e on smithi070 2024-09-13T17:23:57.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2464332750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:57.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.44:6836/3952931988,v1:172.21.15.44:6837/3952931988] up:boot 2024-09-13T17:23:57.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: fsmap 4 up:standby 2024-09-13T17:23:57.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:57 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: Deploying daemon mds.e on smithi070 2024-09-13T17:23:57.935 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2464332750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:23:57.936 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.44:6836/3952931988,v1:172.21.15.44:6837/3952931988] up:boot 2024-09-13T17:23:57.936 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: fsmap 4 up:standby 2024-09-13T17:23:57.936 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:57 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T17:23:58.528 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 systemd[1]: Starting Ceph mds.e for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:23:58.804 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:23:58 smithi070 ceph-mon[34502]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:58.804 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 podman[63504]: 2024-09-13 17:23:58.545214897 +0000 UTC m=+0.015792001 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:23:58.804 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 podman[63504]: 2024-09-13 17:23:58.674396606 +0000 UTC m=+0.144973704 container create ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:23:58.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:23:58 smithi044 ceph-mon[31175]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:58.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:23:59.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 -- 172.21.15.44:0/2097049869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c105b30 msgr2=0x7f354c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 --2- 172.21.15.44:0/2097049869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c105b30 0x7f354c107f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3544009ca0 tx=0x7f354402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 -- 172.21.15.44:0/2097049869 shutdown_connections 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 --2- 172.21.15.44:0/2097049869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c105b30 0x7f354c107f20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 --2- 172.21.15.44:0/2097049869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f354c103200 0x7f354c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 --2- 172.21.15.44:0/2097049869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1008d0 0x7f354c102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.997+0000 7f3554725640 1 -- 172.21.15.44:0/2097049869 >> 172.21.15.44:0/2097049869 conn(0x7f354c0fa430 msgr2=0x7f354c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 -- 172.21.15.44:0/2097049869 shutdown_connections 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 -- 172.21.15.44:0/2097049869 wait complete. 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 Processor -- start 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 -- start start 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f354c1008d0 0x7f354c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 0x7f354c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c105b30 0x7f354c19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:59.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c19d820 con 0x7f354c103200 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c19d990 con 0x7f354c1008d0 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3554725640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c19db50 con 0x7f354c105b30 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3551cc0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 0x7f354c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3551cc0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 0x7f354c196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54352/0 (socket says 172.21.15.44:54352) 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f3551cc0640 1 -- 172.21.15.44:0/743722373 learned_addr learned my addr 172.21.15.44:0/743722373 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.998+0000 7f35524c1640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f354c1008d0 0x7f354c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3552cc2640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c105b30 0x7f354c19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3551cc0640 1 -- 172.21.15.44:0/743722373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c105b30 msgr2=0x7f354c19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3551cc0640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c105b30 0x7f354c19d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3551cc0640 1 -- 172.21.15.44:0/743722373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f354c1008d0 msgr2=0x7f354c195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3551cc0640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f354c1008d0 0x7f354c195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3551cc0640 1 -- 172.21.15.44:0/743722373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544009900 con 0x7f354c103200 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3551cc0640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 0x7f354c196280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f353c00e9c0 tx=0x7f353c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c009700 con 0x7f354c103200 2024-09-13T17:23:59.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f353c00cd80 con 0x7f354c103200 2024-09-13T17:23:59.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c19de30 con 0x7f354c103200 2024-09-13T17:23:59.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c010430 con 0x7f354c103200 2024-09-13T17:23:59.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:58.999+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c19e2b0 con 0x7f354c103200 2024-09-13T17:23:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.000+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c0fbf20 con 0x7f354c103200 2024-09-13T17:23:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.001+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f353c004eb0 con 0x7f354c103200 2024-09-13T17:23:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.002+0000 7f35437fe640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f35380a1110 0x7f35380a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:23:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.002+0000 7f35524c1640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f35380a1110 0x7f35380a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:23:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.002+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f353c014070 con 0x7f354c103200 2024-09-13T17:23:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.003+0000 7f35524c1640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f35380a1110 0x7f35380a35d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f354c197210 tx=0x7f354800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:23:59.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.007+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f353c096540 con 0x7f354c103200 2024-09-13T17:23:59.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:23:58 smithi100 ceph-mon[35055]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:23:59.197 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 podman[63504]: 2024-09-13 17:23:58.943927312 +0000 UTC m=+0.414504415 container init ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-13T17:23:59.197 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 podman[63504]: 2024-09-13 17:23:58.946758486 +0000 UTC m=+0.417335588 container start ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T17:23:59.197 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 ceph-mds[63522]: -- 172.21.15.70:0/1043787261 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8164ecd00 con 0x55a8164c1400 2024-09-13T17:23:59.197 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:58 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e[63518]: starting mds.e at 2024-09-13T17:23:59.197 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:59 smithi070 bash[63504]: ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad 2024-09-13T17:23:59.197 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:23:59 smithi070 systemd[1]: Started Ceph mds.e for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:23:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.217+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f354c02c410 con 0x7f354c103200 2024-09-13T17:23:59.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.217+0000 7f35437fe640 1 -- 172.21.15.44:0/743722373 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3522 (secure 0 0 0) 0x7f353c098470 con 0x7f354c103200 2024-09-13T17:23:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:23:59.219 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:23:59.220 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 5 2024-09-13T17:23:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f35380a1110 msgr2=0x7f35380a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f35380a1110 0x7f35380a35d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f354c197210 tx=0x7f354800b410 comp rx=0 tx=0).stop 2024-09-13T17:23:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 msgr2=0x7f354c196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:23:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 0x7f354c196280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f353c00e9c0 tx=0x7f353c00ee80 comp rx=0 tx=0).stop 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 shutdown_connections 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f35380a1110 0x7f35380a35d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c105b30 0x7f354c19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c103200 0x7f354c196280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 --2- 172.21.15.44:0/743722373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f354c1008d0 0x7f354c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 >> 172.21.15.44:0/743722373 conn(0x7f354c0fa430 msgr2=0x7f354c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 shutdown_connections 2024-09-13T17:23:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:23:59.219+0000 7f3554725640 1 -- 172.21.15.44:0/743722373 wait complete. 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: Deploying daemon mds.f on smithi100 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/743722373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6834/4022128703,v1:172.21.15.70:6835/4022128703] up:boot 2024-09-13T17:24:00.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: fsmap 5 up:standby 2024-09-13T17:24:00.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T17:24:00.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:00 smithi044 ceph-mon[31175]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: Deploying daemon mds.f on smithi100 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/743722373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6834/4022128703,v1:172.21.15.70:6835/4022128703] up:boot 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: fsmap 5 up:standby 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T17:24:00.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:00 smithi070 ceph-mon[34502]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: Deploying daemon mds.f on smithi100 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/743722373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:00.492 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6834/4022128703,v1:172.21.15.70:6835/4022128703] up:boot 2024-09-13T17:24:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: fsmap 5 up:standby 2024-09-13T17:24:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T17:24:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:00 smithi100 ceph-mon[35055]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:01.024 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:00 smithi100 systemd[1]: Starting Ceph mds.f for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:24:01.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 -- 172.21.15.44:0/4134100877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 msgr2=0x7f891c102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 --2- 172.21.15.44:0/4134100877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c102c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8918009900 tx=0x7f891802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 -- 172.21.15.44:0/4134100877 shutdown_connections 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 --2- 172.21.15.44:0/4134100877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891c105a80 0x7f891c107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 --2- 172.21.15.44:0/4134100877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f891c103150 0x7f891c105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 --2- 172.21.15.44:0/4134100877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c102c10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 -- 172.21.15.44:0/4134100877 >> 172.21.15.44:0/4134100877 conn(0x7f891c0fa440 msgr2=0x7f891c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 -- 172.21.15.44:0/4134100877 shutdown_connections 2024-09-13T17:24:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.347+0000 7f892353e640 1 -- 172.21.15.44:0/4134100877 wait complete. 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 Processor -- start 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 -- start start 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891c103150 0x7f891c1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f891c105a80 0x7f891c19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c19d7a0 con 0x7f891c100820 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c19d910 con 0x7f891c105a80 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f892353e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c19dad0 con 0x7f891c103150 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55946/0 (socket says 172.21.15.44:55946) 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 -- 172.21.15.44:0/230879958 learned_addr learned my addr 172.21.15.44:0/230879958 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f8920ad9640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891c103150 0x7f891c1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f8921adb640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f891c105a80 0x7f891c19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 -- 172.21.15.44:0/230879958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891c103150 msgr2=0x7f891c1961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891c103150 0x7f891c1961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 -- 172.21.15.44:0/230879958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f891c105a80 msgr2=0x7f891c19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f891c105a80 0x7f891c19d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 -- 172.21.15.44:0/230879958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009560 con 0x7f891c100820 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.348+0000 7f89212da640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c195c90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89180057c0 tx=0x7f89180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.349+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918031d70 con 0x7f891c100820 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.349+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c19dd50 con 0x7f891c100820 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.349+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89180044e0 con 0x7f891c100820 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.349+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180039c0 con 0x7f891c100820 2024-09-13T17:24:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.349+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c19e1a0 con 0x7f891c100820 2024-09-13T17:24:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.350+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c0fbf30 con 0x7f891c100820 2024-09-13T17:24:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.351+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8918003b40 con 0x7f891c100820 2024-09-13T17:24:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.351+0000 7f890e7fc640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89080a1090 0x7f89080a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.352+0000 7f8920ad9640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89080a1090 0x7f89080a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.352+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89180ed870 con 0x7f891c100820 2024-09-13T17:24:01.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.352+0000 7f8920ad9640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89080a1090 0x7f89080a3550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f891c197190 tx=0x7f891000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:01.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.356+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f89180b7400 con 0x7f891c100820 2024-09-13T17:24:01.406 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 podman[65272]: 2024-09-13 17:24:01.056168558 +0000 UTC m=+0.015794583 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:01.406 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 podman[65272]: 2024-09-13 17:24:01.168706497 +0000 UTC m=+0.128332510 container create 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-13T17:24:01.406 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 podman[65272]: 2024-09-13 17:24:01.40519344 +0000 UTC m=+0.364819453 container init 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-13T17:24:01.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.565+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f891c10b580 con 0x7f891c100820 2024-09-13T17:24:01.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.566+0000 7f890e7fc640 1 -- 172.21.15.44:0/230879958 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5039 (secure 0 0 0) 0x7f89180baa20 con 0x7f891c100820 2024-09-13T17:24:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:01.568 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-09-13T17:24:01.568 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89080a1090 msgr2=0x7f89080a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89080a1090 0x7f89080a3550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f891c197190 tx=0x7f891000a2d0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 msgr2=0x7f891c195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c195c90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89180057c0 tx=0x7f89180042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 shutdown_connections 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89080a1090 0x7f89080a3550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891c103150 0x7f891c1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f891c105a80 0x7f891c19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 --2- 172.21.15.44:0/230879958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c100820 0x7f891c195c90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 >> 172.21.15.44:0/230879958 conn(0x7f891c0fa440 msgr2=0x7f891c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 shutdown_connections 2024-09-13T17:24:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:01.568+0000 7f892353e640 1 -- 172.21.15.44:0/230879958 wait complete. 2024-09-13T17:24:01.774 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 podman[65272]: 2024-09-13 17:24:01.408027903 +0000 UTC m=+0.367653925 container start 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-13T17:24:01.774 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 ceph-mds[65291]: -- 172.21.15.100:0/1352138818 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5570f374ed00 con 0x5570f3723400 2024-09-13T17:24:01.774 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f[65287]: starting mds.f at 2024-09-13T17:24:01.774 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 bash[65272]: 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 2024-09-13T17:24:01.775 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:24:01 smithi100 systemd[1]: Started Ceph mds.f for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:02.853 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6834/4065662345,v1:172.21.15.100:6835/4065662345] up:boot 2024-09-13T17:24:02.853 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: fsmap 6 up:standby 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/230879958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:02.854 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:02 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:02.878 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6834/4065662345,v1:172.21.15.100:6835/4065662345] up:boot 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: fsmap 6 up:standby 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/230879958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:02.879 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:02 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:02.898 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6834/4065662345,v1:172.21.15.100:6835/4065662345] up:boot 2024-09-13T17:24:02.898 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: fsmap 6 up:standby 2024-09-13T17:24:02.898 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/230879958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:02 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:03.476 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 systemd[1]: Starting Ceph mds.c for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:24:03.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/3830753599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fe640 msgr2=0x7f0a780fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/3830753599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fe640 0x7f0a780fea40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009900 tx=0x7f0a7402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/3830753599 shutdown_connections 2024-09-13T17:24:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/3830753599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a780fef80 0x7f0a780ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/3830753599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780ff920 0x7f0a780ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/3830753599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fe640 0x7f0a780fea40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/3830753599 >> 172.21.15.44:0/3830753599 conn(0x7f0a780fa430 msgr2=0x7f0a780fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/3830753599 shutdown_connections 2024-09-13T17:24:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.696+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/3830753599 wait complete. 2024-09-13T17:24:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 Processor -- start 2024-09-13T17:24:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 -- start start 2024-09-13T17:24:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780fe640 0x7f0a78197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 0x7f0a78198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a780ff920 0x7f0a7819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819fa00 con 0x7f0a780fef80 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819fb70 con 0x7f0a780fe640 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7f3ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819fd30 con 0x7f0a780ff920 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 0x7f0a78198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 0x7f0a78198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55962/0 (socket says 172.21.15.44:55962) 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 -- 172.21.15.44:0/285488324 learned_addr learned my addr 172.21.15.44:0/285488324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7d96b640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a780ff920 0x7f0a7819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7d16a640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780fe640 0x7f0a78197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 -- 172.21.15.44:0/285488324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a780ff920 msgr2=0x7f0a7819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a780ff920 0x7f0a7819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 -- 172.21.15.44:0/285488324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780fe640 msgr2=0x7f0a78197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780fe640 0x7f0a78197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7c969640 1 -- 172.21.15.44:0/285488324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74009560 con 0x7f0a780fef80 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.697+0000 7f0a7d16a640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780fe640 0x7f0a78197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:03.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.698+0000 7f0a7c969640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 0x7f0a78198450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e9f0 tx=0x7f0a6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.698+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68009720 con 0x7f0a780fef80 2024-09-13T17:24:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.698+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a6800cd80 con 0x7f0a780fef80 2024-09-13T17:24:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.698+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7819ffb0 con 0x7f0a780fef80 2024-09-13T17:24:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.698+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68010430 con 0x7f0a780fef80 2024-09-13T17:24:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.698+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a0490 con 0x7f0a780fef80 2024-09-13T17:24:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.699+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a78109660 con 0x7f0a780fef80 2024-09-13T17:24:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.700+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0a68004fb0 con 0x7f0a780fef80 2024-09-13T17:24:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.701+0000 7f0a6e7fc640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0a640a1110 0x7f0a640a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.701+0000 7f0a7d16a640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0a640a1110 0x7f0a640a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.701+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a68014070 con 0x7f0a780fef80 2024-09-13T17:24:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.701+0000 7f0a7d16a640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0a640a1110 0x7f0a640a35d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a740057c0 tx=0x7f0a74005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:03.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.705+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0a68096420 con 0x7f0a780fef80 2024-09-13T17:24:03.726 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:03 smithi100 ceph-mon[35055]: Deploying daemon mds.c on smithi100 2024-09-13T17:24:03.727 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 podman[65577]: 2024-09-13 17:24:03.479748984 +0000 UTC m=+0.024041003 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:03.727 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 podman[65577]: 2024-09-13 17:24:03.592111597 +0000 UTC m=+0.136403599 container create f9613c3f418c1953f1942fd498463b1421d215241ecfcc3aef2a9130049fbe57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-13T17:24:03.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:03 smithi044 ceph-mon[31175]: Deploying daemon mds.c on smithi100 2024-09-13T17:24:03.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.913+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a781090d0 con 0x7f0a780fef80 2024-09-13T17:24:03.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.913+0000 7f0a6e7fc640 1 -- 172.21.15.44:0/285488324 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5039 (secure 0 0 0) 0x7f0a68099a40 con 0x7f0a780fef80 2024-09-13T17:24:03.915 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:03.915 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-09-13T17:24:03.916 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2024-09-13T17:24:03.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0a640a1110 msgr2=0x7f0a640a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0a640a1110 0x7f0a640a35d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a740057c0 tx=0x7f0a74005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 msgr2=0x7f0a78198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 0x7f0a78198450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e9f0 tx=0x7f0a6800edf0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 shutdown_connections 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0a640a1110 0x7f0a640a35d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a780ff920 0x7f0a7819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a780fef80 0x7f0a78198450 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 --2- 172.21.15.44:0/285488324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a780fe640 0x7f0a78197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 >> 172.21.15.44:0/285488324 conn(0x7f0a780fa430 msgr2=0x7f0a780fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 shutdown_connections 2024-09-13T17:24:03.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:03.915+0000 7f0a7f3ce640 1 -- 172.21.15.44:0/285488324 wait complete. 2024-09-13T17:24:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:03 smithi070 ceph-mon[34502]: Deploying daemon mds.c on smithi100 2024-09-13T17:24:04.274 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 podman[65577]: 2024-09-13 17:24:03.836852089 +0000 UTC m=+0.381144098 container init f9613c3f418c1953f1942fd498463b1421d215241ecfcc3aef2a9130049fbe57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T17:24:04.274 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 podman[65577]: 2024-09-13 17:24:03.839649679 +0000 UTC m=+0.383941682 container start f9613c3f418c1953f1942fd498463b1421d215241ecfcc3aef2a9130049fbe57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T17:24:04.274 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 ceph-mds[65596]: -- 172.21.15.100:0/3807650694 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e257714d00 con 0x55e2576e8800 2024-09-13T17:24:04.274 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c[65592]: starting mds.c at 2024-09-13T17:24:04.274 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 bash[65577]: f9613c3f418c1953f1942fd498463b1421d215241ecfcc3aef2a9130049fbe57 2024-09-13T17:24:04.275 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:24:03 smithi100 systemd[1]: Started Ceph mds.c for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:04.623 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/285488324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:04.624 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:04 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/285488324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:04 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/285488324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:05.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:05.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:04 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:05 smithi070 ceph-mon[34502]: Deploying daemon mds.h on smithi070 2024-09-13T17:24:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:05 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] up:boot 2024-09-13T17:24:05.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:05 smithi070 ceph-mon[34502]: fsmap 7 up:standby 2024-09-13T17:24:05.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:05 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:24:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:05 smithi044 ceph-mon[31175]: Deploying daemon mds.h on smithi070 2024-09-13T17:24:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:05 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] up:boot 2024-09-13T17:24:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:05 smithi044 ceph-mon[31175]: fsmap 7 up:standby 2024-09-13T17:24:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:05 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:24:05.929 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:05 smithi100 ceph-mon[35055]: Deploying daemon mds.h on smithi070 2024-09-13T17:24:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:05 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] up:boot 2024-09-13T17:24:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:05 smithi100 ceph-mon[35055]: fsmap 7 up:standby 2024-09-13T17:24:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:05 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:24:06.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 -- 172.21.15.44:0/3380534835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 msgr2=0x7ff1701001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:06.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 --2- 172.21.15.44:0/3380534835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff1701001c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009900 tx=0x7ff16c02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:06.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 -- 172.21.15.44:0/3380534835 shutdown_connections 2024-09-13T17:24:06.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 --2- 172.21.15.44:0/3380534835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff170101970 0x7ff170108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 --2- 172.21.15.44:0/3380534835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff170100fb0 0x7ff170101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 --2- 172.21.15.44:0/3380534835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff1701001c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 -- 172.21.15.44:0/3380534835 >> 172.21.15.44:0/3380534835 conn(0x7ff1700fb540 msgr2=0x7ff1700fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 -- 172.21.15.44:0/3380534835 shutdown_connections 2024-09-13T17:24:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.042+0000 7ff177b9c640 1 -- 172.21.15.44:0/3380534835 wait complete. 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 Processor -- start 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 -- start start 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff170195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff170100fb0 0x7ff1701961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff170101970 0x7ff17019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17019d7b0 con 0x7ff1700ffdc0 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17019d920 con 0x7ff170101970 2024-09-13T17:24:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff177b9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17019dae0 con 0x7ff170100fb0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff170195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff170195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55984/0 (socket says 172.21.15.44:55984) 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 -- 172.21.15.44:0/2511025143 learned_addr learned my addr 172.21.15.44:0/2511025143 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff176139640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff170101970 0x7ff17019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175137640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff170100fb0 0x7ff1701961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 -- 172.21.15.44:0/2511025143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff170100fb0 msgr2=0x7ff1701961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff170100fb0 0x7ff1701961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 -- 172.21.15.44:0/2511025143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff170101970 msgr2=0x7ff17019d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff170101970 0x7ff17019d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.043+0000 7ff175938640 1 -- 172.21.15.44:0/2511025143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c009560 con 0x7ff1700ffdc0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.044+0000 7ff175938640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff170195ca0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005ab0 tx=0x7ff16c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.044+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c031d70 con 0x7ff1700ffdc0 2024-09-13T17:24:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.044+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff16c0044e0 con 0x7ff1700ffdc0 2024-09-13T17:24:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.044+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17019dd60 con 0x7ff1700ffdc0 2024-09-13T17:24:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.044+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c003b00 con 0x7ff1700ffdc0 2024-09-13T17:24:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.044+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17019e1c0 con 0x7ff1700ffdc0 2024-09-13T17:24:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.045+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1701072e0 con 0x7ff1700ffdc0 2024-09-13T17:24:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.046+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff16c004c50 con 0x7ff1700ffdc0 2024-09-13T17:24:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.047+0000 7ff166ffd640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff15c0a1090 0x7ff15c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.047+0000 7ff175137640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff15c0a1090 0x7ff15c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.047+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff16c0ed8b0 con 0x7ff1700ffdc0 2024-09-13T17:24:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.047+0000 7ff175137640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff15c0a1090 0x7ff15c0a3550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff160005fd0 tx=0x7ff160005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:06.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.050+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff16c0b74e0 con 0x7ff1700ffdc0 2024-09-13T17:24:06.101 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:05 smithi070 systemd[1]: Starting Ceph mds.h for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:24:06.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.257+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff17010b510 con 0x7ff1700ffdc0 2024-09-13T17:24:06.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.257+0000 7ff166ffd640 1 -- 172.21.15.44:0/2511025143 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5796 (secure 0 0 0) 0x7ff16c0bab00 con 0x7ff1700ffdc0 2024-09-13T17:24:06.260 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:06.260 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-09-13T17:24:06.260 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff15c0a1090 msgr2=0x7ff15c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff15c0a1090 0x7ff15c0a3550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff160005fd0 tx=0x7ff160005850 comp rx=0 tx=0).stop 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 msgr2=0x7ff170195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff170195ca0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005ab0 tx=0x7ff16c0042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 shutdown_connections 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff15c0a1090 0x7ff15c0a3550 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff170100fb0 0x7ff1701961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff170101970 0x7ff17019d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 --2- 172.21.15.44:0/2511025143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1700ffdc0 0x7ff170195ca0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.259+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 >> 172.21.15.44:0/2511025143 conn(0x7ff1700fb540 msgr2=0x7ff1701044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.260+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 shutdown_connections 2024-09-13T17:24:06.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:06.260+0000 7ff177b9c640 1 -- 172.21.15.44:0/2511025143 wait complete. 2024-09-13T17:24:06.400 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 podman[63812]: 2024-09-13 17:24:06.00306232 +0000 UTC m=+0.016224190 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:06.400 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 podman[63812]: 2024-09-13 17:24:06.104700565 +0000 UTC m=+0.117862477 container create 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:24:06.400 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 podman[63812]: 2024-09-13 17:24:06.39944757 +0000 UTC m=+0.412609439 container init 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-13T17:24:06.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:06 smithi070 ceph-mon[34502]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:06.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:06 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2511025143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:06.697 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 podman[63812]: 2024-09-13 17:24:06.402447531 +0000 UTC m=+0.415609411 container start 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:24:06.697 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 ceph-mds[63831]: -- 172.21.15.70:0/413484329 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a41b1c2d00 con 0x55a41b197400 2024-09-13T17:24:06.697 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h[63827]: starting mds.h at 2024-09-13T17:24:06.698 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 bash[63812]: 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 2024-09-13T17:24:06.698 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:24:06 smithi070 systemd[1]: Started Ceph mds.h for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:06.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:06 smithi044 ceph-mon[31175]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:06.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:06 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2511025143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:06.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:06 smithi100 ceph-mon[35055]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:06.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:06 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2511025143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] up:boot 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: fsmap 8 up:standby 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:07.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:07 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:07.879 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] up:boot 2024-09-13T17:24:07.879 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: fsmap 8 up:standby 2024-09-13T17:24:07.879 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:24:07.879 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.879 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.880 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.880 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:07.880 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:07.880 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:07 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] up:boot 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: fsmap 8 up:standby 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:07.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:07 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:08.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/135639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 msgr2=0x7f0bb00ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/135639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb00ff0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba002a410 comp rx=0 tx=0).stop 2024-09-13T17:24:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/135639069 shutdown_connections 2024-09-13T17:24:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/135639069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb00ffec0 0x7f0bb0100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/135639069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb0100880 0x7f0bb0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/135639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb00ff0d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/135639069 >> 172.21.15.44:0/135639069 conn(0x7f0bb00fa430 msgr2=0x7f0bb00fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/135639069 shutdown_connections 2024-09-13T17:24:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.392+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/135639069 wait complete. 2024-09-13T17:24:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 Processor -- start 2024-09-13T17:24:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 -- start start 2024-09-13T17:24:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb0102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb00ffec0 0x7f0bb01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb0100880 0x7f0bb01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0101f40 con 0x7f0bb00fecd0 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb01020b0 con 0x7f0bb00ffec0 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bb4cd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019fe00 con 0x7f0bb0100880 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb0102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb0102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56004/0 (socket says 172.21.15.44:56004) 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 -- 172.21.15.44:0/3984666296 learned_addr learned my addr 172.21.15.44:0/3984666296 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0baed9d640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb0100880 0x7f0bb01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0badd9b640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb00ffec0 0x7f0bb01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 -- 172.21.15.44:0/3984666296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb0100880 msgr2=0x7f0bb01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb0100880 0x7f0bb01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 -- 172.21.15.44:0/3984666296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb00ffec0 msgr2=0x7f0bb01012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb00ffec0 0x7f0bb01012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0bae59c640 1 -- 172.21.15.44:0/3984666296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba0009560 con 0x7f0bb00fecd0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.393+0000 7f0badd9b640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb00ffec0 0x7f0bb01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.394+0000 7f0bae59c640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb0102c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009230 tx=0x7f0ba0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.394+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0032e10 con 0x7f0bb00fecd0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.394+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb01a0050 con 0x7f0bb00fecd0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.394+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ba00046c0 con 0x7f0bb00fecd0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.394+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0003850 con 0x7f0bb00fecd0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.394+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb01a0550 con 0x7f0bb00fecd0 2024-09-13T17:24:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.395+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb010a440 con 0x7f0bb00fecd0 2024-09-13T17:24:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.396+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0ba0004830 con 0x7f0bb00fecd0 2024-09-13T17:24:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.397+0000 7f0b9f7fe640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0b980a1090 0x7f0b980a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.397+0000 7f0badd9b640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0b980a1090 0x7f0b980a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.397+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ba00ed510 con 0x7f0bb00fecd0 2024-09-13T17:24:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.397+0000 7f0badd9b640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0b980a1090 0x7f0b980a3550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0bb01029d0 tx=0x7f0ba4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.400+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0ba00b70a0 con 0x7f0bb00fecd0 2024-09-13T17:24:08.524 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 systemd[1]: Starting Ceph mds.i for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:24:08.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.608+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bb010a5d0 con 0x7f0bb00fecd0 2024-09-13T17:24:08.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.609+0000 7f0b9f7fe640 1 -- 172.21.15.44:0/3984666296 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6553 (secure 0 0 0) 0x7f0ba00ba6c0 con 0x7f0bb00fecd0 2024-09-13T17:24:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:08.611 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-09-13T17:24:08.611 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-09-13T17:24:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0b980a1090 msgr2=0x7f0b980a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0b980a1090 0x7f0b980a3550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0bb01029d0 tx=0x7f0ba4005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 msgr2=0x7f0bb0102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb0102c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009230 tx=0x7f0ba0032c20 comp rx=0 tx=0).stop 2024-09-13T17:24:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 shutdown_connections 2024-09-13T17:24:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0b980a1090 0x7f0b980a3550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb0100880 0x7f0bb01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb00ffec0 0x7f0bb01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 --2- 172.21.15.44:0/3984666296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bb00fecd0 0x7f0bb0102c60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:08.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 >> 172.21.15.44:0/3984666296 conn(0x7f0bb00fa430 msgr2=0x7f0bb0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:08.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 shutdown_connections 2024-09-13T17:24:08.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:08.611+0000 7f0bb4cd7640 1 -- 172.21.15.44:0/3984666296 wait complete. 2024-09-13T17:24:08.803 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:08 smithi100 ceph-mon[35055]: Deploying daemon mds.i on smithi100 2024-09-13T17:24:08.803 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:08 smithi100 ceph-mon[35055]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:08.803 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 podman[65887]: 2024-09-13 17:24:08.458406717 +0000 UTC m=+0.015843504 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:08.803 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 podman[65887]: 2024-09-13 17:24:08.577876374 +0000 UTC m=+0.135313149 container create 54717c131dc7e21ae2d9da18bbe59948cb62ce9710f10de4fa82b844367a91d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:24:08.803 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 podman[65887]: 2024-09-13 17:24:08.802591428 +0000 UTC m=+0.360028202 container init 54717c131dc7e21ae2d9da18bbe59948cb62ce9710f10de4fa82b844367a91d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T17:24:08.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:08 smithi044 ceph-mon[31175]: Deploying daemon mds.i on smithi100 2024-09-13T17:24:08.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:08 smithi044 ceph-mon[31175]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:08 smithi070 ceph-mon[34502]: Deploying daemon mds.i on smithi100 2024-09-13T17:24:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:08 smithi070 ceph-mon[34502]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:09.274 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 podman[65887]: 2024-09-13 17:24:08.805384304 +0000 UTC m=+0.362821079 container start 54717c131dc7e21ae2d9da18bbe59948cb62ce9710f10de4fa82b844367a91d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:24:09.274 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 ceph-mds[65906]: -- 172.21.15.100:0/1148223268 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c05cb0d00 con 0x560c05c85400 2024-09-13T17:24:09.274 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: starting mds.i at 2024-09-13T17:24:09.274 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 bash[65887]: 54717c131dc7e21ae2d9da18bbe59948cb62ce9710f10de4fa82b844367a91d5 2024-09-13T17:24:09.274 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:08 smithi100 systemd[1]: Started Ceph mds.i for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3984666296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:09.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:09 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3984666296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:09 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3984666296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:09 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:10.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:10.651 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:10 smithi044 ceph-mon[31175]: Deploying daemon mds.g on smithi044 2024-09-13T17:24:10.651 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:10 smithi044 ceph-mon[31175]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:10.651 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:10 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] up:boot 2024-09-13T17:24:10.651 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:10 smithi044 ceph-mon[31175]: fsmap 9 up:standby 2024-09-13T17:24:10.651 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:10 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.837+0000 7f49bf72c640 1 -- 172.21.15.44:0/3405489610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b80fe610 msgr2=0x7f49b8100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.837+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3405489610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b80fe610 0x7f49b8100a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009900 tx=0x7f49b402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.837+0000 7f49bf72c640 1 -- 172.21.15.44:0/3405489610 shutdown_connections 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.837+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3405489610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b8100f60 0x7f49b81033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3405489610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49b8103910 0x7f49b8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3405489610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b80fe610 0x7f49b8100a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- 172.21.15.44:0/3405489610 >> 172.21.15.44:0/3405489610 conn(0x7f49b80fa400 msgr2=0x7f49b80fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- 172.21.15.44:0/3405489610 shutdown_connections 2024-09-13T17:24:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- 172.21.15.44:0/3405489610 wait complete. 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 Processor -- start 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- start start 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b80fe610 0x7f49b819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 0x7f49b819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49b8103910 0x7f49b81a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81a3dc0 con 0x7f49b8100f60 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81a3f30 con 0x7f49b8103910 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.838+0000 7f49bf72c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81a40f0 con 0x7f49b80fe610 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 0x7f49b819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 0x7f49b819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37848/0 (socket says 172.21.15.44:37848) 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 -- 172.21.15.44:0/3140171108 learned_addr learned my addr 172.21.15.44:0/3140171108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bd4c8640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b80fe610 0x7f49b819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bdcc9640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49b8103910 0x7f49b81a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 -- 172.21.15.44:0/3140171108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b80fe610 msgr2=0x7f49b819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b80fe610 0x7f49b819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 -- 172.21.15.44:0/3140171108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49b8103910 msgr2=0x7f49b81a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49b8103910 0x7f49b81a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 -- 172.21.15.44:0/3140171108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b4009560 con 0x7f49b8100f60 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bccc7640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 0x7f49b819c810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00e9f0 tx=0x7f49ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac009720 con 0x7f49b8100f60 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49ac00cd80 con 0x7f49b8100f60 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b81a43d0 con 0x7f49b8100f60 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac010430 con 0x7f49b8100f60 2024-09-13T17:24:10.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.839+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b81a48e0 con 0x7f49b8100f60 2024-09-13T17:24:10.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.840+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b80fbef0 con 0x7f49b8100f60 2024-09-13T17:24:10.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.842+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f49ac004f90 con 0x7f49b8100f60 2024-09-13T17:24:10.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.842+0000 7f49aa7fc640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f49a40a1110 0x7f49a40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:10.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.842+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f49ac014070 con 0x7f49b8100f60 2024-09-13T17:24:10.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.842+0000 7f49bd4c8640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f49a40a1110 0x7f49a40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.843+0000 7f49bd4c8640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f49a40a1110 0x7f49a40a35d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f49b4000c00 tx=0x7f49b4005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:10.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:10.850+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f49ac0966e0 con 0x7f49b8100f60 2024-09-13T17:24:10.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:10 smithi070 ceph-mon[34502]: Deploying daemon mds.g on smithi044 2024-09-13T17:24:10.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:10 smithi070 ceph-mon[34502]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:10.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:10 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] up:boot 2024-09-13T17:24:10.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:10 smithi070 ceph-mon[34502]: fsmap 9 up:standby 2024-09-13T17:24:10.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:10 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:24:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:10 smithi100 ceph-mon[35055]: Deploying daemon mds.g on smithi044 2024-09-13T17:24:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:10 smithi100 ceph-mon[35055]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:10 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] up:boot 2024-09-13T17:24:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:10 smithi100 ceph-mon[35055]: fsmap 9 up:standby 2024-09-13T17:24:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:10 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:24:11.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.079+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49b8111b70 con 0x7f49b8100f60 2024-09-13T17:24:11.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.080+0000 7f49aa7fc640 1 -- 172.21.15.44:0/3140171108 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7315 (secure 0 0 0) 0x7f49ac099d00 con 0x7f49b8100f60 2024-09-13T17:24:11.086 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:11.086 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 10 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f49a40a1110 msgr2=0x7f49a40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f49a40a1110 0x7f49a40a35d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f49b4000c00 tx=0x7f49b4005a40 comp rx=0 tx=0).stop 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 msgr2=0x7f49b819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 0x7f49b819c810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00e9f0 tx=0x7f49ac00edf0 comp rx=0 tx=0).stop 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 shutdown_connections 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f49a40a1110 0x7f49a40a35d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49b8103910 0x7f49b81a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b8100f60 0x7f49b819c810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 --2- 172.21.15.44:0/3140171108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b80fe610 0x7f49b819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 >> 172.21.15.44:0/3140171108 conn(0x7f49b80fa400 msgr2=0x7f49b8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 shutdown_connections 2024-09-13T17:24:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:11.082+0000 7f49bf72c640 1 -- 172.21.15.44:0/3140171108 wait complete. 2024-09-13T17:24:11.240 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 podman[71988]: 2024-09-13 17:24:11.107761461 +0000 UTC m=+0.019830709 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:11.240 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 podman[71988]: 2024-09-13 17:24:11.239608344 +0000 UTC m=+0.151677623 container create 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:24:11.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:11 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3140171108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:11 smithi044 ceph-mon[31175]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 podman[71988]: 2024-09-13 17:24:11.509214717 +0000 UTC m=+0.421283954 container init 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 podman[71988]: 2024-09-13 17:24:11.512181829 +0000 UTC m=+0.424251066 container start 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 ceph-mds[72008]: -- 172.21.15.44:0/1509253695 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56275bc5ed00 con 0x56275bc32800 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g[72004]: starting mds.g at 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 bash[71988]: 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a 2024-09-13T17:24:11.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:24:11 smithi044 systemd[1]: Started Ceph mds.g for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:11.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:11 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3140171108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:11.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:11 smithi070 ceph-mon[34502]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:11 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3140171108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:12.027 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:11 smithi100 ceph-mon[35055]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.44:6838/4189498967,v1:172.21.15.44:6839/4189498967] up:boot 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: fsmap 10 up:standby 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:12.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:12.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:12.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:12.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:12 smithi044 ceph-mon[31175]: Deploying daemon mds.a on smithi044 2024-09-13T17:24:12.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.44:6838/4189498967,v1:172.21.15.44:6839/4189498967] up:boot 2024-09-13T17:24:12.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: fsmap 10 up:standby 2024-09-13T17:24:12.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T17:24:12.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:12.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:12.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:12.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:12.949 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:12.949 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:12.949 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:12 smithi070 ceph-mon[34502]: Deploying daemon mds.a on smithi044 2024-09-13T17:24:13.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.44:6838/4189498967,v1:172.21.15.44:6839/4189498967] up:boot 2024-09-13T17:24:13.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: fsmap 10 up:standby 2024-09-13T17:24:13.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:13.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:12 smithi100 ceph-mon[35055]: Deploying daemon mds.a on smithi044 2024-09-13T17:24:13.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:13.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 -- 172.21.15.44:0/1424241851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c105a80 msgr2=0x7f765c107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 --2- 172.21.15.44:0/1424241851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c105a80 0x7f765c107ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7654009ca0 tx=0x7f765402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 -- 172.21.15.44:0/1424241851 shutdown_connections 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 --2- 172.21.15.44:0/1424241851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c103150 0x7f765c105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 --2- 172.21.15.44:0/1424241851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c105a80 0x7f765c107ef0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 --2- 172.21.15.44:0/1424241851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f765c100820 0x7f765c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 -- 172.21.15.44:0/1424241851 >> 172.21.15.44:0/1424241851 conn(0x7f765c0fa420 msgr2=0x7f765c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 -- 172.21.15.44:0/1424241851 shutdown_connections 2024-09-13T17:24:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.218+0000 7f7660f7d640 1 -- 172.21.15.44:0/1424241851 wait complete. 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.219+0000 7f7660f7d640 1 Processor -- start 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 -- start start 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f765c100820 0x7f765c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 0x7f765c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c105a80 0x7f765c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c19d890 con 0x7f765c103150 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c19da00 con 0x7f765c100820 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7660f7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c19dbc0 con 0x7f765c105a80 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 0x7f765c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f765a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f765c100820 0x7f765c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 0x7f765c1962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37892/0 (socket says 172.21.15.44:37892) 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 -- 172.21.15.44:0/2523021796 learned_addr learned my addr 172.21.15.44:0/2523021796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f765ad9d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c105a80 0x7f765c19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 -- 172.21.15.44:0/2523021796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c105a80 msgr2=0x7f765c19d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c105a80 0x7f765c19d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 -- 172.21.15.44:0/2523021796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f765c100820 msgr2=0x7f765c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f765c100820 0x7f765c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f7659d9b640 1 -- 172.21.15.44:0/2523021796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654009900 con 0x7f765c103150 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.220+0000 7f765ad9d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c105a80 0x7f765c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.221+0000 7f7659d9b640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 0x7f765c1962f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f765000e9f0 tx=0x7f765000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.221+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500096c0 con 0x7f765c103150 2024-09-13T17:24:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.221+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f765000ccc0 con 0x7f765c103150 2024-09-13T17:24:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.221+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c19dea0 con 0x7f765c103150 2024-09-13T17:24:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.221+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650004e60 con 0x7f765c103150 2024-09-13T17:24:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.221+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c19e2c0 con 0x7f765c103150 2024-09-13T17:24:13.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.222+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c0fbf10 con 0x7f765c103150 2024-09-13T17:24:13.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.223+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7650005020 con 0x7f765c103150 2024-09-13T17:24:13.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.224+0000 7f764b7fe640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f76440a1090 0x7f76440a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:13.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.224+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7650014070 con 0x7f765c103150 2024-09-13T17:24:13.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.224+0000 7f765a59c640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f76440a1090 0x7f76440a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:13.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.224+0000 7f765a59c640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f76440a1090 0x7f76440a3550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f765c197280 tx=0x7f764c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:13.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.228+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7650096490 con 0x7f765c103150 2024-09-13T17:24:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.441+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765c02c3e0 con 0x7f765c103150 2024-09-13T17:24:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.442+0000 7f764b7fe640 1 -- 172.21.15.44:0/2523021796 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8073 (secure 0 0 0) 0x7f76500983c0 con 0x7f765c103150 2024-09-13T17:24:13.444 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:13.444 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T17:24:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 11 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f76440a1090 msgr2=0x7f76440a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f76440a1090 0x7f76440a3550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f765c197280 tx=0x7f764c00b410 comp rx=0 tx=0).stop 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 msgr2=0x7f765c1962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 0x7f765c1962f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f765000e9f0 tx=0x7f765000eeb0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 shutdown_connections 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f76440a1090 0x7f76440a3550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f765c105a80 0x7f765c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f765c103150 0x7f765c1962f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 --2- 172.21.15.44:0/2523021796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f765c100820 0x7f765c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 >> 172.21.15.44:0/2523021796 conn(0x7f765c0fa420 msgr2=0x7f765c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 shutdown_connections 2024-09-13T17:24:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:13.444+0000 7f7660f7d640 1 -- 172.21.15.44:0/2523021796 wait complete. 2024-09-13T17:24:13.718 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2523021796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:13.718 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:13 smithi044 ceph-mon[31175]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:13.719 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:13 smithi044 systemd[1]: Starting Ceph mds.a for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:24:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2523021796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:13 smithi070 ceph-mon[34502]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2523021796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:13 smithi100 ceph-mon[35055]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:14.078 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:13 smithi044 podman[72331]: 2024-09-13 17:24:13.834817836 +0000 UTC m=+0.016622102 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:14.078 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:13 smithi044 podman[72331]: 2024-09-13 17:24:13.955671504 +0000 UTC m=+0.137475758 container create 5c56a401760415499910d78f9320cc1e8be3ec4aa33e1e64deacabd1691f2206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-09-13T17:24:14.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:14 smithi044 podman[72331]: 2024-09-13 17:24:14.200078136 +0000 UTC m=+0.381882392 container init 5c56a401760415499910d78f9320cc1e8be3ec4aa33e1e64deacabd1691f2206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-13T17:24:14.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:14 smithi044 podman[72331]: 2024-09-13 17:24:14.203088162 +0000 UTC m=+0.384892418 container start 5c56a401760415499910d78f9320cc1e8be3ec4aa33e1e64deacabd1691f2206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True) 2024-09-13T17:24:14.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:14 smithi044 ceph-mds[72350]: -- 172.21.15.44:0/4212272921 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559187424d00 con 0x5591873f8800 2024-09-13T17:24:14.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a[72346]: starting mds.a at 2024-09-13T17:24:14.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:14 smithi044 bash[72331]: 5c56a401760415499910d78f9320cc1e8be3ec4aa33e1e64deacabd1691f2206 2024-09-13T17:24:14.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:24:14 smithi044 systemd[1]: Started Ceph mds.a for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:15.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 -- 172.21.15.44:0/2685339674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440ffea0 msgr2=0x7fc544100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 --2- 172.21.15.44:0/2685339674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440ffea0 0x7fc544100320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc538009900 tx=0x7fc53802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 -- 172.21.15.44:0/2685339674 shutdown_connections 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 --2- 172.21.15.44:0/2685339674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc544100860 0x7fc5441070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 --2- 172.21.15.44:0/2685339674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440ffea0 0x7fc544100320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 --2- 172.21.15.44:0/2685339674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5440fecb0 0x7fc5440ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 -- 172.21.15.44:0/2685339674 >> 172.21.15.44:0/2685339674 conn(0x7fc5440fa430 msgr2=0x7fc5440fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 -- 172.21.15.44:0/2685339674 shutdown_connections 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 -- 172.21.15.44:0/2685339674 wait complete. 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.577+0000 7fc54afe2640 1 Processor -- start 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 -- start start 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 0x7fc544109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5440ffea0 0x7fc544107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc544100860 0x7fc5441080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544108850 con 0x7fc5440fecb0 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441089c0 con 0x7fc544100860 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54afe2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54419fd30 con 0x7fc5440ffea0 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 0x7fc544109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 0x7fc544109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37928/0 (socket says 172.21.15.44:37928) 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 -- 172.21.15.44:0/615550924 learned_addr learned my addr 172.21.15.44:0/615550924 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc54957f640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc544100860 0x7fc5441080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc543fff640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5440ffea0 0x7fc544107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 -- 172.21.15.44:0/615550924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5440ffea0 msgr2=0x7fc544107bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5440ffea0 0x7fc544107bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 -- 172.21.15.44:0/615550924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc544100860 msgr2=0x7fc5441080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc544100860 0x7fc5441080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 -- 172.21.15.44:0/615550924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538009560 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.578+0000 7fc548d7e640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 0x7fc544109570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d7a0 tx=0x7fc53400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.579+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534009c30 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.579+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc53400deb0 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.579+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54419ffe0 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.579+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534003d10 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.579+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5441a0410 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.580+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5441061b0 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.581+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc534003e70 con 0x7fc5440fecb0 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.582+0000 7fc541ffb640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc52c0a1110 0x7fc52c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.582+0000 7fc543fff640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc52c0a1110 0x7fc52c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:15.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.582+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5340cbdd0 con 0x7fc5440fecb0 2024-09-13T17:24:15.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.582+0000 7fc543fff640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc52c0a1110 0x7fc52c0a35d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc538005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:15.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.586+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc534095960 con 0x7fc5440fecb0 2024-09-13T17:24:15.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: Deploying daemon mds.b on smithi070 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.44:6840/1193858292,v1:172.21.15.44:6841/1193858292] up:boot 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: fsmap 11 up:standby 2024-09-13T17:24:15.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:15 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T17:24:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:15.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: Deploying daemon mds.b on smithi070 2024-09-13T17:24:15.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.44:6840/1193858292,v1:172.21.15.44:6841/1193858292] up:boot 2024-09-13T17:24:15.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: fsmap 11 up:standby 2024-09-13T17:24:15.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:15 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T17:24:15.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:15.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T17:24:15.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T17:24:15.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:15.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: Deploying daemon mds.b on smithi070 2024-09-13T17:24:15.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.44:6840/1193858292,v1:172.21.15.44:6841/1193858292] up:boot 2024-09-13T17:24:15.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: fsmap 11 up:standby 2024-09-13T17:24:15.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:15 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T17:24:15.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.797+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc54402c480 con 0x7fc5440fecb0 2024-09-13T17:24:15.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.798+0000 7fc541ffb640 1 -- 172.21.15.44:0/615550924 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8831 (secure 0 0 0) 0x7fc534098f80 con 0x7fc5440fecb0 2024-09-13T17:24:15.801 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:15.801 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T17:24:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 12 2024-09-13T17:24:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc52c0a1110 msgr2=0x7fc52c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc52c0a1110 0x7fc52c0a35d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc538005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 msgr2=0x7fc544109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 0x7fc544109570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d7a0 tx=0x7fc53400dc60 comp rx=0 tx=0).stop 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 shutdown_connections 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc52c0a1110 0x7fc52c0a35d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5440ffea0 0x7fc544107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc544100860 0x7fc5441080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 --2- 172.21.15.44:0/615550924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440fecb0 0x7fc544109570 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 >> 172.21.15.44:0/615550924 conn(0x7fc5440fa430 msgr2=0x7fc544103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 shutdown_connections 2024-09-13T17:24:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:15.800+0000 7fc54afe2640 1 -- 172.21.15.44:0/615550924 wait complete. 2024-09-13T17:24:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:16 smithi044 ceph-mon[31175]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:16.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:16 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/615550924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:16.662 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:16 smithi070 ceph-mon[34502]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:16.662 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:16 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/615550924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:16.663 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:16 smithi070 systemd[1]: Starting Ceph mds.b for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:24:16.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:16 smithi100 ceph-mon[35055]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:16.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:16 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/615550924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:16.915 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:16 smithi070 podman[64116]: 2024-09-13 17:24:16.563971526 +0000 UTC m=+0.015858854 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:24:16.915 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:16 smithi070 podman[64116]: 2024-09-13 17:24:16.768278818 +0000 UTC m=+0.220166194 container create e228f8664847cfb23602484052cbe9d36cd7f81ea50c3ccf1c55a56ac63c21d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-13T17:24:17.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:17 smithi070 podman[64116]: 2024-09-13 17:24:17.029291602 +0000 UTC m=+0.481178933 container init e228f8664847cfb23602484052cbe9d36cd7f81ea50c3ccf1c55a56ac63c21d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:24:17.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:17 smithi070 podman[64116]: 2024-09-13 17:24:17.032322408 +0000 UTC m=+0.484209736 container start e228f8664847cfb23602484052cbe9d36cd7f81ea50c3ccf1c55a56ac63c21d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-13T17:24:17.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:17 smithi070 ceph-mds[64135]: -- 172.21.15.70:0/1729162485 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c84f1a6d00 con 0x55c84f17b400 2024-09-13T17:24:17.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b[64131]: starting mds.b at 2024-09-13T17:24:17.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:17 smithi070 bash[64116]: e228f8664847cfb23602484052cbe9d36cd7f81ea50c3ccf1c55a56ac63c21d3 2024-09-13T17:24:17.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:24:17 smithi070 systemd[1]: Started Ceph mds.b for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:24:17.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:17.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 -- 172.21.15.44:0/2189267892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b980ffe90 msgr2=0x7f5b98100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 --2- 172.21.15.44:0/2189267892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b980ffe90 0x7f5b98100310 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009900 tx=0x7f5b9402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 -- 172.21.15.44:0/2189267892 shutdown_connections 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 --2- 172.21.15.44:0/2189267892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b98100850 0x7f5b981070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 --2- 172.21.15.44:0/2189267892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b980ffe90 0x7f5b98100310 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 --2- 172.21.15.44:0/2189267892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b980feca0 0x7f5b980ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 -- 172.21.15.44:0/2189267892 >> 172.21.15.44:0/2189267892 conn(0x7f5b980fa440 msgr2=0x7f5b980fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 -- 172.21.15.44:0/2189267892 shutdown_connections 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.920+0000 7f5ba0718640 1 -- 172.21.15.44:0/2189267892 wait complete. 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 Processor -- start 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 -- start start 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b980feca0 0x7f5b98195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b980ffe90 0x7f5b98196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 0x7f5b9819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9819d8a0 con 0x7f5b98100850 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9819da10 con 0x7f5b980ffe90 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5ba0718640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9819dbd0 con 0x7f5b980feca0 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 0x7f5b9819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 0x7f5b9819d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37946/0 (socket says 172.21.15.44:37946) 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 -- 172.21.15.44:0/847591533 learned_addr learned my addr 172.21.15.44:0/847591533 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9e4b4640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b980feca0 0x7f5b98195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9dcb3640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b980ffe90 0x7f5b98196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 -- 172.21.15.44:0/847591533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b980feca0 msgr2=0x7f5b98195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b980feca0 0x7f5b98195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 -- 172.21.15.44:0/847591533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b980ffe90 msgr2=0x7f5b98196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b980ffe90 0x7f5b98196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.921+0000 7f5b9ecb5640 1 -- 172.21.15.44:0/847591533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009560 con 0x7f5b98100850 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.922+0000 7f5b9ecb5640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 0x7f5b9819d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000eb10 tx=0x7f5b9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.922+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90004b20 con 0x7f5b98100850 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.922+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9819deb0 con 0x7f5b98100850 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.922+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b90004c80 con 0x7f5b98100850 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.922+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90005a10 con 0x7f5b98100850 2024-09-13T17:24:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.922+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9819e2d0 con 0x7f5b98100850 2024-09-13T17:24:17.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.923+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9810a460 con 0x7f5b98100850 2024-09-13T17:24:17.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.924+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5b90005020 con 0x7f5b98100850 2024-09-13T17:24:17.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.925+0000 7f5b8b7fe640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5b840a1090 0x7f5b840a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:17.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.925+0000 7f5b9e4b4640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5b840a1090 0x7f5b840a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:17.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.925+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b90014070 con 0x7f5b98100850 2024-09-13T17:24:17.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.925+0000 7f5b9e4b4640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5b840a1090 0x7f5b840a3550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b98197290 tx=0x7f5b8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:17.929+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5b90096610 con 0x7f5b98100850 2024-09-13T17:24:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.141+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b9810a5f0 con 0x7f5b98100850 2024-09-13T17:24:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.142+0000 7f5b8b7fe640 1 -- 172.21.15.44:0/847591533 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9589 (secure 0 0 0) 0x7f5b90098540 con 0x7f5b98100850 2024-09-13T17:24:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:18.145 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T17:24:18.145 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 13 2024-09-13T17:24:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5b840a1090 msgr2=0x7f5b840a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5b840a1090 0x7f5b840a3550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b98197290 tx=0x7f5b8c005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 msgr2=0x7f5b9819d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 0x7f5b9819d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000eb10 tx=0x7f5b9000c2d0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 shutdown_connections 2024-09-13T17:24:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5b840a1090 0x7f5b840a3550 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b98100850 0x7f5b9819d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b980ffe90 0x7f5b98196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 --2- 172.21.15.44:0/847591533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b980feca0 0x7f5b98195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 >> 172.21.15.44:0/847591533 conn(0x7f5b980fa440 msgr2=0x7f5b98067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 shutdown_connections 2024-09-13T17:24:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.145+0000 7f5ba0718640 1 -- 172.21.15.44:0/847591533 wait complete. 2024-09-13T17:24:18.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 -- 172.21.15.44:0/2882429469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0fecd0 msgr2=0x7ffa4c0ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 --2- 172.21.15.44:0/2882429469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0fecd0 0x7ffa4c0ff0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009900 tx=0x7ffa4002a410 comp rx=0 tx=0).stop 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 -- 172.21.15.44:0/2882429469 shutdown_connections 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 --2- 172.21.15.44:0/2882429469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa4c100880 0x7ffa4c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 --2- 172.21.15.44:0/2882429469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 --2- 172.21.15.44:0/2882429469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0fecd0 0x7ffa4c0ff0d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 -- 172.21.15.44:0/2882429469 >> 172.21.15.44:0/2882429469 conn(0x7ffa4c0fa430 msgr2=0x7ffa4c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 -- 172.21.15.44:0/2882429469 shutdown_connections 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.260+0000 7ffa531bc640 1 -- 172.21.15.44:0/2882429469 wait complete. 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 Processor -- start 2024-09-13T17:24:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 -- start start 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa4c0fecd0 0x7ffa4c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa4c100880 0x7ffa4c19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19d7f0 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19d960 con 0x7ffa4c0fecd0 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa531bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19db20 con 0x7ffa4c100880 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37956/0 (socket says 172.21.15.44:37956) 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 -- 172.21.15.44:0/776073918 learned_addr learned my addr 172.21.15.44:0/776073918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa50f58640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa4c0fecd0 0x7ffa4c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa51759640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa4c100880 0x7ffa4c19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 -- 172.21.15.44:0/776073918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa4c100880 msgr2=0x7ffa4c19d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa4c100880 0x7ffa4c19d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 -- 172.21.15.44:0/776073918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa4c0fecd0 msgr2=0x7ffa4c195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa4c0fecd0 0x7ffa4c195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.261+0000 7ffa4bfff640 1 -- 172.21.15.44:0/776073918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40009560 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.262+0000 7ffa4bfff640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c196250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c920 tx=0x7ffa3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.262+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38007c70 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.262+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa38007dd0 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.262+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c19de00 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.262+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38018730 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.262+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c19e280 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.263+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c106180 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.264+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ffa3800f450 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.265+0000 7ffa49ffb640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.265+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffa380cc570 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.265+0000 7ffa50f58640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.265+0000 7ffa50f58640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c1971e0 tx=0x7ffa40000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.269+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ffa38096110 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6838/2731404816,v1:172.21.15.70:6839/2731404816] up:boot 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: fsmap 12 up:standby 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:24:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:18.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:18 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/847591533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:18.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.454+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa4c105bf0 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.455+0000 7ffa49ffb640 1 -- 172.21.15.44:0/776073918 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18362 (secure 0 0 0) 0x7ffa38098040 con 0x7ffa4c0ffec0 2024-09-13T17:24:18.457 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:18.457 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":80,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:23:34.441805+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ffa3c0a1090 msgr2=0x7ffa3c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c1971e0 tx=0x7ffa40000bc0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 msgr2=0x7ffa4c196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c196250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c920 tx=0x7ffa3800cde0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 shutdown_connections 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa4c100880 0x7ffa4c19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa4c0ffec0 0x7ffa4c196250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 --2- 172.21.15.44:0/776073918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa4c0fecd0 0x7ffa4c195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 >> 172.21.15.44:0/776073918 conn(0x7ffa4c0fa430 msgr2=0x7ffa4c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:18.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 shutdown_connections 2024-09-13T17:24:18.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.458+0000 7ffa531bc640 1 -- 172.21.15.44:0/776073918 wait complete. 2024-09-13T17:24:18.469 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6838/2731404816,v1:172.21.15.70:6839/2731404816] up:boot 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: fsmap 12 up:standby 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:18.470 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:18 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/847591533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:18.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:24:18.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6838/2731404816,v1:172.21.15.70:6839/2731404816] up:boot 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: fsmap 12 up:standby 2024-09-13T17:24:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:24:18.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:18.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:18 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/847591533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:18.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 -- 172.21.15.44:0/2214242683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 msgr2=0x7f1e9c105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 --2- 172.21.15.44:0/2214242683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c105590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009900 tx=0x7f1e8402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 -- 172.21.15.44:0/2214242683 shutdown_connections 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 --2- 172.21.15.44:0/2214242683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c105ad0 0x7f1e9c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 --2- 172.21.15.44:0/2214242683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c105590 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 --2- 172.21.15.44:0/2214242683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9c100870 0x7f1e9c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 -- 172.21.15.44:0/2214242683 >> 172.21.15.44:0/2214242683 conn(0x7f1e9c0fa3f0 msgr2=0x7f1e9c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 -- 172.21.15.44:0/2214242683 shutdown_connections 2024-09-13T17:24:18.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.593+0000 7f1ea095a640 1 -- 172.21.15.44:0/2214242683 wait complete. 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 Processor -- start 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 -- start start 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c100870 0x7f1e9c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9c105ad0 0x7f1e9c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19d910 con 0x7f1e9c1031a0 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19da80 con 0x7f1e9c100870 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1ea095a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19dc40 con 0x7f1e9c105ad0 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1e93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1e93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37974/0 (socket says 172.21.15.44:37974) 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1e93fff640 1 -- 172.21.15.44:0/3472422147 learned_addr learned my addr 172.21.15.44:0/3472422147 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1e9a59c640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c100870 0x7f1e9c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.594+0000 7f1e9ad9d640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9c105ad0 0x7f1e9c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e93fff640 1 -- 172.21.15.44:0/3472422147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9c105ad0 msgr2=0x7f1e9c19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e93fff640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9c105ad0 0x7f1e9c19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e93fff640 1 -- 172.21.15.44:0/3472422147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c100870 msgr2=0x7f1e9c195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e93fff640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c100870 0x7f1e9c195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e93fff640 1 -- 172.21.15.44:0/3472422147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84009560 con 0x7f1e9c1031a0 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e9a59c640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c100870 0x7f1e9c195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e93fff640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c196360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e8402a930 tx=0x7f1e840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840044d0 con 0x7f1e9c1031a0 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e84003a80 con 0x7f1e9c1031a0 2024-09-13T17:24:18.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c19dec0 con 0x7f1e9c1031a0 2024-09-13T17:24:18.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8403cb50 con 0x7f1e9c1031a0 2024-09-13T17:24:18.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.595+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c19e2e0 con 0x7f1e9c1031a0 2024-09-13T17:24:18.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.596+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c0fbee0 con 0x7f1e9c1031a0 2024-09-13T17:24:18.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.598+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1e84003bf0 con 0x7f1e9c1031a0 2024-09-13T17:24:18.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.599+0000 7f1e937fe640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e880a1090 0x7f1e880a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.599+0000 7f1e9a59c640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e880a1090 0x7f1e880a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.599+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1e840edb50 con 0x7f1e9c1031a0 2024-09-13T17:24:18.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.600+0000 7f1e9a59c640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e880a1090 0x7f1e880a3550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c005d00 tx=0x7f1e8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:18.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.602+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1e840b7790 con 0x7f1e9c1031a0 2024-09-13T17:24:18.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.799+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e9c02c480 con 0x7f1e9c1031a0 2024-09-13T17:24:18.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.800+0000 7f1e937fe640 1 -- 172.21.15.44:0/3472422147 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e840badb0 con 0x7f1e9c1031a0 2024-09-13T17:24:18.802 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:24:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.801+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e880a1090 msgr2=0x7f1e880a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.801+0000 7f1ea095a640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e880a1090 0x7f1e880a3550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c005d00 tx=0x7f1e8c00b410 comp rx=0 tx=0).stop 2024-09-13T17:24:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.801+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 msgr2=0x7f1e9c196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.801+0000 7f1ea095a640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c196360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e8402a930 tx=0x7f1e840048f0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 shutdown_connections 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e880a1090 0x7f1e880a3550 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9c105ad0 0x7f1e9c19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e9c1031a0 0x7f1e9c196360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 --2- 172.21.15.44:0/3472422147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e9c100870 0x7f1e9c195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 >> 172.21.15.44:0/3472422147 conn(0x7f1e9c0fa3f0 msgr2=0x7f1e9c1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 shutdown_connections 2024-09-13T17:24:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.802+0000 7f1ea095a640 1 -- 172.21.15.44:0/3472422147 wait complete. 2024-09-13T17:24:18.814 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:24:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 -- 172.21.15.44:0/3173848616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588100850 msgr2=0x7ff588102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/3173848616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588100850 0x7ff588102c40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff578009900 tx=0x7ff57802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 -- 172.21.15.44:0/3173848616 shutdown_connections 2024-09-13T17:24:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/3173848616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff588103180 0x7ff588105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/3173848616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff588105ab0 0x7ff588107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/3173848616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588100850 0x7ff588102c40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 -- 172.21.15.44:0/3173848616 >> 172.21.15.44:0/3173848616 conn(0x7ff5880fa430 msgr2=0x7ff5880fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 -- 172.21.15.44:0/3173848616 shutdown_connections 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 -- 172.21.15.44:0/3173848616 wait complete. 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.915+0000 7ff58d2b8640 1 Processor -- start 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 -- start start 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff588100850 0x7ff588195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 0x7ff588196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff588105ab0 0x7ff58819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58819d8b0 con 0x7ff588103180 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58819da20 con 0x7ff588105ab0 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff58d2b8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58819dbe0 con 0x7ff588100850 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 0x7ff588196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 0x7ff588196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37986/0 (socket says 172.21.15.44:37986) 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 -- 172.21.15.44:0/2966503942 learned_addr learned my addr 172.21.15.44:0/2966503942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff586ffd640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff588100850 0x7ff588195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5877fe640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff588105ab0 0x7ff58819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 -- 172.21.15.44:0/2966503942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff588100850 msgr2=0x7ff588195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff588100850 0x7ff588195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 -- 172.21.15.44:0/2966503942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff588105ab0 msgr2=0x7ff58819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff588105ab0 0x7ff58819d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.916+0000 7ff5867fc640 1 -- 172.21.15.44:0/2966503942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578009560 con 0x7ff588103180 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.917+0000 7ff5867fc640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 0x7ff588196300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e9f0 tx=0x7ff57c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.917+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0096c0 con 0x7ff588103180 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.917+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff57c00ccc0 con 0x7ff588103180 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.917+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58819dec0 con 0x7ff588103180 2024-09-13T17:24:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.917+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004e60 con 0x7ff588103180 2024-09-13T17:24:18.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.917+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58819e2f0 con 0x7ff588103180 2024-09-13T17:24:18.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.918+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5880fbf20 con 0x7ff588103180 2024-09-13T17:24:18.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.919+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff57c005020 con 0x7ff588103180 2024-09-13T17:24:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.920+0000 7ff56ffff640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5700a1090 0x7ff5700a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.920+0000 7ff586ffd640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5700a1090 0x7ff5700a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.920+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff57c014070 con 0x7ff588103180 2024-09-13T17:24:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.920+0000 7ff586ffd640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5700a1090 0x7ff5700a3550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff57802aa00 tx=0x7ff578005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:18.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:18.924+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff57c096420 con 0x7ff588103180 2024-09-13T17:24:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.100+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff5880fb990 con 0x7ff588103180 2024-09-13T17:24:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.100+0000 7ff56ffff640 1 -- 172.21.15.44:0/2966503942 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff57c099a40 con 0x7ff588103180 2024-09-13T17:24:19.102 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:19.103 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T17:24:19.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:24:19.103 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T17:24:19.103 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T17:24:19.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:24:19.103 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T17:24:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5700a1090 msgr2=0x7ff5700a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5700a1090 0x7ff5700a3550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff57802aa00 tx=0x7ff578005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 msgr2=0x7ff588196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 0x7ff588196300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e9f0 tx=0x7ff57c00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 shutdown_connections 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5700a1090 0x7ff5700a3550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff588105ab0 0x7ff58819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff588103180 0x7ff588196300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 --2- 172.21.15.44:0/2966503942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff588100850 0x7ff588195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 >> 172.21.15.44:0/2966503942 conn(0x7ff5880fa430 msgr2=0x7ff5881011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 shutdown_connections 2024-09-13T17:24:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.102+0000 7ff58d2b8640 1 -- 172.21.15.44:0/2966503942 wait complete. 2024-09-13T17:24:19.117 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:24:19.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 -- 172.21.15.44:0/1376615496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87180fec60 msgr2=0x7f87180ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 --2- 172.21.15.44:0/1376615496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87180fec60 0x7f87180ff060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8714009900 tx=0x7f871402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 -- 172.21.15.44:0/1376615496 shutdown_connections 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 --2- 172.21.15.44:0/1376615496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8718100810 0x7f87181070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 --2- 172.21.15.44:0/1376615496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180ffe50 0x7f87181002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 --2- 172.21.15.44:0/1376615496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87180fec60 0x7f87180ff060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 -- 172.21.15.44:0/1376615496 >> 172.21.15.44:0/1376615496 conn(0x7f87180fa3e0 msgr2=0x7f87180fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 -- 172.21.15.44:0/1376615496 shutdown_connections 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 -- 172.21.15.44:0/1376615496 wait complete. 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.223+0000 7f872024f640 1 Processor -- start 2024-09-13T17:24:19.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 -- start start 2024-09-13T17:24:19.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180fec60 0x7f8718102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:19.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87180ffe50 0x7f8718101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:19.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 0x7f8718101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:19.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8718101ed0 con 0x7f8718100810 2024-09-13T17:24:19.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8718102040 con 0x7f87180fec60 2024-09-13T17:24:19.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f872024f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871819fd10 con 0x7f87180ffe50 2024-09-13T17:24:19.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871e7ec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 0x7f8718101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:19.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871e7ec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 0x7f8718101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38004/0 (socket says 172.21.15.44:38004) 2024-09-13T17:24:19.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180fec60 0x7f8718102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:19.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871e7ec640 1 -- 172.21.15.44:0/272600524 learned_addr learned my addr 172.21.15.44:0/272600524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:19.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871d7ea640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87180ffe50 0x7f8718101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:19.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 -- 172.21.15.44:0/272600524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87180ffe50 msgr2=0x7f8718101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:19.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87180ffe50 0x7f8718101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 -- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 msgr2=0x7f8718101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:19.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 0x7f8718101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:19.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 -- 172.21.15.44:0/272600524 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714009560 con 0x7f87180fec60 2024-09-13T17:24:19.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871e7ec640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 0x7f8718101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:24:19.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.224+0000 7f871dfeb640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180fec60 0x7f8718102bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8714009230 tx=0x7f87140042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:19.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.225+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714031d70 con 0x7f87180fec60 2024-09-13T17:24:19.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.225+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871819ff60 con 0x7f87180fec60 2024-09-13T17:24:19.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.225+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8714005490 con 0x7f87180fec60 2024-09-13T17:24:19.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.225+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714003990 con 0x7f87180fec60 2024-09-13T17:24:19.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.225+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87181a0330 con 0x7f87180fec60 2024-09-13T17:24:19.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.225+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871810a3d0 con 0x7f87180fec60 2024-09-13T17:24:19.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.227+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8714003af0 con 0x7f87180fec60 2024-09-13T17:24:19.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.227+0000 7f870affd640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87040a1090 0x7f87040a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:19.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.228+0000 7f871d7ea640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87040a1090 0x7f87040a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:19.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.228+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87140ed690 con 0x7f87180fec60 2024-09-13T17:24:19.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.228+0000 7f871d7ea640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87040a1090 0x7f87040a3550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8718102960 tx=0x7f870c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:19.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.231+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f87140b72d0 con 0x7f87180fec60 2024-09-13T17:24:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:19.420+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87181a0520 con 0x7f87180fec60 2024-09-13T17:24:19.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/776073918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3472422147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:24:19.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:19 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2966503942' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T17:24:19.635 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/776073918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:19.635 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3472422147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:24:19.636 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:19 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2966503942' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T17:24:19.639 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/776073918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:19.639 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.639 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.639 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.639 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.639 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3472422147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:24:19.640 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:19 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2966503942' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T17:24:20.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.362+0000 7f870affd640 1 -- 172.21.15.44:0/272600524 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87140ba8f0 con 0x7f87180fec60 2024-09-13T17:24:20.365 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T17:24:20.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87040a1090 msgr2=0x7f87040a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:20.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87040a1090 0x7f87040a3550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8718102960 tx=0x7f870c005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:20.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180fec60 msgr2=0x7f8718102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:20.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180fec60 0x7f8718102bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8714009230 tx=0x7f87140042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 shutdown_connections 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87040a1090 0x7f87040a3550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87180ffe50 0x7f8718101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8718100810 0x7f8718101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 --2- 172.21.15.44:0/272600524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87180fec60 0x7f8718102bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 >> 172.21.15.44:0/272600524 conn(0x7f87180fa3e0 msgr2=0x7f8718067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 shutdown_connections 2024-09-13T17:24:20.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.367+0000 7f872024f640 1 -- 172.21.15.44:0/272600524 wait complete. 2024-09-13T17:24:20.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T17:24:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 -- 172.21.15.44:0/3015191073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 msgr2=0x7f96801013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 --2- 172.21.15.44:0/3015191073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f96801013f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f966c009900 tx=0x7f966c02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 -- 172.21.15.44:0/3015191073 shutdown_connections 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 --2- 172.21.15.44:0/3015191073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9680101930 0x7f96801081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 --2- 172.21.15.44:0/3015191073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f96801013f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 --2- 172.21.15.44:0/3015191073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96800ffd80 0x7f9680100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 -- 172.21.15.44:0/3015191073 >> 172.21.15.44:0/3015191073 conn(0x7f96800fb4e0 msgr2=0x7f96800fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 -- 172.21.15.44:0/3015191073 shutdown_connections 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.515+0000 7f9686e80640 1 -- 172.21.15.44:0/3015191073 wait complete. 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 Processor -- start 2024-09-13T17:24:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 -- start start 2024-09-13T17:24:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96800ffd80 0x7f9680195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f9680196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101930 0x7f968019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019d7e0 con 0x7f9680100f70 2024-09-13T17:24:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019d950 con 0x7f96800ffd80 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9686e80640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019db10 con 0x7f9680101930 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f9680196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f9680196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53334/0 (socket says 172.21.15.44:53334) 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 -- 172.21.15.44:0/3565082746 learned_addr learned my addr 172.21.15.44:0/3565082746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f9684c1c640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96800ffd80 0x7f9680195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f968541d640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101930 0x7f968019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 -- 172.21.15.44:0/3565082746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101930 msgr2=0x7f968019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101930 0x7f968019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 -- 172.21.15.44:0/3565082746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96800ffd80 msgr2=0x7f9680195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96800ffd80 0x7f9680195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.516+0000 7f967ffff640 1 -- 172.21.15.44:0/3565082746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009840 con 0x7f9680100f70 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.517+0000 7f967ffff640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f9680196240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f966c0057c0 tx=0x7f966c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.517+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c0054b0 con 0x7f9680100f70 2024-09-13T17:24:20.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.517+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c009560 con 0x7f9680100f70 2024-09-13T17:24:20.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.517+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f966c0044e0 con 0x7f9680100f70 2024-09-13T17:24:20.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.517+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c003920 con 0x7f9680100f70 2024-09-13T17:24:20.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.517+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968019e100 con 0x7f9680100f70 2024-09-13T17:24:20.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.518+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9680107300 con 0x7f9680100f70 2024-09-13T17:24:20.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.520+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f966c003a80 con 0x7f9680100f70 2024-09-13T17:24:20.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.520+0000 7f967dffb640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96700a1110 0x7f96700a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:20.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.521+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f966c0eda70 con 0x7f9680100f70 2024-09-13T17:24:20.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.521+0000 7f9684c1c640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96700a1110 0x7f96700a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:20.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.521+0000 7f9684c1c640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96700a1110 0x7f96700a35d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f96801971d0 tx=0x7f9674005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:20.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.526+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f966c0b74a0 con 0x7f9680100f70 2024-09-13T17:24:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:20 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/272600524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T17:24:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:20 smithi070 ceph-mon[34502]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T17:24:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:20 smithi070 ceph-mon[34502]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:20.698+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f968010b4f0 con 0x7f9680100f70 2024-09-13T17:24:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:20 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/272600524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T17:24:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:20 smithi100 ceph-mon[35055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T17:24:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:20 smithi100 ceph-mon[35055]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:20.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:20 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/272600524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T17:24:20.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:20 smithi044 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T17:24:20.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:20 smithi044 ceph-mon[31175]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.372+0000 7f967dffb640 1 -- 172.21.15.44:0/3565082746 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f966c04a020 con 0x7f9680100f70 2024-09-13T17:24:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T17:24:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.377+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96700a1110 msgr2=0x7f96700a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.377+0000 7f9686e80640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96700a1110 0x7f96700a35d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f96801971d0 tx=0x7f9674005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.377+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 msgr2=0x7f9680196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.377+0000 7f9686e80640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f9680196240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f966c0057c0 tx=0x7f966c0042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.378+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 shutdown_connections 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.378+0000 7f9686e80640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96700a1110 0x7f96700a35d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.378+0000 7f9686e80640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101930 0x7f968019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.378+0000 7f9686e80640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f70 0x7f9680196240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.378+0000 7f9686e80640 1 --2- 172.21.15.44:0/3565082746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96800ffd80 0x7f9680195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.378+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 >> 172.21.15.44:0/3565082746 conn(0x7f96800fb4e0 msgr2=0x7f96801044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.379+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 shutdown_connections 2024-09-13T17:24:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.379+0000 7f9686e80640 1 -- 172.21.15.44:0/3565082746 wait complete. 2024-09-13T17:24:21.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:24:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 -- 172.21.15.44:0/2250300790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd0100840 msgr2=0x7f3bd01070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 --2- 172.21.15.44:0/2250300790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd0100840 0x7f3bd01070d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009ca0 tx=0x7f3bc802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 -- 172.21.15.44:0/2250300790 shutdown_connections 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 --2- 172.21.15.44:0/2250300790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd00ffe80 0x7f3bd0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 --2- 172.21.15.44:0/2250300790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd0100840 0x7f3bd01070d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 --2- 172.21.15.44:0/2250300790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00fec90 0x7f3bd00ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 -- 172.21.15.44:0/2250300790 >> 172.21.15.44:0/2250300790 conn(0x7f3bd00fa3f0 msgr2=0x7f3bd00fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 -- 172.21.15.44:0/2250300790 shutdown_connections 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.501+0000 7f3bd7725640 1 -- 172.21.15.44:0/2250300790 wait complete. 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 Processor -- start 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 -- start start 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 0x7f3bd0195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00ffe80 0x7f3bd0196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd0100840 0x7f3bd019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd019d6d0 con 0x7f3bd00fec90 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd019d840 con 0x7f3bd00ffe80 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd7725640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd019da00 con 0x7f3bd0100840 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 0x7f3bd0195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 0x7f3bd0195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53354/0 (socket says 172.21.15.44:53354) 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 -- 172.21.15.44:0/803765221 learned_addr learned my addr 172.21.15.44:0/803765221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd5cc2640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd0100840 0x7f3bd019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 -- 172.21.15.44:0/803765221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd0100840 msgr2=0x7f3bd019d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd0100840 0x7f3bd019d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 -- 172.21.15.44:0/803765221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00ffe80 msgr2=0x7f3bd0196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd4cc0640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00ffe80 0x7f3bd0196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00ffe80 0x7f3bd0196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.502+0000 7f3bd54c1640 1 -- 172.21.15.44:0/803765221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009900 con 0x7f3bd00fec90 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.503+0000 7f3bd54c1640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 0x7f3bd0195be0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d950 tx=0x7f3bcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.503+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc009c30 con 0x7f3bd00fec90 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.503+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bcc009d90 con 0x7f3bd00fec90 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.503+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd019dce0 con 0x7f3bd00fec90 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.503+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00f6b0 con 0x7f3bd00fec90 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.503+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd019e1a0 con 0x7f3bd00fec90 2024-09-13T17:24:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.504+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0106120 con 0x7f3bd00fec90 2024-09-13T17:24:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.505+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3bcc010460 con 0x7f3bd00fec90 2024-09-13T17:24:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.506+0000 7f3bc67fc640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3bbc0a1090 0x7f3bbc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.506+0000 7f3bd4cc0640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3bbc0a1090 0x7f3bbc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.506+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f3bcc0cc800 con 0x7f3bd00fec90 2024-09-13T17:24:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.506+0000 7f3bd4cc0640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3bbc0a1090 0x7f3bbc0a3550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005d80 tx=0x7f3bc0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.509+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3bcc096140 con 0x7f3bd00fec90 2024-09-13T17:24:21.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:21 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3565082746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T17:24:21.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.720+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3bd010b540 con 0x7f3bd00fec90 2024-09-13T17:24:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.728+0000 7f3bc67fc640 1 -- 172.21.15.44:0/803765221 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3bcc010710 con 0x7f3bd00fec90 2024-09-13T17:24:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3bbc0a1090 msgr2=0x7f3bbc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3bbc0a1090 0x7f3bbc0a3550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005d80 tx=0x7f3bc0005c90 comp rx=0 tx=0).stop 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 msgr2=0x7f3bd0195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 0x7f3bd0195be0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d950 tx=0x7f3bcc00de10 comp rx=0 tx=0).stop 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 shutdown_connections 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3bbc0a1090 0x7f3bbc0a3550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd0100840 0x7f3bd019d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00ffe80 0x7f3bd0196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 --2- 172.21.15.44:0/803765221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd00fec90 0x7f3bd0195be0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d950 tx=0x7f3bcc00de10 comp rx=0 tx=0).stop 2024-09-13T17:24:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 >> 172.21.15.44:0/803765221 conn(0x7f3bd00fa3f0 msgr2=0x7f3bd01032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 shutdown_connections 2024-09-13T17:24:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.733+0000 7f3bd7725640 1 -- 172.21.15.44:0/803765221 wait complete. 2024-09-13T17:24:21.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T17:24:21.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T17:24:21.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:21 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3565082746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T17:24:21.753 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T17:24:21.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:24:21.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:24:21.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:21.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:21 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3565082746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T17:24:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 -- 172.21.15.44:0/112017785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c0fec60 msgr2=0x7f529c0ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 --2- 172.21.15.44:0/112017785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c0fec60 0x7f529c0ff060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5298009900 tx=0x7f529802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 -- 172.21.15.44:0/112017785 shutdown_connections 2024-09-13T17:24:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 --2- 172.21.15.44:0/112017785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c0ffe50 0x7f529c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 --2- 172.21.15.44:0/112017785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f529c100810 0x7f529c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 --2- 172.21.15.44:0/112017785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c0fec60 0x7f529c0ff060 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.861+0000 7f52a3353640 1 -- 172.21.15.44:0/112017785 >> 172.21.15.44:0/112017785 conn(0x7f529c0fa3e0 msgr2=0x7f529c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 -- 172.21.15.44:0/112017785 shutdown_connections 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 -- 172.21.15.44:0/112017785 wait complete. 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 Processor -- start 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 -- start start 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c0fec60 0x7f529c068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f529c0ffe50 0x7f529c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 0x7f529c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c19d760 con 0x7f529c100810 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c19d8d0 con 0x7f529c0ffe50 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a3353640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c19da90 con 0x7f529c0fec60 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a18f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 0x7f529c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a18f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 0x7f529c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53376/0 (socket says 172.21.15.44:53376) 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.862+0000 7f52a18f0640 1 -- 172.21.15.44:0/1824825751 learned_addr learned my addr 172.21.15.44:0/1824825751 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a10ef640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c0fec60 0x7f529c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a18f0640 1 -- 172.21.15.44:0/1824825751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c0fec60 msgr2=0x7f529c068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a18f0640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c0fec60 0x7f529c068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a18f0640 1 -- 172.21.15.44:0/1824825751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f529c0ffe50 msgr2=0x7f529c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a08ee640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f529c0ffe50 0x7f529c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a18f0640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f529c0ffe50 0x7f529c19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a18f0640 1 -- 172.21.15.44:0/1824825751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298009560 con 0x7f529c100810 2024-09-13T17:24:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a18f0640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 0x7f529c19cf80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f529400cce0 tx=0x7f529400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400f450 con 0x7f529c100810 2024-09-13T17:24:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f529400fa40 con 0x7f529c100810 2024-09-13T17:24:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c19dd10 con 0x7f529c100810 2024-09-13T17:24:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940189a0 con 0x7f529c100810 2024-09-13T17:24:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.863+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c19e1d0 con 0x7f529c100810 2024-09-13T17:24:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.864+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c106160 con 0x7f529c100810 2024-09-13T17:24:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.865+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f529400fbb0 con 0x7f529c100810 2024-09-13T17:24:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.866+0000 7f52927fc640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52880a1110 0x7f52880a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.866+0000 7f52a10ef640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52880a1110 0x7f52880a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.866+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52940cccb0 con 0x7f529c100810 2024-09-13T17:24:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.866+0000 7f52a10ef640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52880a1110 0x7f52880a35d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f5298005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:21.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:21.870+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f52940964f0 con 0x7f529c100810 2024-09-13T17:24:22.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.050+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f529c02c3e0 con 0x7f529c100810 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3565082746' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: pgmap v272: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/803765221' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/803765221' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-13T17:24:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:22 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1824825751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T17:24:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3565082746' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T17:24:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T17:24:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: pgmap v272: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/803765221' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T17:24:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-13T17:24:22.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T17:24:22.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/803765221' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T17:24:22.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-13T17:24:22.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-13T17:24:22.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:22 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1824825751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T17:24:22.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.731+0000 7f52927fc640 1 -- 172.21.15.44:0/1824825751 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5294099b10 con 0x7f529c100810 2024-09-13T17:24:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52880a1110 msgr2=0x7f52880a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52880a1110 0x7f52880a35d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f5298005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 msgr2=0x7f529c19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 0x7f529c19cf80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f529400cce0 tx=0x7f529400a620 comp rx=0 tx=0).stop 2024-09-13T17:24:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 shutdown_connections 2024-09-13T17:24:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52880a1110 0x7f52880a35d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f529c100810 0x7f529c19cf80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f529c0ffe50 0x7f529c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 --2- 172.21.15.44:0/1824825751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c0fec60 0x7f529c068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 >> 172.21.15.44:0/1824825751 conn(0x7f529c0fa3e0 msgr2=0x7f529c103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 shutdown_connections 2024-09-13T17:24:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.733+0000 7f52a3353640 1 -- 172.21.15.44:0/1824825751 wait complete. 2024-09-13T17:24:22.745 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:24:22.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3565082746' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: pgmap v272: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/803765221' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/803765221' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-13T17:24:22.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:22 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1824825751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 -- 172.21.15.44:0/3912532013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 msgr2=0x7f1f8c102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/3912532013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c102c40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009900 tx=0x7f1f7402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 -- 172.21.15.44:0/3912532013 shutdown_connections 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/3912532013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c105ab0 0x7f1f8c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/3912532013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c103180 0x7f1f8c105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/3912532013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c102c40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 -- 172.21.15.44:0/3912532013 >> 172.21.15.44:0/3912532013 conn(0x7f1f8c0fa430 msgr2=0x7f1f8c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 -- 172.21.15.44:0/3912532013 shutdown_connections 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.858+0000 7f1f92b2a640 1 -- 172.21.15.44:0/3912532013 wait complete. 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 Processor -- start 2024-09-13T17:24:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 -- start start 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c103180 0x7f1f8c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c105ab0 0x7f1f8c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19d8b0 con 0x7f1f8c100850 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19da20 con 0x7f1f8c105ab0 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f92b2a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19dbe0 con 0x7f1f8c103180 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53392/0 (socket says 172.21.15.44:53392) 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 -- 172.21.15.44:0/2607008790 learned_addr learned my addr 172.21.15.44:0/2607008790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f910c7640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c105ab0 0x7f1f8c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f8bfff640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c103180 0x7f1f8c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 -- 172.21.15.44:0/2607008790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c103180 msgr2=0x7f1f8c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c103180 0x7f1f8c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 -- 172.21.15.44:0/2607008790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c105ab0 msgr2=0x7f1f8c19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c105ab0 0x7f1f8c19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 -- 172.21.15.44:0/2607008790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009560 con 0x7f1f8c100850 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f8bfff640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c103180 0x7f1f8c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f910c7640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c105ab0 0x7f1f8c19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.859+0000 7f1f908c6640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c195dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f740057c0 tx=0x7f1f740052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.860+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74031d70 con 0x7f1f8c100850 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.860+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f740054a0 con 0x7f1f8c100850 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.860+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c19de60 con 0x7f1f8c100850 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.860+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74003b10 con 0x7f1f8c100850 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.860+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c19e280 con 0x7f1f8c100850 2024-09-13T17:24:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.861+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c0fbf20 con 0x7f1f8c100850 2024-09-13T17:24:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.863+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1f74003c70 con 0x7f1f8c100850 2024-09-13T17:24:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.863+0000 7f1f89ffb640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f780a1110 0x7f1f780a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.863+0000 7f1f8bfff640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f780a1110 0x7f1f780a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.863+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f1f740edf40 con 0x7f1f8c100850 2024-09-13T17:24:22.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.864+0000 7f1f8bfff640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f780a1110 0x7f1f780a35d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c1972d0 tx=0x7f1f7c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:22.866+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1f740b76f0 con 0x7f1f8c100850 2024-09-13T17:24:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:23.039+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f8c10b600 con 0x7f1f8c100850 2024-09-13T17:24:23.675 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T17:24:23.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] up:active 2024-09-13T17:24:23.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:23.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1824825751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T17:24:23.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T17:24:23.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:23.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:23 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2607008790' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] up:active 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1824825751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:23 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2607008790' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T17:24:23.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T17:24:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] up:active 2024-09-13T17:24:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1824825751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T17:24:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T17:24:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:23 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2607008790' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T17:24:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:24 smithi070 ceph-mon[34502]: pgmap v275: 97 pgs: 9 creating+peering, 25 active+clean, 63 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-13T17:24:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:24 smithi070 ceph-mon[34502]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T17:24:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.739+0000 7f1f89ffb640 1 -- 172.21.15.44:0/2607008790 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1f7404a100 con 0x7f1f8c100850 2024-09-13T17:24:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data_ec' created 2024-09-13T17:24:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f780a1110 msgr2=0x7f1f780a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f780a1110 0x7f1f780a35d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c1972d0 tx=0x7f1f7c005e50 comp rx=0 tx=0).stop 2024-09-13T17:24:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 msgr2=0x7f1f8c195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c195dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f740057c0 tx=0x7f1f740052b0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 shutdown_connections 2024-09-13T17:24:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f780a1110 0x7f1f780a35d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f8c103180 0x7f1f8c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8c105ab0 0x7f1f8c19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 --2- 172.21.15.44:0/2607008790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c100850 0x7f1f8c195dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 >> 172.21.15.44:0/2607008790 conn(0x7f1f8c0fa430 msgr2=0x7f1f8c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.744+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 shutdown_connections 2024-09-13T17:24:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.745+0000 7f1f92b2a640 1 -- 172.21.15.44:0/2607008790 wait complete. 2024-09-13T17:24:24.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:24 smithi044 ceph-mon[31175]: pgmap v275: 97 pgs: 9 creating+peering, 25 active+clean, 63 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-13T17:24:24.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:24 smithi044 ceph-mon[31175]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T17:24:24.760 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T17:24:24.774 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:24:24 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: 2024-09-13T17:24:24.558+0000 7f8426e46640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:24:24.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:24 smithi100 ceph-mon[35055]: pgmap v275: 97 pgs: 9 creating+peering, 25 active+clean, 63 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-13T17:24:24.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:24 smithi100 ceph-mon[35055]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 -- 172.21.15.44:0/4132296943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 msgr2=0x7f870c107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 --2- 172.21.15.44:0/4132296943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c107ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8704009ca0 tx=0x7f870402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 -- 172.21.15.44:0/4132296943 shutdown_connections 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 --2- 172.21.15.44:0/4132296943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c103120 0x7f870c105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 --2- 172.21.15.44:0/4132296943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c107ec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 --2- 172.21.15.44:0/4132296943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f870c1007f0 0x7f870c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 -- 172.21.15.44:0/4132296943 >> 172.21.15.44:0/4132296943 conn(0x7f870c0fa3d0 msgr2=0x7f870c0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 -- 172.21.15.44:0/4132296943 shutdown_connections 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.872+0000 7f8711641640 1 -- 172.21.15.44:0/4132296943 wait complete. 2024-09-13T17:24:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 Processor -- start 2024-09-13T17:24:24.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 -- start start 2024-09-13T17:24:24.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c1007f0 0x7f870c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f870c103120 0x7f870c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c19d7d0 con 0x7f870c105a50 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c19d940 con 0x7f870c103120 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f8711641640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c19db00 con 0x7f870c1007f0 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53420/0 (socket says 172.21.15.44:53420) 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f870b7fe640 1 -- 172.21.15.44:0/2048274182 learned_addr learned my addr 172.21.15.44:0/2048274182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.873+0000 7f870affd640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c1007f0 0x7f870c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870a7fc640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f870c103120 0x7f870c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870b7fe640 1 -- 172.21.15.44:0/2048274182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c1007f0 msgr2=0x7f870c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:24.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870b7fe640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c1007f0 0x7f870c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870b7fe640 1 -- 172.21.15.44:0/2048274182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f870c103120 msgr2=0x7f870c196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870b7fe640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f870c103120 0x7f870c196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870b7fe640 1 -- 172.21.15.44:0/2048274182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8704009900 con 0x7f870c105a50 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870affd640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c1007f0 0x7f870c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f870b7fe640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c19d260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f870402aeb0 tx=0x7f8704003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040307c0 con 0x7f870c105a50 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8704040070 con 0x7f870c105a50 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c19dd80 con 0x7f870c105a50 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040043c0 con 0x7f870c105a50 2024-09-13T17:24:24.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.874+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c19e230 con 0x7f870c105a50 2024-09-13T17:24:24.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.875+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c0fbec0 con 0x7f870c105a50 2024-09-13T17:24:24.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.876+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8704042030 con 0x7f870c105a50 2024-09-13T17:24:24.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.877+0000 7f86f3fff640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86f40a1090 0x7f86f40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:24.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.877+0000 7f870affd640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86f40a1090 0x7f86f40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:24.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.877+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f87040ee1d0 con 0x7f870c105a50 2024-09-13T17:24:24.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.878+0000 7f870affd640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86f40a1090 0x7f86f40a3550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f870c196b90 tx=0x7f86fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:24.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:24.880+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f87040b7790 con 0x7f870c105a50 2024-09-13T17:24:25.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.051+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f870c10b520 con 0x7f870c105a50 2024-09-13T17:24:25.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:25 smithi070 ceph-mon[34502]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-13T17:24:25.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:25 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:24:25.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:25 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2607008790' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T17:24:25.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:25 smithi070 ceph-mon[34502]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T17:24:25.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:25 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2048274182' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T17:24:25.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.737+0000 7f86f3fff640 1 -- 172.21.15.44:0/2048274182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f87040badb0 con 0x7f870c105a50 2024-09-13T17:24:25.739 INFO:teuthology.orchestra.run.smithi044.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.742+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86f40a1090 msgr2=0x7f86f40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.742+0000 7f8711641640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86f40a1090 0x7f86f40a3550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f870c196b90 tx=0x7f86fc00b410 comp rx=0 tx=0).stop 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.742+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 msgr2=0x7f870c19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.742+0000 7f8711641640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c19d260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f870402aeb0 tx=0x7f8704003c20 comp rx=0 tx=0).stop 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.743+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 shutdown_connections 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.743+0000 7f8711641640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f86f40a1090 0x7f86f40a3550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.743+0000 7f8711641640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c105a50 0x7f870c19d260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.743+0000 7f8711641640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f870c103120 0x7f870c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.743+0000 7f8711641640 1 --2- 172.21.15.44:0/2048274182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f870c1007f0 0x7f870c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.743+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 >> 172.21.15.44:0/2048274182 conn(0x7f870c0fa3d0 msgr2=0x7f870c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:25.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.744+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 shutdown_connections 2024-09-13T17:24:25.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.744+0000 7f8711641640 1 -- 172.21.15.44:0/2048274182 wait complete. 2024-09-13T17:24:25.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:25 smithi044 ceph-mon[31175]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-13T17:24:25.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:25 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:24:25.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:25 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2607008790' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T17:24:25.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:25 smithi044 ceph-mon[31175]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T17:24:25.765 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:25 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2048274182' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T17:24:25.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T17:24:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:25 smithi100 ceph-mon[35055]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-13T17:24:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:25 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:24:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:25 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2607008790' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T17:24:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:25 smithi100 ceph-mon[35055]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T17:24:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:25 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2048274182' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 -- 172.21.15.44:0/2224747957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 msgr2=0x7f0214107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2224747957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f0214107020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f020c009ca0 tx=0x7f020c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 -- 172.21.15.44:0/2224747957 shutdown_connections 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2224747957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f0214107020 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2224747957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02140ffdd0 0x7f0214100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2224747957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02140febe0 0x7f02140fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 -- 172.21.15.44:0/2224747957 >> 172.21.15.44:0/2224747957 conn(0x7f02140fa360 msgr2=0x7f02140fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 -- 172.21.15.44:0/2224747957 shutdown_connections 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 -- 172.21.15.44:0/2224747957 wait complete. 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 Processor -- start 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.877+0000 7f021a1a0640 1 -- start start 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02140febe0 0x7f0214102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02140ffdd0 0x7f02141011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f02141016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214101e50 con 0x7f0214100790 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214101fc0 con 0x7f02140febe0 2024-09-13T17:24:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021419fc10 con 0x7f02140ffdd0 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f02141016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f02141016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53438/0 (socket says 172.21.15.44:53438) 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 -- 172.21.15.44:0/2734288755 learned_addr learned my addr 172.21.15.44:0/2734288755 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021899d640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02140ffdd0 0x7f02141011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021919e640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02140febe0 0x7f0214102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 -- 172.21.15.44:0/2734288755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02140ffdd0 msgr2=0x7f02141011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02140ffdd0 0x7f02141011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 -- 172.21.15.44:0/2734288755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02140febe0 msgr2=0x7f0214102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02140febe0 0x7f0214102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 -- 172.21.15.44:0/2734288755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c009900 con 0x7f0214100790 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021999f640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f02141016f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f020c0098d0 tx=0x7f020c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0307c0 con 0x7f0214100790 2024-09-13T17:24:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f020c040070 con 0x7f0214100790 2024-09-13T17:24:25.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021419fe60 con 0x7f0214100790 2024-09-13T17:24:25.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.878+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c004460 con 0x7f0214100790 2024-09-13T17:24:25.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.879+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02141a02e0 con 0x7f0214100790 2024-09-13T17:24:25.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.879+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021410a330 con 0x7f0214100790 2024-09-13T17:24:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.881+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f020c042030 con 0x7f0214100790 2024-09-13T17:24:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.881+0000 7f02067fc640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f02000a1090 0x7f02000a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.881+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f020c0ee1d0 con 0x7f0214100790 2024-09-13T17:24:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.881+0000 7f021919e640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f02000a1090 0x7f02000a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.882+0000 7f021919e640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f02000a1090 0x7f02000a3550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02140ffc30 tx=0x7f0208005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:25.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:25.884+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f020c0b7790 con 0x7f0214100790 2024-09-13T17:24:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.096+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f02141a06e0 con 0x7f0214100790 2024-09-13T17:24:26.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.102+0000 7f02067fc640 1 -- 172.21.15.44:0/2734288755 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f020c0badb0 con 0x7f0214100790 2024-09-13T17:24:26.103 INFO:teuthology.orchestra.run.smithi044.stderr:added data pool 4 to fsmap 2024-09-13T17:24:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f02000a1090 msgr2=0x7f02000a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f02000a1090 0x7f02000a3550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02140ffc30 tx=0x7f0208005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 msgr2=0x7f02141016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f02141016f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f020c0098d0 tx=0x7f020c003c20 comp rx=0 tx=0).stop 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 shutdown_connections 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f02000a1090 0x7f02000a3550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02140ffdd0 0x7f02141011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0214100790 0x7f02141016f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 --2- 172.21.15.44:0/2734288755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02140febe0 0x7f0214102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 >> 172.21.15.44:0/2734288755 conn(0x7f02140fa360 msgr2=0x7f0214067630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 shutdown_connections 2024-09-13T17:24:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.104+0000 7f021a1a0640 1 -- 172.21.15.44:0/2734288755 wait complete. 2024-09-13T17:24:26.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:26.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 -- 172.21.15.44:0/4060149903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544103150 msgr2=0x7f0544105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 --2- 172.21.15.44:0/4060149903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544103150 0x7f0544105540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0534009900 tx=0x7f053402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:26.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 -- 172.21.15.44:0/4060149903 shutdown_connections 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 --2- 172.21.15.44:0/4060149903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0544105a80 0x7f0544107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 --2- 172.21.15.44:0/4060149903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544103150 0x7f0544105540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 --2- 172.21.15.44:0/4060149903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0544100820 0x7f0544102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 -- 172.21.15.44:0/4060149903 >> 172.21.15.44:0/4060149903 conn(0x7f05440fa420 msgr2=0x7f05440fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 -- 172.21.15.44:0/4060149903 shutdown_connections 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.217+0000 7f054bed0640 1 -- 172.21.15.44:0/4060149903 wait complete. 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 Processor -- start 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 -- start start 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 0x7f0544100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0544103150 0x7f05440fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0544105a80 0x7f05440fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440ff470 con 0x7f0544100820 2024-09-13T17:24:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440ff5e0 con 0x7f0544105a80 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054bed0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054419fe00 con 0x7f0544103150 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 0x7f0544100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 0x7f0544100190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53462/0 (socket says 172.21.15.44:53462) 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 -- 172.21.15.44:0/2802013829 learned_addr learned my addr 172.21.15.44:0/2802013829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054a46d640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0544105a80 0x7f05440fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f054946b640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0544103150 0x7f05440fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 -- 172.21.15.44:0/2802013829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0544103150 msgr2=0x7f05440fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0544103150 0x7f05440fe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 -- 172.21.15.44:0/2802013829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0544105a80 msgr2=0x7f05440fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0544105a80 0x7f05440fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 -- 172.21.15.44:0/2802013829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534009560 con 0x7f0544100820 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.218+0000 7f0549c6c640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 0x7f0544100190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f054000c8c0 tx=0x7f054000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.219+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540004220 con 0x7f0544100820 2024-09-13T17:24:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.219+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0540007e40 con 0x7f0544100820 2024-09-13T17:24:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.219+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05441a00b0 con 0x7f0544100820 2024-09-13T17:24:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.219+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400047a0 con 0x7f0544100820 2024-09-13T17:24:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.219+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05441a05e0 con 0x7f0544100820 2024-09-13T17:24:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.220+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05440fbf10 con 0x7f0544100820 2024-09-13T17:24:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.221+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f05400079b0 con 0x7f0544100820 2024-09-13T17:24:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.221+0000 7f053affd640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f05300a1110 0x7f05300a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.222+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f05400cc750 con 0x7f0544100820 2024-09-13T17:24:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.222+0000 7f054946b640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f05300a1110 0x7f05300a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.222+0000 7f054946b640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f05300a1110 0x7f05300a35d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0534006010 tx=0x7f0534005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.226+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0540095cf0 con 0x7f0544100820 2024-09-13T17:24:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.435+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f054410b520 con 0x7f0544100820 2024-09-13T17:24:26.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.436+0000 7f053affd640 1 -- 172.21.15.44:0/2802013829 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10798 (secure 0 0 0) 0x7f0540099310 con 0x7f0544100820 2024-09-13T17:24:26.437 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:26.437 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:26.097886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T17:24:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-09-13T17:24:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.437+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f05300a1110 msgr2=0x7f05300a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.437+0000 7f054bed0640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f05300a1110 0x7f05300a35d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0534006010 tx=0x7f0534005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 msgr2=0x7f0544100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 0x7f0544100190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f054000c8c0 tx=0x7f054000cd80 comp rx=0 tx=0).stop 2024-09-13T17:24:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 shutdown_connections 2024-09-13T17:24:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f05300a1110 0x7f05300a35d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0544103150 0x7f05440fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0544105a80 0x7f05440fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 --2- 172.21.15.44:0/2802013829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0544100820 0x7f0544100190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 >> 172.21.15.44:0/2802013829 conn(0x7f05440fa420 msgr2=0x7f05441012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 shutdown_connections 2024-09-13T17:24:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.438+0000 7f054bed0640 1 -- 172.21.15.44:0/2802013829 wait complete. 2024-09-13T17:24:26.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 -- 172.21.15.44:0/3139092497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 msgr2=0x7fe270102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 --2- 172.21.15.44:0/3139092497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270102c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe260009900 tx=0x7fe26002a410 comp rx=0 tx=0).stop 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 -- 172.21.15.44:0/3139092497 shutdown_connections 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 --2- 172.21.15.44:0/3139092497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe270103180 0x7fe270105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 --2- 172.21.15.44:0/3139092497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe270105ab0 0x7fe270107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 --2- 172.21.15.44:0/3139092497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270102c40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 -- 172.21.15.44:0/3139092497 >> 172.21.15.44:0/3139092497 conn(0x7fe2700fa430 msgr2=0x7fe2700fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 -- 172.21.15.44:0/3139092497 shutdown_connections 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.551+0000 7fe27582b640 1 -- 172.21.15.44:0/3139092497 wait complete. 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 Processor -- start 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 -- start start 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe270103180 0x7fe270196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe270105ab0 0x7fe27019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27019d8a0 con 0x7fe270100850 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27019da10 con 0x7fe270103180 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe27582b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27019dbd0 con 0x7fe270105ab0 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe26effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe26effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53474/0 (socket says 172.21.15.44:53474) 2024-09-13T17:24:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe26effd640 1 -- 172.21.15.44:0/3234134479 learned_addr learned my addr 172.21.15.44:0/3234134479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.552+0000 7fe267fff640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe270103180 0x7fe270196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26f7fe640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe270105ab0 0x7fe27019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26effd640 1 -- 172.21.15.44:0/3234134479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe270105ab0 msgr2=0x7fe27019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26effd640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe270105ab0 0x7fe27019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26effd640 1 -- 172.21.15.44:0/3234134479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe270103180 msgr2=0x7fe270196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26effd640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe270103180 0x7fe270196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26effd640 1 -- 172.21.15.44:0/3234134479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009560 con 0x7fe270100850 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe26effd640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270195dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2600057c0 tx=0x7fe2600052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260031d70 con 0x7fe270100850 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27019de50 con 0x7fe270100850 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2600054a0 con 0x7fe270100850 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260003b10 con 0x7fe270100850 2024-09-13T17:24:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.553+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27019e220 con 0x7fe270100850 2024-09-13T17:24:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.554+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2700fbf20 con 0x7fe270100850 2024-09-13T17:24:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.555+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe260003c70 con 0x7fe270100850 2024-09-13T17:24:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.556+0000 7fe274829640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe25c0a1090 0x7fe25c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.556+0000 7fe267fff640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe25c0a1090 0x7fe25c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.556+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe2600edf70 con 0x7fe270100850 2024-09-13T17:24:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.557+0000 7fe267fff640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe25c0a1090 0x7fe25c0a3550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe270069150 tx=0x7fe25800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.560+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe2600b7420 con 0x7fe270100850 2024-09-13T17:24:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: pgmap v278: 129 pgs: 10 creating+peering, 71 active+clean, 48 unknown; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-09-13T17:24:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2048274182' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T17:24:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T17:24:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2734288755' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T17:24:26.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T17:24:26.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2734288755' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T17:24:26.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:26 smithi070 ceph-mon[34502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:26.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.735+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe27010b580 con 0x7fe270100850 2024-09-13T17:24:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.737+0000 7fe274829640 1 -- 172.21.15.44:0/3234134479 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23900 (secure 0 0 0) 0x7fe26004a090 con 0x7fe270100850 2024-09-13T17:24:26.738 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:26.738 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:26.097662+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe25c0a1090 msgr2=0x7fe25c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe25c0a1090 0x7fe25c0a3550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe270069150 tx=0x7fe25800b390 comp rx=0 tx=0).stop 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 msgr2=0x7fe270195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270195dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2600057c0 tx=0x7fe2600052b0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 shutdown_connections 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe25c0a1090 0x7fe25c0a3550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe270105ab0 0x7fe27019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe270103180 0x7fe270196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 --2- 172.21.15.44:0/3234134479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe270100850 0x7fe270195dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 >> 172.21.15.44:0/3234134479 conn(0x7fe2700fa430 msgr2=0x7fe2701011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:26.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 shutdown_connections 2024-09-13T17:24:26.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.739+0000 7fe27582b640 1 -- 172.21.15.44:0/3234134479 wait complete. 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: pgmap v278: 129 pgs: 10 creating+peering, 71 active+clean, 48 unknown; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2048274182' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2734288755' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2734288755' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T17:24:26.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:26 smithi044 ceph-mon[31175]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:26.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: pgmap v278: 129 pgs: 10 creating+peering, 71 active+clean, 48 unknown; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2048274182' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2734288755' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2734288755' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T17:24:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:26 smithi100 ceph-mon[35055]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:26.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 -- 172.21.15.44:0/3328874106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac100850 msgr2=0x7f80ac1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 --2- 172.21.15.44:0/3328874106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac100850 0x7f80ac1070e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009ca0 tx=0x7f80a402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 -- 172.21.15.44:0/3328874106 shutdown_connections 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 --2- 172.21.15.44:0/3328874106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80ac0ffe90 0x7f80ac100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 --2- 172.21.15.44:0/3328874106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac100850 0x7f80ac1070e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 --2- 172.21.15.44:0/3328874106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80ac0feca0 0x7f80ac0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 -- 172.21.15.44:0/3328874106 >> 172.21.15.44:0/3328874106 conn(0x7f80ac0fa440 msgr2=0x7f80ac0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 -- 172.21.15.44:0/3328874106 shutdown_connections 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 -- 172.21.15.44:0/3328874106 wait complete. 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 Processor -- start 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.854+0000 7f80b24f0640 1 -- start start 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80ac0feca0 0x7f80ac068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 0x7f80ac19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80ac100850 0x7f80ac19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac19d720 con 0x7f80ac0ffe90 2024-09-13T17:24:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac19d890 con 0x7f80ac100850 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac19da50 con 0x7f80ac0feca0 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 0x7f80ac19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 0x7f80ac19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53486/0 (socket says 172.21.15.44:53486) 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 -- 172.21.15.44:0/164826592 learned_addr learned my addr 172.21.15.44:0/164826592 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b0a8d640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80ac100850 0x7f80ac19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80abfff640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80ac0feca0 0x7f80ac068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 -- 172.21.15.44:0/164826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80ac0feca0 msgr2=0x7f80ac068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80ac0feca0 0x7f80ac068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 -- 172.21.15.44:0/164826592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80ac100850 msgr2=0x7f80ac19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80ac100850 0x7f80ac19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 -- 172.21.15.44:0/164826592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4009900 con 0x7f80ac0ffe90 2024-09-13T17:24:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80ab7fe640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 0x7f80ac19a920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80a000e9f0 tx=0x7f80a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0009720 con 0x7f80ac0ffe90 2024-09-13T17:24:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80a000cd80 con 0x7f80ac0ffe90 2024-09-13T17:24:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac19dcd0 con 0x7f80ac0ffe90 2024-09-13T17:24:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0010430 con 0x7f80ac0ffe90 2024-09-13T17:24:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.855+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac19e190 con 0x7f80ac0ffe90 2024-09-13T17:24:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.856+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac106100 con 0x7f80ac0ffe90 2024-09-13T17:24:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.858+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f80a0004fb0 con 0x7f80ac0ffe90 2024-09-13T17:24:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.858+0000 7f80a97fa640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80940a1110 0x7f80940a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.858+0000 7f80abfff640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80940a1110 0x7f80940a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.858+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f80a0014070 con 0x7f80ac0ffe90 2024-09-13T17:24:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.859+0000 7f80abfff640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80940a1110 0x7f80940a35d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f809c0058c0 tx=0x7f809c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:26.862+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f80a0096420 con 0x7f80ac0ffe90 2024-09-13T17:24:27.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.035+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80ac10a410 con 0x7f80ac0ffe90 2024-09-13T17:24:27.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.036+0000 7f80a97fa640 1 -- 172.21.15.44:0/164826592 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23900 (secure 0 0 0) 0x7f80a0099a40 con 0x7f80ac0ffe90 2024-09-13T17:24:27.038 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:27.038 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:26.097662+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:27.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80940a1110 msgr2=0x7f80940a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80940a1110 0x7f80940a35d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f809c0058c0 tx=0x7f809c005850 comp rx=0 tx=0).stop 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 msgr2=0x7f80ac19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 0x7f80ac19a920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80a000e9f0 tx=0x7f80a000edf0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 shutdown_connections 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80940a1110 0x7f80940a35d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80ac100850 0x7f80ac19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ac0ffe90 0x7f80ac19a920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 --2- 172.21.15.44:0/164826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80ac0feca0 0x7f80ac068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.038+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 >> 172.21.15.44:0/164826592 conn(0x7f80ac0fa440 msgr2=0x7f80ac1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.039+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 shutdown_connections 2024-09-13T17:24:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.039+0000 7f80b24f0640 1 -- 172.21.15.44:0/164826592 wait complete. 2024-09-13T17:24:27.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/2000436118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540ffec0 msgr2=0x7f7e54100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/2000436118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540ffec0 0x7f7e54100340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009900 tx=0x7f7e4402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/2000436118 shutdown_connections 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/2000436118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 0x7f7e54107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/2000436118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540ffec0 0x7f7e54100340 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/2000436118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e540fecd0 0x7f7e540ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/2000436118 >> 172.21.15.44:0/2000436118 conn(0x7f7e540fa430 msgr2=0x7f7e540fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/2000436118 shutdown_connections 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.167+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/2000436118 wait complete. 2024-09-13T17:24:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 Processor -- start 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 -- start start 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 0x7f7e54195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e540ffec0 0x7f7e54196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 0x7f7e5419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419d8e0 con 0x7f7e540fecd0 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419da50 con 0x7f7e540ffec0 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5c6f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419dc10 con 0x7f7e54100880 2024-09-13T17:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 0x7f7e54195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 0x7f7e54195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53502/0 (socket says 172.21.15.44:53502) 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 -- 172.21.15.44:0/742910266 learned_addr learned my addr 172.21.15.44:0/742910266 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e59c8b640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e540ffec0 0x7f7e54196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5ac8d640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 0x7f7e5419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 -- 172.21.15.44:0/742910266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 msgr2=0x7f7e5419d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 0x7f7e5419d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 -- 172.21.15.44:0/742910266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e540ffec0 msgr2=0x7f7e54196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e540ffec0 0x7f7e54196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 -- 172.21.15.44:0/742910266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44009560 con 0x7f7e540fecd0 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5ac8d640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 0x7f7e5419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.168+0000 7f7e5a48c640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 0x7f7e54195df0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009fd0 tx=0x7f7e5000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.169+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50011470 con 0x7f7e540fecd0 2024-09-13T17:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.169+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e50011a60 con 0x7f7e540fecd0 2024-09-13T17:24:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.169+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5419def0 con 0x7f7e540fecd0 2024-09-13T17:24:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.169+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5001a9c0 con 0x7f7e540fecd0 2024-09-13T17:24:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.169+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5419e310 con 0x7f7e540fecd0 2024-09-13T17:24:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.170+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e541062a0 con 0x7f7e540fecd0 2024-09-13T17:24:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.171+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7e50011bd0 con 0x7f7e540fecd0 2024-09-13T17:24:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.172+0000 7f7e4b7fe640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7e400a1090 0x7f7e400a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.172+0000 7f7e59c8b640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7e400a1090 0x7f7e400a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.172+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7e500cd110 con 0x7f7e540fecd0 2024-09-13T17:24:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.172+0000 7f7e59c8b640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7e400a1090 0x7f7e400a3550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000c00 tx=0x7f7e44004040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.176+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7e50096650 con 0x7f7e540fecd0 2024-09-13T17:24:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.385+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e5410b600 con 0x7f7e540fecd0 2024-09-13T17:24:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.386+0000 7f7e4b7fe640 1 -- 172.21.15.44:0/742910266 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10798 (secure 0 0 0) 0x7f7e50099c70 con 0x7f7e540fecd0 2024-09-13T17:24:27.387 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:27.387 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:26.097886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T17:24:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.387+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7e400a1090 msgr2=0x7f7e400a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.387+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7e400a1090 0x7f7e400a3550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000c00 tx=0x7f7e44004040 comp rx=0 tx=0).stop 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 msgr2=0x7f7e54195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 0x7f7e54195df0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009fd0 tx=0x7f7e5000c270 comp rx=0 tx=0).stop 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 shutdown_connections 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7e400a1090 0x7f7e400a3550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54100880 0x7f7e5419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e540ffec0 0x7f7e54196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 --2- 172.21.15.44:0/742910266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e540fecd0 0x7f7e54195df0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 >> 172.21.15.44:0/742910266 conn(0x7f7e540fa430 msgr2=0x7f7e54103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 shutdown_connections 2024-09-13T17:24:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.388+0000 7f7e5c6f0640 1 -- 172.21.15.44:0/742910266 wait complete. 2024-09-13T17:24:27.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 -- 172.21.15.44:0/3662894213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 msgr2=0x7f9c2c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 --2- 172.21.15.44:0/3662894213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c1001a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c2802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 -- 172.21.15.44:0/3662894213 shutdown_connections 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 --2- 172.21.15.44:0/3662894213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2c100f90 0x7f9c2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 --2- 172.21.15.44:0/3662894213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c101950 0x7f9c2c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 --2- 172.21.15.44:0/3662894213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c1001a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 -- 172.21.15.44:0/3662894213 >> 172.21.15.44:0/3662894213 conn(0x7f9c2c0fb520 msgr2=0x7f9c2c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 -- 172.21.15.44:0/3662894213 shutdown_connections 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 -- 172.21.15.44:0/3662894213 wait complete. 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 Processor -- start 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.501+0000 7f9c33301640 1 -- start start 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c100f90 0x7f9c2c196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2c101950 0x7f9c2c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c19d7e0 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c19d950 con 0x7f9c2c100f90 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c19db10 con 0x7f9c2c101950 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53526/0 (socket says 172.21.15.44:53526) 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 -- 172.21.15.44:0/1531954072 learned_addr learned my addr 172.21.15.44:0/1531954072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3189e640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2c101950 0x7f9c2c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3089c640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c100f90 0x7f9c2c196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 -- 172.21.15.44:0/1531954072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2c101950 msgr2=0x7f9c2c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2c101950 0x7f9c2c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 -- 172.21.15.44:0/1531954072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c100f90 msgr2=0x7f9c2c196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c100f90 0x7f9c2c196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 -- 172.21.15.44:0/1531954072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c28009560 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c3109d640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c195d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005ab0 tx=0x7f9c280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28031d70 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c280044e0 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c19dd90 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28003b00 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.502+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c19e1d0 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.503+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c107370 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.505+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9c28004c50 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.506+0000 7f9c227fc640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9c180a1110 0x7f9c180a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.506+0000 7f9c3089c640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9c180a1110 0x7f9c180a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.506+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c280edf80 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.506+0000 7f9c3089c640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9c180a1110 0x7f9c180a35d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c005fd0 tx=0x7f9c1c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.509+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9c280b7430 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.684+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c2c106de0 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.685+0000 7f9c227fc640 1 -- 172.21.15.44:0/1531954072 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23900 (secure 0 0 0) 0x7f9c2804a090 con 0x7f9c2c0ffda0 2024-09-13T17:24:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:27.687 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:26.097662+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:27.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9c180a1110 msgr2=0x7f9c180a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9c180a1110 0x7f9c180a35d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c005fd0 tx=0x7f9c1c005850 comp rx=0 tx=0).stop 2024-09-13T17:24:27.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 msgr2=0x7f9c2c195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c195d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005ab0 tx=0x7f9c280042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 shutdown_connections 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9c180a1110 0x7f9c180a35d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2c101950 0x7f9c2c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c100f90 0x7f9c2c196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 --2- 172.21.15.44:0/1531954072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c2c0ffda0 0x7f9c2c195d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 >> 172.21.15.44:0/1531954072 conn(0x7f9c2c0fb520 msgr2=0x7f9c2c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 shutdown_connections 2024-09-13T17:24:27.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.687+0000 7f9c33301640 1 -- 172.21.15.44:0/1531954072 wait complete. 2024-09-13T17:24:27.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:27 smithi070 ceph-mon[34502]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T17:24:27.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:27 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2802013829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:27 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3234134479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:27 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/164826592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:27 smithi044 ceph-mon[31175]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T17:24:27.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:27 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2802013829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:27 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3234134479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:27 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/164826592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:24:21.722369+0000', 'modified': '2024-09-13T17:24:26.097886+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.100:6839/3416533473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 3416533473}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 3416533473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T17:24:27.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:24:27.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T17:24:27.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:27 smithi100 ceph-mon[35055]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T17:24:27.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:27 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2802013829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:27 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3234134479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:27 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/164826592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 -- 172.21.15.44:0/4263612442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 msgr2=0x7f9640105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 --2- 172.21.15.44:0/4263612442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f9640105510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9634009900 tx=0x7f963402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 -- 172.21.15.44:0/4263612442 shutdown_connections 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 --2- 172.21.15.44:0/4263612442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9640105a50 0x7f9640107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 --2- 172.21.15.44:0/4263612442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f9640105510 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 --2- 172.21.15.44:0/4263612442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96401007f0 0x7f9640102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 -- 172.21.15.44:0/4263612442 >> 172.21.15.44:0/4263612442 conn(0x7f96400fa410 msgr2=0x7f96400fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.802+0000 7f96454d7640 1 -- 172.21.15.44:0/4263612442 shutdown_connections 2024-09-13T17:24:27.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 -- 172.21.15.44:0/4263612442 wait complete. 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 Processor -- start 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 -- start start 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96401007f0 0x7f9640100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f96400fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9640105a50 0x7f96400fece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400ff440 con 0x7f9640103120 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400ff5b0 con 0x7f96401007f0 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f96454d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019fcf0 con 0x7f9640105a50 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.803+0000 7f963e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f96400fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f96400fe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53556/0 (socket says 172.21.15.44:53556) 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 -- 172.21.15.44:0/1788895804 learned_addr learned my addr 172.21.15.44:0/1788895804 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963effd640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96401007f0 0x7f9640100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963f7fe640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9640105a50 0x7f96400fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 -- 172.21.15.44:0/1788895804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9640105a50 msgr2=0x7f96400fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9640105a50 0x7f96400fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 -- 172.21.15.44:0/1788895804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96401007f0 msgr2=0x7f9640100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96401007f0 0x7f9640100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 -- 172.21.15.44:0/1788895804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009560 con 0x7f9640103120 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963f7fe640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9640105a50 0x7f96400fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963effd640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96401007f0 0x7f9640100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f963e7fc640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f96400fe7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9634009230 tx=0x7f96340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f9627fff640 1 -- 172.21.15.44:0/1788895804 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634031d70 con 0x7f9640103120 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f9627fff640 1 -- 172.21.15.44:0/1788895804 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96340044e0 con 0x7f9640103120 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964019ff40 con 0x7f9640103120 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f9627fff640 1 -- 172.21.15.44:0/1788895804 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634003b00 con 0x7f9640103120 2024-09-13T17:24:27.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.804+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401a0360 con 0x7f9640103120 2024-09-13T17:24:27.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.806+0000 7f9627fff640 1 -- 172.21.15.44:0/1788895804 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9634004c50 con 0x7f9640103120 2024-09-13T17:24:27.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 Processor -- start 2024-09-13T17:24:27.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 -- start start 2024-09-13T17:24:27.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed49900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49e40 0x55d11ed4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed51390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11eca47c0 con 0x55d11ed4eed0 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ecb3b70 con 0x55d11ed49500 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f96481f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ecb1300 con 0x55d11ed49e40 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f9627fff640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96280a1090 0x7f96280a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f963e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed51390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f963e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed51390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53570/0 (socket says 172.21.15.44:53570) 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f963e7fc640 1 -- 172.21.15.44:0/92887876 learned_addr learned my addr 172.21.15.44:0/92887876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f963f7fe640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96280a1090 0x7f96280a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f963effd640 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49e40 0x55d11ed4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f963f7fe640 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed49900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.807+0000 7f9627fff640 1 -- 172.21.15.44:0/1788895804 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f96340edc80 con 0x7f9640103120 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963e7fc640 1 -- 172.21.15.44:0/92887876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49e40 msgr2=0x55d11ed4e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963e7fc640 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49e40 0x55d11ed4e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963e7fc640 1 -- 172.21.15.44:0/92887876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 msgr2=0x55d11ed49900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963e7fc640 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed49900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963e7fc640 1 -- 172.21.15.44:0/92887876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11ec394b0 con 0x55d11ed4eed0 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963f7fe640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96280a1090 0x7f96280a3550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f9638009d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f963e7fc640 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed51390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96340eb530 tx=0x7f9634009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1951b23c1fd4609 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f96257fa640 1 -- 172.21.15.44:0/92887876 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340ba7e0 con 0x55d11ed4eed0 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f96257fa640 1 -- 172.21.15.44:0/92887876 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96340bbe90 con 0x55d11ed4eed0 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.808+0000 7f96257fa640 1 -- 172.21.15.44:0/92887876 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340b7590 con 0x55d11ed4eed0 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- 172.21.15.44:0/92887876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 msgr2=0x55d11ed51390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed51390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96340eb530 tx=0x7f9634009260 comp rx=0 tx=0).stop 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- 172.21.15.44:0/92887876 shutdown_connections 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49e40 0x55d11ed4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed51390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- 172.21.15.44:0/92887876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed49900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- 172.21.15.44:0/92887876 >> 172.21.15.44:0/92887876 conn(0x55d11ed44b70 msgr2=0x55d11ed44f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- 172.21.15.44:0/92887876 shutdown_connections 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- 172.21.15.44:0/92887876 wait complete. 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- start start 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 Processor -- start 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed52e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 0x55d11ed533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed4eed0 0x55d11ed5a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ecacbf0 con 0x55d11ed49e40 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ec6b250 con 0x55d11ed49500 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ec6aef0 con 0x55d11ed4eed0 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f96481f4740 10 client.0 _ll_register_callbacks cb 0x55d11ec9d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f963e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 0x55d11ed533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f963e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 0x55d11ed533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53580/0 (socket says 172.21.15.44:53580) 2024-09-13T17:24:27.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.809+0000 7f963e7fc640 1 -- 172.21.15.44:0/4116595314 learned_addr learned my addr 172.21.15.44:0/4116595314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963effd640 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed52e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963e7fc640 1 -- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed4eed0 msgr2=0x55d11ed5a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963e7fc640 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed4eed0 0x55d11ed5a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963e7fc640 1 -- 172.21.15.44:0/4116595314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 msgr2=0x55d11ed52e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963e7fc640 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed52e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963e7fc640 1 -- 172.21.15.44:0/4116595314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401a0360 con 0x55d11ed49e40 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f963e7fc640 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 0x55d11ed533a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f963402d020 tx=0x7f96340c3450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02c34cf0bd0f954 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f9622ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f9622ff5640 1 -- 172.21.15.44:0/4116595314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340bae80 con 0x55d11ed49e40 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f9622ff5640 1 -- 172.21.15.44:0/4116595314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9634003c60 con 0x55d11ed49e40 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d11e9a03c0 con 0x55d11ed49e40 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f9622ff5640 1 -- 172.21.15.44:0/4116595314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9634003e20 con 0x55d11ed49e40 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f96481f4740 10 client.14760 fetch_fsmap learned FSMap version 17 2024-09-13T17:24:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55d11ecab270 con 0x55d11ed49e40 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f9622ff5640 1 -- 172.21.15.44:0/4116595314 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96340d2080 con 0x55d11ed49e40 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f9622ff5640 1 -- 172.21.15.44:0/4116595314 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f96340ba470 con 0x55d11ed49e40 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f96481f4740 10 client.14760 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.810+0000 7f96481f4740 10 client.14760 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f96481f4740 20 client.14760 populate_metadata read hostname 'smithi044' 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f96481f4740 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11ed5eed0 0x55d11ed63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f96481f4740 4 client.14760 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d11e9b16c0 con 0x55d11ed5eed0 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f963effd640 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11ed5eed0 0x55d11ed63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f9622ff5640 10 client.14760 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.811+0000 7f963effd640 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11ed5eed0 0x55d11ed63050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f1d4fb79e80e89 server_cookie=a7da2d0f2a226cb3 in_seq=0 out_seq=0 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f9622ff5640 1 -- 172.21.15.44:0/4116595314 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d11e9b16c0 con 0x55d11ed5eed0 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f9622ff5640 10 client.14760 handle_command_reply: tid=0 2024-09-13T17:24:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 client.14760 shutdown 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 msgr2=0x55d11ed533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 0x55d11ed533a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f963402d020 tx=0x7f96340c3450 comp rx=0 tx=0).stop 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 shutdown_connections 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11ed5eed0 0x55d11ed63050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed4eed0 0x55d11ed5a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed49e40 0x55d11ed533a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f9622ff5640 0 client.14760 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 --2- 172.21.15.44:0/4116595314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49500 0x55d11ed52e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 >> 172.21.15.44:0/4116595314 conn(0x55d11ed44b70 msgr2=0x55d11ed46380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 shutdown_connections 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.812+0000 7f96481f4740 1 -- 172.21.15.44:0/4116595314 wait complete. 2024-09-13T17:24:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.813+0000 7f96481f4740 20 client.14760 trim_cache size 0 max 16384 2024-09-13T17:24:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 Processor -- start 2024-09-13T17:24:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 -- start start 2024-09-13T17:24:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed52510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ec30460 con 0x55d11ed4eed0 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ec6b250 con 0x55d11ed49e40 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f96481f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ed5afe0 con 0x55d11ed49500 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.827+0000 7f963f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed52510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed52510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53594/0 (socket says 172.21.15.44:53594) 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 -- 172.21.15.44:0/3348169004 learned_addr learned my addr 172.21.15.44:0/3348169004 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963effd640 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 -- 172.21.15.44:0/3348169004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 msgr2=0x55d11ed51fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed51fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 -- 172.21.15.44:0/3348169004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 msgr2=0x55d11ed5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963e7fc640 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed5a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 -- 172.21.15.44:0/3348169004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11ec4d5f0 con 0x55d11ed4eed0 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f963f7fe640 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed52510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9638014960 tx=0x7f963801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b00634b80ef029 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f9625ffb640 1 -- 172.21.15.44:0/3348169004 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638014110 con 0x55d11ed4eed0 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f9625ffb640 1 -- 172.21.15.44:0/3348169004 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f963801a040 con 0x55d11ed4eed0 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.828+0000 7f9625ffb640 1 -- 172.21.15.44:0/3348169004 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638029920 con 0x55d11ed4eed0 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- 172.21.15.44:0/3348169004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 msgr2=0x55d11ed52510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed52510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9638014960 tx=0x7f963801b000 comp rx=0 tx=0).stop 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- 172.21.15.44:0/3348169004 shutdown_connections 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11ed52510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- 172.21.15.44:0/3348169004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- 172.21.15.44:0/3348169004 >> 172.21.15.44:0/3348169004 conn(0x55d11ed44b70 msgr2=0x55d11ed4adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- 172.21.15.44:0/3348169004 shutdown_connections 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- 172.21.15.44:0/3348169004 wait complete. 2024-09-13T17:24:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- start start 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 Processor -- start 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed55730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11edb4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ed44890 con 0x55d11ed4eed0 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ecb3b70 con 0x55d11ed49e40 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ed55e40 con 0x55d11ed49500 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f96481f4740 10 client.0 _ll_register_callbacks cb 0x7f9634005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f963f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11edb4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f963f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11edb4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53610/0 (socket says 172.21.15.44:53610) 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.829+0000 7f963f7fe640 1 -- 172.21.15.44:0/3263386701 learned_addr learned my addr 172.21.15.44:0/3263386701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963f7fe640 1 -- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 msgr2=0x55d11ed551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963f7fe640 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963f7fe640 1 -- 172.21.15.44:0/3263386701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 msgr2=0x55d11ed55730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963e7fc640 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed55730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963f7fe640 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed55730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963f7fe640 1 -- 172.21.15.44:0/3263386701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d11ec72d00 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f963f7fe640 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11edb4d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9638009bf0 tx=0x7f9638004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7423771ab834fe2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f9622ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f9622ff5640 1 -- 172.21.15.44:0/3263386701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963802e070 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f9622ff5640 1 -- 172.21.15.44:0/3263386701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9638012b00 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96340bbe90 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f9622ff5640 1 -- 172.21.15.44:0/3263386701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9638014630 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f96481f4740 10 client.14766 fetch_fsmap learned FSMap version 17 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55d11ec59720 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.830+0000 7f9622ff5640 1 -- 172.21.15.44:0/3263386701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f963803e020 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f9622ff5640 1 -- 172.21.15.44:0/3263386701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9638020450 con 0x55d11ed4eed0 2024-09-13T17:24:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f96481f4740 10 client.14766 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f96481f4740 10 client.14766 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f96481f4740 20 client.14766 populate_metadata read hostname 'smithi044' 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f96481f4740 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11edb9f20 0x55d11edb68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f96481f4740 4 client.14766 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d11ea04d10 con 0x55d11edb9f20 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f963effd640 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11edb9f20 0x55d11edb68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f963effd640 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11edb9f20 0x55d11edb68c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa52963b7ec7fe4 server_cookie=f5a27779241d4212 in_seq=0 out_seq=0 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.831+0000 7f9622ff5640 10 client.14766 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f9622ff5640 1 -- 172.21.15.44:0/3263386701 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d11ea04d10 con 0x55d11edb9f20 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f9622ff5640 10 client.14766 handle_command_reply: tid=0 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 client.14766 shutdown 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 msgr2=0x55d11edb4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11edb4d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9638009bf0 tx=0x7f9638004c70 comp rx=0 tx=0).stop 2024-09-13T17:24:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 shutdown_connections 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d11edb9f20 0x55d11edb68c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d11ed4eed0 0x55d11edb4d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d11ed49e40 0x55d11ed55730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f9622ff5640 0 client.14766 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 --2- 172.21.15.44:0/3263386701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d11ed49500 0x55d11ed551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 >> 172.21.15.44:0/3263386701 conn(0x55d11ed44b70 msgr2=0x55d11ed4adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 shutdown_connections 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 1 -- 172.21.15.44:0/3263386701 wait complete. 2024-09-13T17:24:27.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96481f4740 20 client.14766 trim_cache size 0 max 16384 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24517, 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.10683095, 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 88, 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 19.002018320000001 2024-09-13T17:24:27.840 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:24:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96280a1090 msgr2=0x7f96280a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96280a1090 0x7f96280a3550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f9638009d60 comp rx=0 tx=0).stop 2024-09-13T17:24:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 msgr2=0x7f96400fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f96400fe7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9634009230 tx=0x7f96340042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 shutdown_connections 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f96280a1090 0x7f96280a3550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9640105a50 0x7f96400fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9640103120 0x7f96400fe7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 --2- 172.21.15.44:0/1788895804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96401007f0 0x7f9640100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 >> 172.21.15.44:0/1788895804 conn(0x7f96400fa410 msgr2=0x7f9640101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 shutdown_connections 2024-09-13T17:24:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:27.832+0000 7f96454d7640 1 -- 172.21.15.44:0/1788895804 wait complete. 2024-09-13T17:24:27.845 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:24:27.845 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:27.845 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-13T17:24:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:24:28.523 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:28 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/742910266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:28.523 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:28 smithi044 ceph-mon[31175]: pgmap v281: 129 pgs: 4 creating+peering, 111 active+clean, 14 unknown; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 9 op/s 2024-09-13T17:24:28.523 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:28 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1531954072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:28 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/742910266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:28 smithi070 ceph-mon[34502]: pgmap v281: 129 pgs: 4 creating+peering, 111 active+clean, 14 unknown; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 9 op/s 2024-09-13T17:24:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:28 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1531954072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:28 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/742910266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:28 smithi100 ceph-mon[35055]: pgmap v281: 129 pgs: 4 creating+peering, 111 active+clean, 14 unknown; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 9 op/s 2024-09-13T17:24:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:28 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1531954072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:29.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:29 smithi044 ceph-mon[31175]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T17:24:29.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:29 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:24:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:29 smithi070 ceph-mon[34502]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T17:24:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:29 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:24:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:29 smithi100 ceph-mon[35055]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T17:24:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:29 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 -- 172.21.15.44:0/927588643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd824100990 msgr2=0x7fd8241049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 --2- 172.21.15.44:0/927588643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd824100990 0x7fd8241049e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd814009cd0 tx=0x7fd81402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 -- 172.21.15.44:0/927588643 shutdown_connections 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 --2- 172.21.15.44:0/927588643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824105110 0x7fd82410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 --2- 172.21.15.44:0/927588643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd824100990 0x7fd8241049e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 --2- 172.21.15.44:0/927588643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8240fffe0 0x7fd8241003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 -- 172.21.15.44:0/927588643 >> 172.21.15.44:0/927588643 conn(0x7fd8240fbae0 msgr2=0x7fd8240fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 -- 172.21.15.44:0/927588643 shutdown_connections 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.924+0000 7fd82b094640 1 -- 172.21.15.44:0/927588643 wait complete. 2024-09-13T17:24:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 Processor -- start 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 -- start start 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 0x7fd824196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd824100990 0x7fd8241967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824105110 0x7fd82419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419b1a0 con 0x7fd8240fffe0 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419b310 con 0x7fd824100990 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd82b094640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419b480 con 0x7fd824105110 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 0x7fd824196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 0x7fd824196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48214/0 (socket says 172.21.15.44:48214) 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 -- 172.21.15.44:0/2850293967 learned_addr learned my addr 172.21.15.44:0/2850293967 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd829631640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824105110 0x7fd82419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd823fff640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd824100990 0x7fd8241967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 -- 172.21.15.44:0/2850293967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824105110 msgr2=0x7fd82419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824105110 0x7fd82419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 -- 172.21.15.44:0/2850293967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd824100990 msgr2=0x7fd8241967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd824100990 0x7fd8241967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.925+0000 7fd828e30640 1 -- 172.21.15.44:0/2850293967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009900 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.926+0000 7fd828e30640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 0x7fd824196260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00b4a0 tx=0x7fd80c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.926+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c010610 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.926+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82419b760 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.926+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd80c010c00 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.926+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c017ca0 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.926+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8241a3020 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.927+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8241014c0 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.928+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd80c010770 con 0x7fd8240fffe0 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.929+0000 7fd821ffb640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd8100a1080 0x7fd8100a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.929+0000 7fd823fff640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd8100a1080 0x7fd8100a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.929+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd80c0cc4b0 con 0x7fd8240fffe0 2024-09-13T17:24:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.930+0000 7fd823fff640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd8100a1080 0x7fd8100a3540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd824197810 tx=0x7fd814009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:29.933+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd80c095a50 con 0x7fd8240fffe0 2024-09-13T17:24:30.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.144+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd8241003c0 con 0x7fd8240fffe0 2024-09-13T17:24:30.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.144+0000 7fd821ffb640 1 -- 172.21.15.44:0/2850293967 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd80c099070 con 0x7fd8240fffe0 2024-09-13T17:24:30.146 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.146+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd8100a1080 msgr2=0x7fd8100a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.146+0000 7fd82b094640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd8100a1080 0x7fd8100a3540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd824197810 tx=0x7fd814009890 comp rx=0 tx=0).stop 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.146+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 msgr2=0x7fd824196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.146+0000 7fd82b094640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 0x7fd824196260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00b4a0 tx=0x7fd80c00b960 comp rx=0 tx=0).stop 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 shutdown_connections 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd8100a1080 0x7fd8100a3540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824105110 0x7fd82419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd824100990 0x7fd8241967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 --2- 172.21.15.44:0/2850293967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8240fffe0 0x7fd824196260 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 >> 172.21.15.44:0/2850293967 conn(0x7fd8240fbae0 msgr2=0x7fd8240fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 shutdown_connections 2024-09-13T17:24:30.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:30.147+0000 7fd82b094640 1 -- 172.21.15.44:0/2850293967 wait complete. 2024-09-13T17:24:30.462 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:30 smithi044 ceph-mon[31175]: pgmap v282: 129 pgs: 4 creating+peering, 121 active+clean, 4 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-13T17:24:30.463 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:30 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2850293967' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T17:24:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:30 smithi070 ceph-mon[34502]: pgmap v282: 129 pgs: 4 creating+peering, 121 active+clean, 4 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-13T17:24:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:30 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2850293967' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:24:30.762 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T17:24:30.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:30.763 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T17:24:30.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:30 smithi100 ceph-mon[35055]: pgmap v282: 129 pgs: 4 creating+peering, 121 active+clean, 4 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-13T17:24:30.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:30 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2850293967' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T17:24:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42298sec preferred_lft 42298sec 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:24:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T17:24:30.783 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:24:30.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:24:30.783 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:24:30.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:30.957 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:24:30.982 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T17:24:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:30.984 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T17:24:31.052 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T17:24:31.052 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:24:31.052 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-13T17:24:31.052 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-13T17:24:31.052 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:24:31.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.053 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:24:31.053 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:24:31.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T17:24:31.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T17:24:31.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T17:24:31.053 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:24:31.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.318 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T17:24:31.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.334 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-09-13T17:24:31.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.390 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:24:31.390 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:24:31.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T17:24:31.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T17:24:31.390 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:24:31.469 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T17:24:31.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T17:24:31.470 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:24:31.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.603 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:24:31.603 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:24:31.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T17:24:31.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:24:31.603 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:24:31.642 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T17:24:31.642 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:24:31.642 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:31.698 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T17:24:31.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:24:31.698 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:31.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-09-13T17:24:31.816 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:31 smithi044 ceph-mon[31175]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-13T17:24:31.820 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:24:31.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:24:31.886 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:24:31.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:24:31.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:31.888 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-09-13T17:24:31.944 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:24:31.944 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T17:24:31.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:31 smithi070 ceph-mon[34502]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-13T17:24:31.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-09-13T17:24:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:31 smithi100 ceph-mon[35055]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-13T17:24:32.030 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:24:32.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:24:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:24:32.057 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:24:32.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:32.058 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-09-13T17:24:32.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:2024-09-13T17:24:32.087+0000 7fa4d6328480 -1 init, newargv = 0x55c8aa0e2410 newargc=15 2024-09-13T17:24:32.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[74460]: starting ceph client 2024-09-13T17:24:32.094 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[74460]: starting fuse 2024-09-13T17:24:32.104 INFO:teuthology.orchestra.run.smithi044.stdout:120 2024-09-13T17:24:32.105 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:24:32.105 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-09-13T17:24:32.106 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-09-13T17:24:32.107 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-09-13T17:24:32.107 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:24:32.107 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:24:32.107 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.admin")) 2024-09-13T17:24:32.107 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-09-13T17:24:32.222 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.admin.74460.asok 2024-09-13T17:24:32.408 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74460.asok 2024-09-13T17:24:32.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:32.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74460.asok status 2024-09-13T17:24:32.549 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:24:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-09-13T17:24:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:24:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "admin", 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "74460", 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 14778, 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 14778 2024-09-13T17:24:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 545441566 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 545441566 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.14778 192.168.0.1:0/545441566", 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/545441566", 2024-09-13T17:24:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2024-09-13T17:24:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 17, 2024-09-13T17:24:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 88, 2024-09-13T17:24:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:24:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-09-13T17:24:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-09-13T17:24:32.552 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:24:32.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:32.558 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-09-13T17:24:32.558 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:32.576 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-09-13T17:24:32.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:32.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:32.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:32.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:32.643 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T17:24:32.713 INFO:teuthology.orchestra.run.smithi044.stdout:# file: . 2024-09-13T17:24:32.713 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T17:24:32.713 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:32.713 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:24:32.713 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:24:32.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 -- 172.21.15.44:0/1600048291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac101970 msgr2=0x7f54ac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 --2- 172.21.15.44:0/1600048291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac101970 0x7f54ac108200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009ca0 tx=0x7f54a402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 -- 172.21.15.44:0/1600048291 shutdown_connections 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 --2- 172.21.15.44:0/1600048291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54ac100fb0 0x7f54ac101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 --2- 172.21.15.44:0/1600048291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac101970 0x7f54ac108200 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 --2- 172.21.15.44:0/1600048291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54ac0ffdc0 0x7f54ac1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 -- 172.21.15.44:0/1600048291 >> 172.21.15.44:0/1600048291 conn(0x7f54ac0fb540 msgr2=0x7f54ac0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 -- 172.21.15.44:0/1600048291 shutdown_connections 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.913+0000 7f54b1533640 1 -- 172.21.15.44:0/1600048291 wait complete. 2024-09-13T17:24:32.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 Processor -- start 2024-09-13T17:24:32.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 -- start start 2024-09-13T17:24:32.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54ac0ffdc0 0x7f54ac195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:32.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 0x7f54ac1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:32.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54ac101970 0x7f54ac19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:32.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac19d7a0 con 0x7f54ac100fb0 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac19d910 con 0x7f54ac0ffdc0 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.914+0000 7f54b1533640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac19dad0 con 0x7f54ac101970 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 0x7f54ac1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 0x7f54ac1961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48238/0 (socket says 172.21.15.44:48238) 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 -- 172.21.15.44:0/4112507102 learned_addr learned my addr 172.21.15.44:0/4112507102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aaffd640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54ac0ffdc0 0x7f54ac195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 -- 172.21.15.44:0/4112507102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54ac101970 msgr2=0x7f54ac19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54ab7fe640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54ac101970 0x7f54ac19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54ac101970 0x7f54ac19d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 -- 172.21.15.44:0/4112507102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54ac0ffdc0 msgr2=0x7f54ac195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54ac0ffdc0 0x7f54ac195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 -- 172.21.15.44:0/4112507102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4009900 con 0x7f54ac100fb0 2024-09-13T17:24:32.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54aa7fc640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 0x7f54ac1961d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f54a000d770 tx=0x7f54a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:32.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0009c30 con 0x7f54ac100fb0 2024-09-13T17:24:32.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54a000de80 con 0x7f54ac100fb0 2024-09-13T17:24:32.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac19ddb0 con 0x7f54ac100fb0 2024-09-13T17:24:32.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0004d40 con 0x7f54ac100fb0 2024-09-13T17:24:32.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.915+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac19e210 con 0x7f54ac100fb0 2024-09-13T17:24:32.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.916+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac1072f0 con 0x7f54ac100fb0 2024-09-13T17:24:32.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.918+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f54a0004ea0 con 0x7f54ac100fb0 2024-09-13T17:24:32.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.919+0000 7f5493fff640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f54940a1090 0x7f54940a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:32.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.919+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54a00cc980 con 0x7f54ac100fb0 2024-09-13T17:24:32.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.919+0000 7f54aaffd640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f54940a1090 0x7f54940a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:32.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.920+0000 7f54aaffd640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f54940a1090 0x7f54940a3550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54ac196b60 tx=0x7f549c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:32.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:32.923+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f54a0095ec0 con 0x7f54ac100fb0 2024-09-13T17:24:33.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.095+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54ac02c480 con 0x7f54ac100fb0 2024-09-13T17:24:33.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.096+0000 7f5493fff640 1 -- 172.21.15.44:0/4112507102 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23900 (secure 0 0 0) 0x7f54a0017020 con 0x7f54ac100fb0 2024-09-13T17:24:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:33.098 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:26.097662+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f54940a1090 msgr2=0x7f54940a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f54940a1090 0x7f54940a3550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54ac196b60 tx=0x7f549c00b390 comp rx=0 tx=0).stop 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 msgr2=0x7f54ac1961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 0x7f54ac1961d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f54a000d770 tx=0x7f54a000dc30 comp rx=0 tx=0).stop 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 shutdown_connections 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f54940a1090 0x7f54940a3550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54ac101970 0x7f54ac19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac100fb0 0x7f54ac1961d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 --2- 172.21.15.44:0/4112507102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54ac0ffdc0 0x7f54ac195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.098+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 >> 172.21.15.44:0/4112507102 conn(0x7f54ac0fb540 msgr2=0x7f54ac1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.099+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 shutdown_connections 2024-09-13T17:24:33.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.099+0000 7f54b1533640 1 -- 172.21.15.44:0/4112507102 wait complete. 2024-09-13T17:24:33.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:24:33.111 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:24:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 -- 172.21.15.44:0/753123357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40fec60 msgr2=0x7fb6e40ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/753123357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40fec60 0x7fb6e40ff060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009900 tx=0x7fb6e002a410 comp rx=0 tx=0).stop 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 -- 172.21.15.44:0/753123357 shutdown_connections 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/753123357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e40ffe50 0x7fb6e41002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/753123357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e4100810 0x7fb6e41070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/753123357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40fec60 0x7fb6e40ff060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 -- 172.21.15.44:0/753123357 >> 172.21.15.44:0/753123357 conn(0x7fb6e40fa3e0 msgr2=0x7fb6e40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 -- 172.21.15.44:0/753123357 shutdown_connections 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.216+0000 7fb6ebf53640 1 -- 172.21.15.44:0/753123357 wait complete. 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 Processor -- start 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 -- start start 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e40fec60 0x7fb6e4102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 0x7fb6e4101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4100810 0x7fb6e4101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4101ed0 con 0x7fb6e40ffe50 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4102040 con 0x7fb6e40fec60 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ebf53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e419fdb0 con 0x7fb6e4100810 2024-09-13T17:24:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 0x7fb6e4101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 0x7fb6e4101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48268/0 (socket says 172.21.15.44:48268) 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 -- 172.21.15.44:0/611942864 learned_addr learned my addr 172.21.15.44:0/611942864 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e9cef640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e40fec60 0x7fb6e4102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6ea4f0640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4100810 0x7fb6e4101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 -- 172.21.15.44:0/611942864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4100810 msgr2=0x7fb6e4101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4100810 0x7fb6e4101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 -- 172.21.15.44:0/611942864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e40fec60 msgr2=0x7fb6e4102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e40fec60 0x7fb6e4102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 -- 172.21.15.44:0/611942864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0009560 con 0x7fb6e40ffe50 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.217+0000 7fb6e94ee640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 0x7fb6e4101230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800ea20 tx=0x7fb6d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.218+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8009720 con 0x7fb6e40ffe50 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.218+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e41a0060 con 0x7fb6e40ffe50 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.218+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6d800cd80 con 0x7fb6e40ffe50 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.218+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8010430 con 0x7fb6e40ffe50 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.218+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e41a0520 con 0x7fb6e40ffe50 2024-09-13T17:24:33.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.219+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e410a420 con 0x7fb6e40ffe50 2024-09-13T17:24:33.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.221+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb6d8004f30 con 0x7fb6e40ffe50 2024-09-13T17:24:33.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.222+0000 7fb6d6ffd640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb6d00a1090 0x7fb6d00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.222+0000 7fb6e9cef640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb6d00a1090 0x7fb6d00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.222+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb6d8014070 con 0x7fb6e40ffe50 2024-09-13T17:24:33.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.223+0000 7fb6e9cef640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb6d00a1090 0x7fb6d00a3550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0006010 tx=0x7fb6e0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.226+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb6d8096540 con 0x7fb6e40ffe50 2024-09-13T17:24:33.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.398+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6e41a07a0 con 0x7fb6e40ffe50 2024-09-13T17:24:33.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.399+0000 7fb6d6ffd640 1 -- 172.21.15.44:0/611942864 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6d8099b60 con 0x7fb6e40ffe50 2024-09-13T17:24:33.400 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:24:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb6d00a1090 msgr2=0x7fb6d00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb6d00a1090 0x7fb6d00a3550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0006010 tx=0x7fb6e0005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 msgr2=0x7fb6e4101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 0x7fb6e4101230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800ea20 tx=0x7fb6d800ee20 comp rx=0 tx=0).stop 2024-09-13T17:24:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 shutdown_connections 2024-09-13T17:24:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb6d00a1090 0x7fb6d00a3550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e4100810 0x7fb6e4101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e40ffe50 0x7fb6e4101230 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 --2- 172.21.15.44:0/611942864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e40fec60 0x7fb6e4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 >> 172.21.15.44:0/611942864 conn(0x7fb6e40fa3e0 msgr2=0x7fb6e4067650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:33.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 shutdown_connections 2024-09-13T17:24:33.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.401+0000 7fb6ebf53640 1 -- 172.21.15.44:0/611942864 wait complete. 2024-09-13T17:24:33.412 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:33 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4112507102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:33.413 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:24:33.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:33 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4112507102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 -- 172.21.15.44:0/3065706764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940fec60 msgr2=0x7ff7940ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3065706764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940fec60 0x7ff7940ff060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff784009ca0 tx=0x7ff78402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 -- 172.21.15.44:0/3065706764 shutdown_connections 2024-09-13T17:24:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3065706764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 0x7ff7941070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3065706764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7940ffe50 0x7ff7941002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3065706764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940fec60 0x7ff7940ff060 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.517+0000 7ff799dd4640 1 -- 172.21.15.44:0/3065706764 >> 172.21.15.44:0/3065706764 conn(0x7ff7940fa3e0 msgr2=0x7ff7940fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.518+0000 7ff799dd4640 1 -- 172.21.15.44:0/3065706764 shutdown_connections 2024-09-13T17:24:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.518+0000 7ff799dd4640 1 -- 172.21.15.44:0/3065706764 wait complete. 2024-09-13T17:24:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.518+0000 7ff799dd4640 1 Processor -- start 2024-09-13T17:24:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.518+0000 7ff799dd4640 1 -- start start 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.518+0000 7ff799dd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7940fec60 0x7ff794195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.518+0000 7ff799dd4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 0x7ff7941962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff799dd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 0x7ff79419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff799dd4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79419d800 con 0x7ff7940ffe50 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff799dd4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79419d970 con 0x7ff7940fec60 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff799dd4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79419db30 con 0x7ff794100810 2024-09-13T17:24:33.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 0x7ff7941962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 0x7ff7941962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48282/0 (socket says 172.21.15.44:48282) 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 -- 172.21.15.44:0/3509161722 learned_addr learned my addr 172.21.15.44:0/3509161722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff7937fe640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7940fec60 0x7ff794195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff793fff640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 0x7ff79419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 -- 172.21.15.44:0/3509161722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 msgr2=0x7ff79419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 0x7ff79419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 -- 172.21.15.44:0/3509161722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7940fec60 msgr2=0x7ff794195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7940fec60 0x7ff794195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 -- 172.21.15.44:0/3509161722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784009900 con 0x7ff7940ffe50 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff793fff640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 0x7ff79419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff792ffd640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 0x7ff7941962b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c890 tx=0x7ff78800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788004220 con 0x7ff7940ffe50 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff788007e40 con 0x7ff7940ffe50 2024-09-13T17:24:33.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79419de10 con 0x7ff7940ffe50 2024-09-13T17:24:33.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.519+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788004a00 con 0x7ff7940ffe50 2024-09-13T17:24:33.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.520+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79419e230 con 0x7ff7940ffe50 2024-09-13T17:24:33.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.520+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79410a3b0 con 0x7ff7940ffe50 2024-09-13T17:24:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:33 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4112507102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:33.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.522+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff7880079b0 con 0x7ff7940ffe50 2024-09-13T17:24:33.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.522+0000 7ff790ff9640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff77c0a1090 0x7ff77c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.522+0000 7ff7937fe640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff77c0a1090 0x7ff77c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.522+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7880ccb90 con 0x7ff7940ffe50 2024-09-13T17:24:33.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.523+0000 7ff7937fe640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff77c0a1090 0x7ff77c0a3550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff784005e30 tx=0x7ff784009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:33.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.526+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff7880960a0 con 0x7ff7940ffe50 2024-09-13T17:24:33.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.699+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff79419e6f0 con 0x7ff7940ffe50 2024-09-13T17:24:33.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.699+0000 7ff790ff9640 1 -- 172.21.15.44:0/3509161722 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff78801d0a0 con 0x7ff7940ffe50 2024-09-13T17:24:33.700 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff77c0a1090 msgr2=0x7ff77c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff77c0a1090 0x7ff77c0a3550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff784005e30 tx=0x7ff784009800 comp rx=0 tx=0).stop 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 msgr2=0x7ff7941962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 0x7ff7941962b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c890 tx=0x7ff78800cd50 comp rx=0 tx=0).stop 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 shutdown_connections 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff77c0a1090 0x7ff77c0a3550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff794100810 0x7ff79419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7940ffe50 0x7ff7941962b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 --2- 172.21.15.44:0/3509161722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7940fec60 0x7ff794195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 >> 172.21.15.44:0/3509161722 conn(0x7ff7940fa3e0 msgr2=0x7ff7940675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:33.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 shutdown_connections 2024-09-13T17:24:33.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.701+0000 7ff799dd4640 1 -- 172.21.15.44:0/3509161722 wait complete. 2024-09-13T17:24:33.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 -- 172.21.15.44:0/23669944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80101950 msgr2=0x7f1f801081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 --2- 172.21.15.44:0/23669944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80101950 0x7f1f801081e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009ca0 tx=0x7f1f7802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 -- 172.21.15.44:0/23669944 shutdown_connections 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 --2- 172.21.15.44:0/23669944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80101950 0x7f1f801081e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 --2- 172.21.15.44:0/23669944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f80100f90 0x7f1f80101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 --2- 172.21.15.44:0/23669944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f800ffda0 0x7f1f801001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 -- 172.21.15.44:0/23669944 >> 172.21.15.44:0/23669944 conn(0x7f1f800fb520 msgr2=0x7f1f800fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 -- 172.21.15.44:0/23669944 shutdown_connections 2024-09-13T17:24:33.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.820+0000 7f1f87965640 1 -- 172.21.15.44:0/23669944 wait complete. 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 Processor -- start 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 -- start start 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f800ffda0 0x7f1f80195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 0x7f1f80196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f80101950 0x7f1f8019d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019d710 con 0x7f1f80100f90 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019d880 con 0x7f1f800ffda0 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019da40 con 0x7f1f80101950 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 0x7f1f80196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 0x7f1f80196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48312/0 (socket says 172.21.15.44:48312) 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 -- 172.21.15.44:0/2164441972 learned_addr learned my addr 172.21.15.44:0/2164441972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f85701640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f800ffda0 0x7f1f80195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f85f02640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f80101950 0x7f1f8019d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 -- 172.21.15.44:0/2164441972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f80101950 msgr2=0x7f1f8019d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f80101950 0x7f1f8019d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 -- 172.21.15.44:0/2164441972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f800ffda0 msgr2=0x7f1f80195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f800ffda0 0x7f1f80195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 -- 172.21.15.44:0/2164441972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009900 con 0x7f1f80100f90 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f84f00640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 0x7f1f80196170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9f0 tx=0x7f1f7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70009700 con 0x7f1f80100f90 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f7000cd80 con 0x7f1f80100f90 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.821+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8019dd20 con 0x7f1f80100f90 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.822+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f700187e0 con 0x7f1f80100f90 2024-09-13T17:24:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.822+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8019e1d0 con 0x7f1f80100f90 2024-09-13T17:24:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.822+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f80107230 con 0x7f1f80100f90 2024-09-13T17:24:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.824+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1f70010430 con 0x7f1f80100f90 2024-09-13T17:24:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.824+0000 7f1f767fc640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f6c0a1110 0x7f1f6c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.824+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1f70014070 con 0x7f1f80100f90 2024-09-13T17:24:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.824+0000 7f1f85701640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f6c0a1110 0x7f1f6c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.825+0000 7f1f85701640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f6c0a1110 0x7f1f6c0a35d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f80197100 tx=0x7f1f7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:33.828+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1f70052e20 con 0x7f1f80100f90 2024-09-13T17:24:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.002+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f8002c480 con 0x7f1f80100f90 2024-09-13T17:24:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.002+0000 7f1f767fc640 1 -- 172.21.15.44:0/2164441972 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f700963b0 con 0x7f1f80100f90 2024-09-13T17:24:34.004 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.004+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f6c0a1110 msgr2=0x7f1f6c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.004+0000 7f1f87965640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f6c0a1110 0x7f1f6c0a35d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f80197100 tx=0x7f1f7c00b390 comp rx=0 tx=0).stop 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 msgr2=0x7f1f80196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 0x7f1f80196170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9f0 tx=0x7f1f7000eeb0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 shutdown_connections 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1f6c0a1110 0x7f1f6c0a35d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f80101950 0x7f1f8019d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f80100f90 0x7f1f80196170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 --2- 172.21.15.44:0/2164441972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f800ffda0 0x7f1f80195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 >> 172.21.15.44:0/2164441972 conn(0x7f1f800fb520 msgr2=0x7f1f80104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 shutdown_connections 2024-09-13T17:24:34.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.005+0000 7f1f87965640 1 -- 172.21.15.44:0/2164441972 wait complete. 2024-09-13T17:24:34.016 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:24:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 -- 172.21.15.44:0/3422661125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994101950 msgr2=0x7fa9941081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 --2- 172.21.15.44:0/3422661125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994101950 0x7fa9941081e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009ca0 tx=0x7fa98c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 -- 172.21.15.44:0/3422661125 shutdown_connections 2024-09-13T17:24:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 --2- 172.21.15.44:0/3422661125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994101950 0x7fa9941081e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 --2- 172.21.15.44:0/3422661125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa994100f90 0x7fa994101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 --2- 172.21.15.44:0/3422661125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9940ffda0 0x7fa9941001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 -- 172.21.15.44:0/3422661125 >> 172.21.15.44:0/3422661125 conn(0x7fa9940fb520 msgr2=0x7fa9940fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 -- 172.21.15.44:0/3422661125 shutdown_connections 2024-09-13T17:24:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 -- 172.21.15.44:0/3422661125 wait complete. 2024-09-13T17:24:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 Processor -- start 2024-09-13T17:24:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.120+0000 7fa999a8f640 1 -- start start 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa999a8f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940ffda0 0x7fa994195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa999a8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 0x7fa994196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa999a8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994101950 0x7fa99419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa999a8f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99419d730 con 0x7fa994100f90 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa999a8f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99419d8a0 con 0x7fa9940ffda0 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa999a8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99419da60 con 0x7fa994101950 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 0x7fa994196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 0x7fa994196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48326/0 (socket says 172.21.15.44:48326) 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 -- 172.21.15.44:0/2308968662 learned_addr learned my addr 172.21.15.44:0/2308968662 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa9937fe640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940ffda0 0x7fa994195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa993fff640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994101950 0x7fa99419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 -- 172.21.15.44:0/2308968662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994101950 msgr2=0x7fa99419d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994101950 0x7fa99419d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 -- 172.21.15.44:0/2308968662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940ffda0 msgr2=0x7fa994195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940ffda0 0x7fa994195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 -- 172.21.15.44:0/2308968662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009900 con 0x7fa994100f90 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa9937fe640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940ffda0 0x7fa994195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa992ffd640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 0x7fa994196190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa98800d950 tx=0x7fa98800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.121+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988009c30 con 0x7fa994100f90 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.122+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa988009d90 con 0x7fa994100f90 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.122+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99419dd40 con 0x7fa994100f90 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.122+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800f620 con 0x7fa994100f90 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.122+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99419e1a0 con 0x7fa994100f90 2024-09-13T17:24:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.123+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa994107230 con 0x7fa994100f90 2024-09-13T17:24:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.124+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa9880109f0 con 0x7fa994100f90 2024-09-13T17:24:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.124+0000 7fa990ff9640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa97c0a1090 0x7fa97c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.125+0000 7fa9937fe640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa97c0a1090 0x7fa97c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.125+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9880cc710 con 0x7fa994100f90 2024-09-13T17:24:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.125+0000 7fa9937fe640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa97c0a1090 0x7fa97c0a3550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9840058c0 tx=0x7fa98400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.128+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa988095cd0 con 0x7fa994100f90 2024-09-13T17:24:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.302+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa99402c480 con 0x7fa994100f90 2024-09-13T17:24:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.302+0000 7fa990ff9640 1 -- 172.21.15.44:0/2308968662 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa988097c00 con 0x7fa994100f90 2024-09-13T17:24:34.304 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa97c0a1090 msgr2=0x7fa97c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa97c0a1090 0x7fa97c0a3550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9840058c0 tx=0x7fa98400a2d0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 msgr2=0x7fa994196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 0x7fa994196190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa98800d950 tx=0x7fa98800de10 comp rx=0 tx=0).stop 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 shutdown_connections 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa97c0a1090 0x7fa97c0a3550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994101950 0x7fa99419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994100f90 0x7fa994196190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 --2- 172.21.15.44:0/2308968662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940ffda0 0x7fa994195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.304+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 >> 172.21.15.44:0/2308968662 conn(0x7fa9940fb520 msgr2=0x7fa994104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.305+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 shutdown_connections 2024-09-13T17:24:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.305+0000 7fa999a8f640 1 -- 172.21.15.44:0/2308968662 wait complete. 2024-09-13T17:24:34.315 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/611942864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.316 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:34 smithi044 ceph-mon[31175]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-09-13T17:24:34.316 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3509161722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.316 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2164441972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:24:34.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/611942864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:34 smithi100 ceph-mon[35055]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-09-13T17:24:34.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3509161722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2164441972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 -- 172.21.15.44:0/1337335312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0105a90 msgr2=0x7f44b0107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 --2- 172.21.15.44:0/1337335312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0105a90 0x7f44b0107f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009be0 tx=0x7f44a802a6c0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 -- 172.21.15.44:0/1337335312 shutdown_connections 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 --2- 172.21.15.44:0/1337335312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0105a90 0x7f44b0107f00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 --2- 172.21.15.44:0/1337335312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b0103160 0x7f44b0105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 --2- 172.21.15.44:0/1337335312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0100830 0x7f44b0102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 -- 172.21.15.44:0/1337335312 >> 172.21.15.44:0/1337335312 conn(0x7f44b00fa430 msgr2=0x7f44b00fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 -- 172.21.15.44:0/1337335312 shutdown_connections 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.419+0000 7f44b71db640 1 -- 172.21.15.44:0/1337335312 wait complete. 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 Processor -- start 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 -- start start 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 0x7f44b01001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0103160 0x7f44b00fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b0105a90 0x7f44b00fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00ff480 con 0x7f44b0105a90 2024-09-13T17:24:34.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00ff5f0 con 0x7f44b0100830 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b71db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b019fdf0 con 0x7f44b0103160 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 0x7f44b01001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b5778640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b0105a90 0x7f44b00fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 0x7f44b01001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:50130/0 (socket says 172.21.15.44:50130) 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 -- 172.21.15.44:0/1038707986 learned_addr learned my addr 172.21.15.44:0/1038707986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44affff640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0103160 0x7f44b00fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 -- 172.21.15.44:0/1038707986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0103160 msgr2=0x7f44b00fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0103160 0x7f44b00fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 -- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b0105a90 msgr2=0x7f44b00fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b0105a90 0x7f44b00fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 -- 172.21.15.44:0/1038707986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009560 con 0x7f44b0100830 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44affff640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0103160 0x7f44b00fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.420+0000 7f44b4f77640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 0x7f44b01001a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44a000d7a0 tx=0x7f44a000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:34.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.421+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0009c30 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.421+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44a000deb0 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.421+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a8009840 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.421+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0004d60 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.421+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b01a0420 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.422+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b00fbf20 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.423+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f44a0004ec0 con 0x7f44b0100830 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.423+0000 7f44adffb640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f44980a1110 0x7f44980a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.424+0000 7f44affff640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f44980a1110 0x7f44980a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.424+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44a00cc530 con 0x7f44b0100830 2024-09-13T17:24:34.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.424+0000 7f44affff640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f44980a1110 0x7f44980a35d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44b00fff10 tx=0x7f44a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:34.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.428+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f44a0095a70 con 0x7f44b0100830 2024-09-13T17:24:34.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/611942864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:34 smithi070 ceph-mon[34502]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-09-13T17:24:34.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3509161722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2164441972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:24:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.612+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44b010b560 con 0x7f44b0100830 2024-09-13T17:24:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.612+0000 7f44adffb640 1 -- 172.21.15.44:0/1038707986 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+590 (secure 0 0 0) 0x7f44a0099090 con 0x7f44b0100830 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2209604760 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/815607991 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1683772085 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1586278527 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/771352345 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2038542363 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:34.614 INFO:teuthology.orchestra.run.smithi044.stderr:listed 10 entries 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.614+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f44980a1110 msgr2=0x7f44980a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f44980a1110 0x7f44980a35d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44b00fff10 tx=0x7f44a4009260 comp rx=0 tx=0).stop 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 msgr2=0x7f44b01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 0x7f44b01001a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44a000d7a0 tx=0x7f44a000dc60 comp rx=0 tx=0).stop 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 shutdown_connections 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f44980a1110 0x7f44980a35d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b0103160 0x7f44b00fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b0105a90 0x7f44b00fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 --2- 172.21.15.44:0/1038707986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b0100830 0x7f44b01001a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 >> 172.21.15.44:0/1038707986 conn(0x7f44b00fa430 msgr2=0x7f44b0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 shutdown_connections 2024-09-13T17:24:34.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.615+0000 7f44b71db640 1 -- 172.21.15.44:0/1038707986 wait complete. 2024-09-13T17:24:34.627 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:24:34.627 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:24:34.644 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 -- 172.21.15.44:0/144586540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac301013d0 msgr2=0x7fac301017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 --2- 172.21.15.44:0/144586540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac301013d0 0x7fac301017d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac20009930 tx=0x7fac2002a420 comp rx=0 tx=0).stop 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 -- 172.21.15.44:0/144586540 shutdown_connections 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 --2- 172.21.15.44:0/144586540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac300fde40 0x7fac300fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 --2- 172.21.15.44:0/144586540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac300fe800 0x7fac301083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 --2- 172.21.15.44:0/144586540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac301013d0 0x7fac301017d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 -- 172.21.15.44:0/144586540 >> 172.21.15.44:0/144586540 conn(0x7fac300f9bb0 msgr2=0x7fac300fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 -- 172.21.15.44:0/144586540 shutdown_connections 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.786+0000 7fac34e4f640 1 -- 172.21.15.44:0/144586540 wait complete. 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 Processor -- start 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 -- start start 2024-09-13T17:24:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac300fde40 0x7fac30198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 0x7fac30198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac301013d0 0x7fac3019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3019fb00 con 0x7fac300fe800 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3019fc70 con 0x7fac301013d0 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac34e4f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3019fe30 con 0x7fac300fde40 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 0x7fac30198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 0x7fac30198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48350/0 (socket says 172.21.15.44:48350) 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.787+0000 7fac27fff640 1 -- 172.21.15.44:0/4026422531 learned_addr learned my addr 172.21.15.44:0/4026422531 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac2ed9d640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac301013d0 0x7fac3019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac2e59c640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac300fde40 0x7fac30198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac27fff640 1 -- 172.21.15.44:0/4026422531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac300fde40 msgr2=0x7fac30198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac27fff640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac300fde40 0x7fac30198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac27fff640 1 -- 172.21.15.44:0/4026422531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac301013d0 msgr2=0x7fac3019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac27fff640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac301013d0 0x7fac3019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac27fff640 1 -- 172.21.15.44:0/4026422531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20009560 con 0x7fac300fe800 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac2ed9d640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac301013d0 0x7fac3019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac27fff640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 0x7fac30198550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fac1800d770 tx=0x7fac1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18009c30 con 0x7fac300fe800 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac1800de80 con 0x7fac300fe800 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac301a0110 con 0x7fac300fe800 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004d40 con 0x7fac300fe800 2024-09-13T17:24:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.788+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac301a0620 con 0x7fac300fe800 2024-09-13T17:24:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.789+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac301924f0 con 0x7fac300fe800 2024-09-13T17:24:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.790+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fac18004ea0 con 0x7fac300fe800 2024-09-13T17:24:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.791+0000 7fac277fe640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac1c0a1110 0x7fac1c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.791+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fac180cc580 con 0x7fac300fe800 2024-09-13T17:24:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.791+0000 7fac2e59c640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac1c0a1110 0x7fac1c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.791+0000 7fac2e59c640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac1c0a1110 0x7fac1c0a35d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac20009900 tx=0x7fac20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.795+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fac18095b20 con 0x7fac300fe800 2024-09-13T17:24:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.971+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac3010b580 con 0x7fac300fe800 2024-09-13T17:24:34.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.972+0000 7fac277fe640 1 -- 172.21.15.44:0/4026422531 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23900 (secure 0 0 0) 0x7fac18099140 con 0x7fac300fe800 2024-09-13T17:24:34.974 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:34.974 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:26.097662+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac1c0a1110 msgr2=0x7fac1c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac1c0a1110 0x7fac1c0a35d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac20009900 tx=0x7fac20000bc0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 msgr2=0x7fac30198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 0x7fac30198550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fac1800d770 tx=0x7fac1800dc30 comp rx=0 tx=0).stop 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 shutdown_connections 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fac1c0a1110 0x7fac1c0a35d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac301013d0 0x7fac3019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac300fe800 0x7fac30198550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 --2- 172.21.15.44:0/4026422531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac300fde40 0x7fac30198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.974+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 >> 172.21.15.44:0/4026422531 conn(0x7fac300f9bb0 msgr2=0x7fac30106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.975+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 shutdown_connections 2024-09-13T17:24:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:34.975+0000 7fac34e4f640 1 -- 172.21.15.44:0/4026422531 wait complete. 2024-09-13T17:24:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:24:34.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 -- 172.21.15.44:0/2935053592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 msgr2=0x7f3ab4107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/2935053592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab4107ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 -- 172.21.15.44:0/2935053592 shutdown_connections 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/2935053592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab4107ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/2935053592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab41031c0 0x7f3ab41055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/2935053592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab4100890 0x7f3ab4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 -- 172.21.15.44:0/2935053592 >> 172.21.15.44:0/2935053592 conn(0x7f3ab40fa430 msgr2=0x7f3ab40fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 -- 172.21.15.44:0/2935053592 shutdown_connections 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.091+0000 7f3abc3f4640 1 -- 172.21.15.44:0/2935053592 wait complete. 2024-09-13T17:24:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 Processor -- start 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 -- start start 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab4100890 0x7f3ab4195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab41031c0 0x7f3ab4196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab419d810 con 0x7f3ab4105af0 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab419d980 con 0x7f3ab41031c0 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3abc3f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab419db40 con 0x7f3ab4100890 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3aba991640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3aba991640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab419d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48364/0 (socket says 172.21.15.44:48364) 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3aba991640 1 -- 172.21.15.44:0/3765853841 learned_addr learned my addr 172.21.15.44:0/3765853841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3aba190640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab4100890 0x7f3ab4195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.092+0000 7f3ab998f640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab41031c0 0x7f3ab4196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aba991640 1 -- 172.21.15.44:0/3765853841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab4100890 msgr2=0x7f3ab4195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aba991640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab4100890 0x7f3ab4195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aba991640 1 -- 172.21.15.44:0/3765853841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab41031c0 msgr2=0x7f3ab4196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aba991640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab41031c0 0x7f3ab4196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aba991640 1 -- 172.21.15.44:0/3765853841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac009900 con 0x7f3ab4105af0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aba991640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab419d2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3aac02aeb0 tx=0x7f3aac030fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac003b70 con 0x7f3ab4105af0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3aac040070 con 0x7f3ab4105af0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab419ddc0 con 0x7f3ab4105af0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004360 con 0x7f3ab4105af0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.093+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab419e210 con 0x7f3ab4105af0 2024-09-13T17:24:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.094+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab40fbf20 con 0x7f3ab4105af0 2024-09-13T17:24:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.095+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3aac042030 con 0x7f3ab4105af0 2024-09-13T17:24:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.096+0000 7f3aab7fe640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3aa00a1090 0x7f3aa00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.096+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3aac0ee0a0 con 0x7f3ab4105af0 2024-09-13T17:24:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.096+0000 7f3aba190640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3aa00a1090 0x7f3aa00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.096+0000 7f3aba190640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3aa00a1090 0x7f3aa00a3550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4197200 tx=0x7f3ab000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.100+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3aac0b75b0 con 0x7f3ab4105af0 2024-09-13T17:24:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2308968662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:24:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1038707986' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:24:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4026422531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.275+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ab402c480 con 0x7f3ab4105af0 2024-09-13T17:24:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.275+0000 7f3aab7fe640 1 -- 172.21.15.44:0/3765853841 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3aac0b94e0 con 0x7f3ab4105af0 2024-09-13T17:24:35.276 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:24:35.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3aa00a1090 msgr2=0x7f3aa00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3aa00a1090 0x7f3aa00a3550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4197200 tx=0x7f3ab000b410 comp rx=0 tx=0).stop 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 msgr2=0x7f3ab419d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab419d2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3aac02aeb0 tx=0x7f3aac030fa0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 shutdown_connections 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3aa00a1090 0x7f3aa00a3550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab4105af0 0x7f3ab419d2a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab41031c0 0x7f3ab4196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 --2- 172.21.15.44:0/3765853841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab4100890 0x7f3ab4195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 >> 172.21.15.44:0/3765853841 conn(0x7f3ab40fa430 msgr2=0x7f3ab41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 shutdown_connections 2024-09-13T17:24:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.278+0000 7f3abc3f4640 1 -- 172.21.15.44:0/3765853841 wait complete. 2024-09-13T17:24:35.288 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2308968662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:24:35.289 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1038707986' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:24:35.289 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4026422531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:35.290 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:24:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.394+0000 7fda7a37d640 1 -- 172.21.15.44:0/2270287488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74103140 msgr2=0x7fda74105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.394+0000 7fda7a37d640 1 --2- 172.21.15.44:0/2270287488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74103140 0x7fda74105530 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009900 tx=0x7fda5c02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- 172.21.15.44:0/2270287488 shutdown_connections 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 --2- 172.21.15.44:0/2270287488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda74105a70 0x7fda74107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 --2- 172.21.15.44:0/2270287488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74103140 0x7fda74105530 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 --2- 172.21.15.44:0/2270287488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda74100810 0x7fda74102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- 172.21.15.44:0/2270287488 >> 172.21.15.44:0/2270287488 conn(0x7fda740fa3f0 msgr2=0x7fda740fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- 172.21.15.44:0/2270287488 shutdown_connections 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- 172.21.15.44:0/2270287488 wait complete. 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 Processor -- start 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- start start 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda74100810 0x7fda74195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda74103140 0x7fda741962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 0x7fda7419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7419d880 con 0x7fda74105a70 2024-09-13T17:24:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7419d9f0 con 0x7fda74103140 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7a37d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7419dbb0 con 0x7fda74100810 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7891a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 0x7fda7419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7891a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 0x7fda7419d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48376/0 (socket says 172.21.15.44:48376) 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.395+0000 7fda7891a640 1 -- 172.21.15.44:0/3294098380 learned_addr learned my addr 172.21.15.44:0/3294098380 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda6bfff640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda74103140 0x7fda741962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda73fff640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda74100810 0x7fda74195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7891a640 1 -- 172.21.15.44:0/3294098380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda74100810 msgr2=0x7fda74195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7891a640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda74100810 0x7fda74195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7891a640 1 -- 172.21.15.44:0/3294098380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda74103140 msgr2=0x7fda741962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7891a640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda74103140 0x7fda741962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7891a640 1 -- 172.21.15.44:0/3294098380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009560 con 0x7fda74105a70 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7891a640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 0x7fda7419d310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00eb40 tx=0x7fda6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c004b20 con 0x7fda74105a70 2024-09-13T17:24:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6c004c80 con 0x7fda74105a70 2024-09-13T17:24:35.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7419de90 con 0x7fda74105a70 2024-09-13T17:24:35.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c010690 con 0x7fda74105a70 2024-09-13T17:24:35.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.396+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7419e2b0 con 0x7fda74105a70 2024-09-13T17:24:35.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.397+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740fbee0 con 0x7fda74105a70 2024-09-13T17:24:35.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.398+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fda6c005020 con 0x7fda74105a70 2024-09-13T17:24:35.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.399+0000 7fda71ffb640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda600a1090 0x7fda600a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.399+0000 7fda73fff640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda600a1090 0x7fda600a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.399+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda6c014070 con 0x7fda74105a70 2024-09-13T17:24:35.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.400+0000 7fda73fff640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda600a1090 0x7fda600a3550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda640058c0 tx=0x7fda64005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.403+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fda6c096880 con 0x7fda74105a70 2024-09-13T17:24:35.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2308968662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:24:35.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1038707986' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:24:35.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4026422531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:35.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.574+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda7402c480 con 0x7fda74105a70 2024-09-13T17:24:35.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.574+0000 7fda71ffb640 1 -- 172.21.15.44:0/3294098380 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda6c0987b0 con 0x7fda74105a70 2024-09-13T17:24:35.576 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.576+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda600a1090 msgr2=0x7fda600a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.576+0000 7fda7a37d640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda600a1090 0x7fda600a3550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda640058c0 tx=0x7fda64005c80 comp rx=0 tx=0).stop 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.576+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 msgr2=0x7fda7419d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.576+0000 7fda7a37d640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 0x7fda7419d310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00eb40 tx=0x7fda6c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 shutdown_connections 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda600a1090 0x7fda600a3550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74105a70 0x7fda7419d310 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda74103140 0x7fda741962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 --2- 172.21.15.44:0/3294098380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda74100810 0x7fda74195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 >> 172.21.15.44:0/3294098380 conn(0x7fda740fa3f0 msgr2=0x7fda74101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 shutdown_connections 2024-09-13T17:24:35.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.577+0000 7fda7a37d640 1 -- 172.21.15.44:0/3294098380 wait complete. 2024-09-13T17:24:35.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:24:35.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 -- 172.21.15.44:0/761467654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8800100840 msgr2=0x7f88001070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 --2- 172.21.15.44:0/761467654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8800100840 0x7f88001070d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009ca0 tx=0x7f87f802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 -- 172.21.15.44:0/761467654 shutdown_connections 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 --2- 172.21.15.44:0/761467654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8800100840 0x7f88001070d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 --2- 172.21.15.44:0/761467654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88000ffe80 0x7f8800100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 --2- 172.21.15.44:0/761467654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88000fec90 0x7f88000ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 -- 172.21.15.44:0/761467654 >> 172.21.15.44:0/761467654 conn(0x7f88000fa3d0 msgr2=0x7f88000fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 -- 172.21.15.44:0/761467654 shutdown_connections 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.693+0000 7f88053bc640 1 -- 172.21.15.44:0/761467654 wait complete. 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 Processor -- start 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 -- start start 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 0x7f8800195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88000ffe80 0x7f8800196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8800100840 0x7f880019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d720 con 0x7f88000fec90 2024-09-13T17:24:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d890 con 0x7f8800100840 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f88053bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019da50 con 0x7f88000ffe80 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f87feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 0x7f8800195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f87feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 0x7f8800195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48390/0 (socket says 172.21.15.44:48390) 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f87feffd640 1 -- 172.21.15.44:0/3238693430 learned_addr learned my addr 172.21.15.44:0/3238693430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.694+0000 7f87fe7fc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88000ffe80 0x7f8800196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87ff7fe640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8800100840 0x7f880019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87feffd640 1 -- 172.21.15.44:0/3238693430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88000ffe80 msgr2=0x7f8800196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87feffd640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88000ffe80 0x7f8800196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87feffd640 1 -- 172.21.15.44:0/3238693430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8800100840 msgr2=0x7f880019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87feffd640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8800100840 0x7f880019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87feffd640 1 -- 172.21.15.44:0/3238693430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8009900 con 0x7f88000fec90 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87ff7fe640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8800100840 0x7f880019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87feffd640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 0x7f8800195c40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87f400d950 tx=0x7f87f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4009c30 con 0x7f88000fec90 2024-09-13T17:24:35.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87f4009d90 con 0x7f88000fec90 2024-09-13T17:24:35.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880019dd30 con 0x7f88000fec90 2024-09-13T17:24:35.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400f620 con 0x7f88000fec90 2024-09-13T17:24:35.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.695+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880019e190 con 0x7f88000fec90 2024-09-13T17:24:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.696+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800106120 con 0x7f88000fec90 2024-09-13T17:24:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.697+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f87f4010980 con 0x7f88000fec90 2024-09-13T17:24:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.698+0000 7f87e7fff640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87e80a1110 0x7f87e80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.698+0000 7f87fe7fc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87e80a1110 0x7f87e80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.698+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f87f40cc890 con 0x7f88000fec90 2024-09-13T17:24:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.699+0000 7f87fe7fc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87e80a1110 0x7f87e80a35d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009bd0 tx=0x7f87f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.701+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f87f4095dc0 con 0x7f88000fec90 2024-09-13T17:24:35.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.871+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f880002c3e0 con 0x7f88000fec90 2024-09-13T17:24:35.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.871+0000 7f87e7fff640 1 -- 172.21.15.44:0/3238693430 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87f4021020 con 0x7f88000fec90 2024-09-13T17:24:35.872 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87e80a1110 msgr2=0x7f87e80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87e80a1110 0x7f87e80a35d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009bd0 tx=0x7f87f0005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 msgr2=0x7f8800195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 0x7f8800195c40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87f400d950 tx=0x7f87f400de10 comp rx=0 tx=0).stop 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 shutdown_connections 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87e80a1110 0x7f87e80a35d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88000ffe80 0x7f8800196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8800100840 0x7f880019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 --2- 172.21.15.44:0/3238693430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88000fec90 0x7f8800195c40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.873+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 >> 172.21.15.44:0/3238693430 conn(0x7f88000fa3d0 msgr2=0x7f88001032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.874+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 shutdown_connections 2024-09-13T17:24:35.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.874+0000 7f88053bc640 1 -- 172.21.15.44:0/3238693430 wait complete. 2024-09-13T17:24:35.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:24:35.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 -- 172.21.15.44:0/805345308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80fec70 msgr2=0x7f28a80ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 --2- 172.21.15.44:0/805345308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80fec70 0x7f28a80ff070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2898009900 tx=0x7f289802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 -- 172.21.15.44:0/805345308 shutdown_connections 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 --2- 172.21.15.44:0/805345308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a80ffe60 0x7f28a81002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 --2- 172.21.15.44:0/805345308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a8100820 0x7f28a81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 --2- 172.21.15.44:0/805345308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80fec70 0x7f28a80ff070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 -- 172.21.15.44:0/805345308 >> 172.21.15.44:0/805345308 conn(0x7f28a80fa3d0 msgr2=0x7f28a80fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 -- 172.21.15.44:0/805345308 shutdown_connections 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.989+0000 7f28ad07d640 1 -- 172.21.15.44:0/805345308 wait complete. 2024-09-13T17:24:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 Processor -- start 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 -- start start 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a80fec70 0x7f28a8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 0x7f28a8196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a8100820 0x7f28a819d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819d800 con 0x7f28a80ffe60 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819d970 con 0x7f28a80fec70 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28ad07d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819db30 con 0x7f28a8100820 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28a659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 0x7f28a8196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28a659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 0x7f28a8196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48404/0 (socket says 172.21.15.44:48404) 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.990+0000 7f28a659c640 1 -- 172.21.15.44:0/3107991281 learned_addr learned my addr 172.21.15.44:0/3107991281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a6d9d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a80fec70 0x7f28a8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a659c640 1 -- 172.21.15.44:0/3107991281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a8100820 msgr2=0x7f28a819d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a659c640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a8100820 0x7f28a819d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a659c640 1 -- 172.21.15.44:0/3107991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a80fec70 msgr2=0x7f28a8195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a659c640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a80fec70 0x7f28a8195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a659c640 1 -- 172.21.15.44:0/3107991281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898009560 con 0x7f28a80ffe60 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a6d9d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a80fec70 0x7f28a8195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28a659c640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 0x7f28a8196260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f289c00e9f0 tx=0x7f289c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c009720 con 0x7f28a80ffe60 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f289c00cd80 con 0x7f28a80ffe60 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a819de10 con 0x7f28a80ffe60 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c010430 con 0x7f28a80ffe60 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.991+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a819e210 con 0x7f28a80ffe60 2024-09-13T17:24:35.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.992+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a810a3e0 con 0x7f28a80ffe60 2024-09-13T17:24:35.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.993+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f289c004fb0 con 0x7f28a80ffe60 2024-09-13T17:24:35.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.994+0000 7f288ffff640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28900a1090 0x7f28900a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:35.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.994+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f289c014070 con 0x7f28a80ffe60 2024-09-13T17:24:35.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.994+0000 7f28a6d9d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28900a1090 0x7f28900a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:35.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.994+0000 7f28a6d9d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28900a1090 0x7f28900a3550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2898000c00 tx=0x7f2898005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:35.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:35.998+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f289c096560 con 0x7f28a80ffe60 2024-09-13T17:24:36.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.176+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f28a819e490 con 0x7f28a80ffe60 2024-09-13T17:24:36.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.176+0000 7f288ffff640 1 -- 172.21.15.44:0/3107991281 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f289c098490 con 0x7f28a80ffe60 2024-09-13T17:24:36.178 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28900a1090 msgr2=0x7f28900a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28900a1090 0x7f28900a3550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2898000c00 tx=0x7f2898005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 msgr2=0x7f28a8196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 0x7f28a8196260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f289c00e9f0 tx=0x7f289c00edf0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 shutdown_connections 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28900a1090 0x7f28900a3550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a8100820 0x7f28a819d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a80ffe60 0x7f28a8196260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 --2- 172.21.15.44:0/3107991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28a80fec70 0x7f28a8195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 >> 172.21.15.44:0/3107991281 conn(0x7f28a80fa3d0 msgr2=0x7f28a8068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 shutdown_connections 2024-09-13T17:24:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.179+0000 7f28ad07d640 1 -- 172.21.15.44:0/3107991281 wait complete. 2024-09-13T17:24:36.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:24:36.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 -- 172.21.15.44:0/2179369130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c0ffd90 msgr2=0x7f9d8c100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2179369130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c0ffd90 0x7f9d8c100190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 -- 172.21.15.44:0/2179369130 shutdown_connections 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2179369130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c101940 0x7f9d8c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2179369130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c100f80 0x7f9d8c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2179369130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c0ffd90 0x7f9d8c100190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 -- 172.21.15.44:0/2179369130 >> 172.21.15.44:0/2179369130 conn(0x7f9d8c0fb4d0 msgr2=0x7f9d8c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 -- 172.21.15.44:0/2179369130 shutdown_connections 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.293+0000 7f9d9155e640 1 -- 172.21.15.44:0/2179369130 wait complete. 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 Processor -- start 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 -- start start 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0ffd90 0x7f9d8c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 0x7f9d8c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c101940 0x7f9d8c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c19d710 con 0x7f9d8c100f80 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c19d880 con 0x7f9d8c101940 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d9155e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c19da40 con 0x7f9d8c0ffd90 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d8a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 0x7f9d8c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d8a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 0x7f9d8c196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48424/0 (socket says 172.21.15.44:48424) 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d8a7fc640 1 -- 172.21.15.44:0/2361171231 learned_addr learned my addr 172.21.15.44:0/2361171231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.294+0000 7f9d8affd640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0ffd90 0x7f9d8c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8b7fe640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c101940 0x7f9d8c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8a7fc640 1 -- 172.21.15.44:0/2361171231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0ffd90 msgr2=0x7f9d8c195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8a7fc640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0ffd90 0x7f9d8c195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8a7fc640 1 -- 172.21.15.44:0/2361171231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c101940 msgr2=0x7f9d8c19d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8a7fc640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c101940 0x7f9d8c19d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8a7fc640 1 -- 172.21.15.44:0/2361171231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c009560 con 0x7f9d8c100f80 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8b7fe640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c101940 0x7f9d8c19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d8a7fc640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 0x7f9d8c196170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c920 tx=0x7f9d8000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80007c70 con 0x7f9d8c100f80 2024-09-13T17:24:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d80007dd0 con 0x7f9d8c100f80 2024-09-13T17:24:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c19dd20 con 0x7f9d8c100f80 2024-09-13T17:24:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80018730 con 0x7f9d8c100f80 2024-09-13T17:24:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.295+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c19e180 con 0x7f9d8c100f80 2024-09-13T17:24:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.296+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c107240 con 0x7f9d8c100f80 2024-09-13T17:24:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.297+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9d8000f450 con 0x7f9d8c100f80 2024-09-13T17:24:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.298+0000 7f9d73fff640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9d740a1090 0x7f9d740a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.298+0000 7f9d8affd640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9d740a1090 0x7f9d740a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.298+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9d800ccd10 con 0x7f9d8c100f80 2024-09-13T17:24:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.298+0000 7f9d8affd640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9d740a1090 0x7f9d740a3550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c197100 tx=0x7f9d7c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.302+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9d800d2020 con 0x7f9d8c100f80 2024-09-13T17:24:36.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:36 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3765853841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:36 smithi070 ceph-mon[34502]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-09-13T17:24:36.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:36 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3294098380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:36 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3238693430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.485+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d8c10b500 con 0x7f9d8c100f80 2024-09-13T17:24:36.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.485+0000 7f9d73fff640 1 -- 172.21.15.44:0/2361171231 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+590 (secure 0 0 0) 0x7f9d800961c0 con 0x7f9d8c100f80 2024-09-13T17:24:36.486 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2209604760 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:36.486 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/815607991 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1683772085 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1586278527 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/771352345 2024-09-14T17:17:41.472796+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2038542363 2024-09-14T17:17:17.599897+0000 2024-09-13T17:24:36.487 INFO:teuthology.orchestra.run.smithi044.stderr:listed 10 entries 2024-09-13T17:24:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9d740a1090 msgr2=0x7f9d740a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9d740a1090 0x7f9d740a3550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c197100 tx=0x7f9d7c005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 msgr2=0x7f9d8c196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 0x7f9d8c196170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c920 tx=0x7f9d8000cde0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 shutdown_connections 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9d740a1090 0x7f9d740a3550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8c101940 0x7f9d8c19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d8c100f80 0x7f9d8c196170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 --2- 172.21.15.44:0/2361171231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0ffd90 0x7f9d8c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 >> 172.21.15.44:0/2361171231 conn(0x7f9d8c0fb4d0 msgr2=0x7f9d8c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 shutdown_connections 2024-09-13T17:24:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:36.487+0000 7f9d9155e640 1 -- 172.21.15.44:0/2361171231 wait complete. 2024-09-13T17:24:36.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:36 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3765853841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:36 smithi044 ceph-mon[31175]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-09-13T17:24:36.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:36 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3294098380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.499 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:36 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3238693430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.499 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2024-09-13T17:24:36.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:36.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:24:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:36 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3765853841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:36 smithi100 ceph-mon[35055]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-09-13T17:24:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:36 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3294098380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:36 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3238693430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:24:36.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[74460]: fuse finished with error 0 and tester_r 0 2024-09-13T17:24:36.526 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:24:37.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3107991281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:24:37.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2361171231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:24:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3107991281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:24:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2361171231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:24:37.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3107991281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:24:37.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2361171231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:24:38.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:38 smithi070 ceph-mon[34502]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 268 B/s wr, 0 op/s 2024-09-13T17:24:38.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:38 smithi100 ceph-mon[35055]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 268 B/s wr, 0 op/s 2024-09-13T17:24:38.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:38 smithi044 ceph-mon[31175]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 268 B/s wr, 0 op/s 2024-09-13T17:24:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:39 smithi070 ceph-mon[34502]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-13T17:24:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:39 smithi100 ceph-mon[35055]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-13T17:24:40.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:39 smithi044 ceph-mon[31175]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-13T17:24:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-13T17:24:41.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-13T17:24:41.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:41 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-13T17:24:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-13T17:24:41.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-13T17:24:41.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:41 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-13T17:24:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-13T17:24:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-13T17:24:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-13T17:24:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-13T17:24:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-13T17:24:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-13T17:24:42.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-13T17:24:42.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-13T17:24:42.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-13T17:24:42.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:41 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-13T17:24:42.629 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:24:42.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:24:42.629 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T17:24:42.667 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 -- 172.21.15.44:0/3142847864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc0fe0a0 msgr2=0x7f90fc0fe4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 --2- 172.21.15.44:0/3142847864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc0fe0a0 0x7f90fc0fe4a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009900 tx=0x7f90ec02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 -- 172.21.15.44:0/3142847864 shutdown_connections 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 --2- 172.21.15.44:0/3142847864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc101110 0x7f90fc101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 --2- 172.21.15.44:0/3142847864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc101ad0 0x7f90fc108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 --2- 172.21.15.44:0/3142847864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc0fe0a0 0x7f90fc0fe4a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 -- 172.21.15.44:0/3142847864 >> 172.21.15.44:0/3142847864 conn(0x7f90fc0f9bc0 msgr2=0x7f90fc0fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:42.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 -- 172.21.15.44:0/3142847864 shutdown_connections 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.826+0000 7f9101701640 1 -- 172.21.15.44:0/3142847864 wait complete. 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 Processor -- start 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 -- start start 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc0fe0a0 0x7f90fc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 0x7f90fc1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc101ad0 0x7f90fc19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19fa70 con 0x7f90fc101110 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19fbe0 con 0x7f90fc0fe0a0 2024-09-13T17:24:42.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.827+0000 7f9101701640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19fda0 con 0x7f90fc101ad0 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 0x7f90fc1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 0x7f90fc1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45426/0 (socket says 172.21.15.44:45426) 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 -- 172.21.15.44:0/1237022363 learned_addr learned my addr 172.21.15.44:0/1237022363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fb7fe640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc101ad0 0x7f90fc19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90faffd640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc0fe0a0 0x7f90fc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 -- 172.21.15.44:0/1237022363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc101ad0 msgr2=0x7f90fc19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc101ad0 0x7f90fc19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 -- 172.21.15.44:0/1237022363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc0fe0a0 msgr2=0x7f90fc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc0fe0a0 0x7f90fc197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 -- 172.21.15.44:0/1237022363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec009560 con 0x7f90fc101110 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90fa7fc640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 0x7f90fc1984c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9f0 tx=0x7f90f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0009720 con 0x7f90fc101110 2024-09-13T17:24:42.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90f000cd80 con 0x7f90fc101110 2024-09-13T17:24:42.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc1a0080 con 0x7f90fc101110 2024-09-13T17:24:42.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0010430 con 0x7f90fc101110 2024-09-13T17:24:42.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.828+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc1a0590 con 0x7f90fc101110 2024-09-13T17:24:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.830+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc1924b0 con 0x7f90fc101110 2024-09-13T17:24:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.831+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f90f0004fb0 con 0x7f90fc101110 2024-09-13T17:24:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.831+0000 7f90e3fff640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90e40a1110 0x7f90e40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.831+0000 7f90faffd640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90e40a1110 0x7f90e40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.831+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90f0014070 con 0x7f90fc101110 2024-09-13T17:24:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.832+0000 7f90faffd640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90e40a1110 0x7f90e40a35d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0057c0 tx=0x7f90ec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:42.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:42.835+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f90f0096360 con 0x7f90fc101110 2024-09-13T17:24:42.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2024-09-13T17:24:42.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:42 smithi044 ceph-mon[31175]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-13T17:24:42.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-13T17:24:42.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-13T17:24:42.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-13T17:24:42.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T17:24:42.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2024-09-13T17:24:42.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:42 smithi070 ceph-mon[34502]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T17:24:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.010+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90fc10b540 con 0x7f90fc101110 2024-09-13T17:24:43.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.012+0000 7f90e3fff640 1 -- 172.21.15.44:0/1237022363 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24427 (secure 0 0 0) 0x7f90f0099980 con 0x7f90fc101110 2024-09-13T17:24:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:43.013 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:42.549640+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90e40a1110 msgr2=0x7f90e40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90e40a1110 0x7f90e40a35d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0057c0 tx=0x7f90ec005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 msgr2=0x7f90fc1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 0x7f90fc1984c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9f0 tx=0x7f90f000edf0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 shutdown_connections 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90e40a1110 0x7f90e40a35d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc101ad0 0x7f90fc19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90fc101110 0x7f90fc1984c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 --2- 172.21.15.44:0/1237022363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc0fe0a0 0x7f90fc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 >> 172.21.15.44:0/1237022363 conn(0x7f90fc0f9bc0 msgr2=0x7f90fc106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 shutdown_connections 2024-09-13T17:24:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.014+0000 7f9101701640 1 -- 172.21.15.44:0/1237022363 wait complete. 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-13T17:24:43.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-13T17:24:43.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-13T17:24:43.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-13T17:24:43.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T17:24:43.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2024-09-13T17:24:43.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:42 smithi100 ceph-mon[35055]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T17:24:43.026 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:43.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 -- 172.21.15.44:0/2354764638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140ffec0 msgr2=0x7fd714100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 --2- 172.21.15.44:0/2354764638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140ffec0 0x7fd714100340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd710009900 tx=0x7fd71002a410 comp rx=0 tx=0).stop 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 -- 172.21.15.44:0/2354764638 shutdown_connections 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 --2- 172.21.15.44:0/2354764638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd714100880 0x7fd714107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 --2- 172.21.15.44:0/2354764638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140ffec0 0x7fd714100340 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 --2- 172.21.15.44:0/2354764638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7140fecd0 0x7fd7140ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 -- 172.21.15.44:0/2354764638 >> 172.21.15.44:0/2354764638 conn(0x7fd7140fa430 msgr2=0x7fd7140fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 -- 172.21.15.44:0/2354764638 shutdown_connections 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 -- 172.21.15.44:0/2354764638 wait complete. 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 Processor -- start 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.128+0000 7fd71c419640 1 -- start start 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71c419640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 0x7fd714194c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71c419640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7140ffec0 0x7fd7141951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71c419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd714100880 0x7fd71419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71c419640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419c740 con 0x7fd7140fecd0 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71c419640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419c8b0 con 0x7fd714100880 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71c419640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419ca70 con 0x7fd7140ffec0 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 0x7fd714194c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 0x7fd714194c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45452/0 (socket says 172.21.15.44:45452) 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 -- 172.21.15.44:0/978885007 learned_addr learned my addr 172.21.15.44:0/978885007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd7199b4640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7140ffec0 0x7fd7141951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a9b6640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd714100880 0x7fd71419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 -- 172.21.15.44:0/978885007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7140ffec0 msgr2=0x7fd7141951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7140ffec0 0x7fd7141951a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 -- 172.21.15.44:0/978885007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd714100880 msgr2=0x7fd71419c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd714100880 0x7fd71419c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 -- 172.21.15.44:0/978885007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009560 con 0x7fd7140fecd0 2024-09-13T17:24:43.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd71a1b5640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 0x7fd714194c60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd70400e9c0 tx=0x7fd70400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:43.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.129+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040096e0 con 0x7fd7140fecd0 2024-09-13T17:24:43.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.130+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7040044c0 con 0x7fd7140fecd0 2024-09-13T17:24:43.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.130+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71419cd50 con 0x7fd7140fecd0 2024-09-13T17:24:43.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.130+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704003e30 con 0x7fd7140fecd0 2024-09-13T17:24:43.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.130+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71419d1b0 con 0x7fd7140fecd0 2024-09-13T17:24:43.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.130+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd714106270 con 0x7fd7140fecd0 2024-09-13T17:24:43.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.132+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd7040035f0 con 0x7fd7140fecd0 2024-09-13T17:24:43.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.132+0000 7fd70b7fe640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd7000a1110 0x7fd7000a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.132+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd704014070 con 0x7fd7140fecd0 2024-09-13T17:24:43.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.132+0000 7fd7199b4640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd7000a1110 0x7fd7000a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.133+0000 7fd7199b4640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd7000a1110 0x7fd7000a35d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd71002aa00 tx=0x7fd710005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:43.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.136+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd704096330 con 0x7fd7140fecd0 2024-09-13T17:24:43.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.311+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd71410a440 con 0x7fd7140fecd0 2024-09-13T17:24:43.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.313+0000 7fd70b7fe640 1 -- 172.21.15.44:0/978885007 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24427 (secure 0 0 0) 0x7fd704098260 con 0x7fd7140fecd0 2024-09-13T17:24:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:43.315 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:42.549640+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd7000a1110 msgr2=0x7fd7000a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd7000a1110 0x7fd7000a35d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd71002aa00 tx=0x7fd710005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 msgr2=0x7fd714194c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 0x7fd714194c60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd70400e9c0 tx=0x7fd70400ee80 comp rx=0 tx=0).stop 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 shutdown_connections 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd7000a1110 0x7fd7000a35d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7140ffec0 0x7fd7141951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd714100880 0x7fd71419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 --2- 172.21.15.44:0/978885007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7140fecd0 0x7fd714194c60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 >> 172.21.15.44:0/978885007 conn(0x7fd7140fa430 msgr2=0x7fd714103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:43.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 shutdown_connections 2024-09-13T17:24:43.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.315+0000 7fd71c419640 1 -- 172.21.15.44:0/978885007 wait complete. 2024-09-13T17:24:43.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.434+0000 7ff6baded640 1 -- 172.21.15.44:0/2468933017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4100810 msgr2=0x7ff6b4102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.434+0000 7ff6baded640 1 --2- 172.21.15.44:0/2468933017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4100810 0x7ff6b4102c00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009ca0 tx=0x7ff6a402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- 172.21.15.44:0/2468933017 shutdown_connections 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 --2- 172.21.15.44:0/2468933017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b4105a70 0x7ff6b4107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 --2- 172.21.15.44:0/2468933017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 0x7ff6b4105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 --2- 172.21.15.44:0/2468933017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4100810 0x7ff6b4102c00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- 172.21.15.44:0/2468933017 >> 172.21.15.44:0/2468933017 conn(0x7ff6b40fa430 msgr2=0x7ff6b40fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:43.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- 172.21.15.44:0/2468933017 shutdown_connections 2024-09-13T17:24:43.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- 172.21.15.44:0/2468933017 wait complete. 2024-09-13T17:24:43.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 Processor -- start 2024-09-13T17:24:43.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- start start 2024-09-13T17:24:43.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b4100810 0x7ff6b4195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 0x7ff6b41961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 0x7ff6b419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b419d7b0 con 0x7ff6b4105a70 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b419d920 con 0x7ff6b4103140 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.435+0000 7ff6baded640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b419dae0 con 0x7ff6b4100810 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 0x7ff6b419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 0x7ff6b419d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45466/0 (socket says 172.21.15.44:45466) 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 -- 172.21.15.44:0/498618284 learned_addr learned my addr 172.21.15.44:0/498618284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b3fff640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 0x7ff6b41961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b8b89640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b4100810 0x7ff6b4195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 -- 172.21.15.44:0/498618284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b4100810 msgr2=0x7ff6b4195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b4100810 0x7ff6b4195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 -- 172.21.15.44:0/498618284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 msgr2=0x7ff6b41961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:43.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 0x7ff6b41961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:43.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 -- 172.21.15.44:0/498618284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a4009900 con 0x7ff6b4105a70 2024-09-13T17:24:43.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b3fff640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 0x7ff6b41961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b938a640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 0x7ff6b419d240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00e970 tx=0x7ff6ac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0096c0 con 0x7ff6b4105a70 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6ac00ccc0 con 0x7ff6b4105a70 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b419ddc0 con 0x7ff6b4105a70 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004e50 con 0x7ff6b4105a70 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.436+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b419e220 con 0x7ff6b4105a70 2024-09-13T17:24:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.437+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b40fbf20 con 0x7ff6b4105a70 2024-09-13T17:24:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.439+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff6ac004fb0 con 0x7ff6b4105a70 2024-09-13T17:24:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.439+0000 7ff6b1ffb640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff69c0a1090 0x7ff69c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.439+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6ac014070 con 0x7ff6b4105a70 2024-09-13T17:24:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.439+0000 7ff6b8b89640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff69c0a1090 0x7ff69c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.440+0000 7ff6b8b89640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff69c0a1090 0x7ff69c0a3550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4031040 tx=0x7ff6a4009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.443+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff6ac0965a0 con 0x7ff6b4105a70 2024-09-13T17:24:43.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:43 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1237022363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:43.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:43 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/978885007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:43.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:43 smithi044 ceph-mon[31175]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-13T17:24:43.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:43.654+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6b402c480 con 0x7ff6b4105a70 2024-09-13T17:24:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:43 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1237022363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:43 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/978885007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:43 smithi070 ceph-mon[34502]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-13T17:24:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:43 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1237022363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:43 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/978885007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:43 smithi100 ceph-mon[35055]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-13T17:24:44.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.579+0000 7ff6b1ffb640 1 -- 172.21.15.44:0/498618284 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff6ac099bc0 con 0x7ff6b4105a70 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.581+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff69c0a1090 msgr2=0x7ff69c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.581+0000 7ff6baded640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff69c0a1090 0x7ff69c0a3550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4031040 tx=0x7ff6a4009890 comp rx=0 tx=0).stop 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.581+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 msgr2=0x7ff6b419d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.581+0000 7ff6baded640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 0x7ff6b419d240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00e970 tx=0x7ff6ac00ee30 comp rx=0 tx=0).stop 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 shutdown_connections 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff69c0a1090 0x7ff69c0a3550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b4105a70 0x7ff6b419d240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b4103140 0x7ff6b41961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 --2- 172.21.15.44:0/498618284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b4100810 0x7ff6b4195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 >> 172.21.15.44:0/498618284 conn(0x7ff6b40fa430 msgr2=0x7ff6b41010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:44.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 shutdown_connections 2024-09-13T17:24:44.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.582+0000 7ff6baded640 1 -- 172.21.15.44:0/498618284 wait complete. 2024-09-13T17:24:44.596 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-13T17:24:44.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 -- 172.21.15.44:0/2839452547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c100810 msgr2=0x7f6c6c1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 --2- 172.21.15.44:0/2839452547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c100810 0x7f6c6c1070a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009ca0 tx=0x7f6c6402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 -- 172.21.15.44:0/2839452547 shutdown_connections 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 --2- 172.21.15.44:0/2839452547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c100810 0x7f6c6c1070a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 --2- 172.21.15.44:0/2839452547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 --2- 172.21.15.44:0/2839452547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c0fec60 0x7f6c6c0ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 -- 172.21.15.44:0/2839452547 >> 172.21.15.44:0/2839452547 conn(0x7f6c6c0fa3e0 msgr2=0x7f6c6c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 -- 172.21.15.44:0/2839452547 shutdown_connections 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 -- 172.21.15.44:0/2839452547 wait complete. 2024-09-13T17:24:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 Processor -- start 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.708+0000 7f6c726d0640 1 -- start start 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c726d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6c0fec60 0x7f6c6c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c726d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c726d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c100810 0x7f6c6c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c726d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c19d750 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c726d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c19d8c0 con 0x7f6c6c0fec60 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c726d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c19da80 con 0x7f6c6c100810 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45488/0 (socket says 172.21.15.44:45488) 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 -- 172.21.15.44:0/3459642197 learned_addr learned my addr 172.21.15.44:0/3459642197 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6bfff640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6c0fec60 0x7f6c6c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c70c6d640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c100810 0x7f6c6c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 -- 172.21.15.44:0/3459642197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c100810 msgr2=0x7f6c6c19d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c100810 0x7f6c6c19d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 -- 172.21.15.44:0/3459642197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6c0fec60 msgr2=0x7f6c6c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6c0fec60 0x7f6c6c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 -- 172.21.15.44:0/3459642197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009900 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c70c6d640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c100810 0x7f6c6c19d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.709+0000 7f6c6b7fe640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c196200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000d950 tx=0x7f6c6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.710+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60009a30 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.710+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c19dd60 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.710+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c600044b0 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.710+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c600047a0 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.710+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c19e200 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.711+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c10a3d0 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.713+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6c60004020 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.713+0000 7f6c697fa640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f6c540a1090 0x7f6c540a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.713+0000 7f6c6bfff640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f6c540a1090 0x7f6c540a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.713+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6c600cc4e0 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.714+0000 7f6c6bfff640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f6c540a1090 0x7f6c540a3550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c197140 tx=0x7f6c5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.716+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6c600958d0 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/498618284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T17:24:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:44.926+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f6c6c19e670 con 0x7f6c6c0ffe50 2024-09-13T17:24:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/498618284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T17:24:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/498618284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T17:24:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.584+0000 7f6c697fa640 1 -- 172.21.15.44:0/3459642197 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6c600170b0 con 0x7f6c6c0ffe50 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.586+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f6c540a1090 msgr2=0x7f6c540a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.586+0000 7f6c726d0640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f6c540a1090 0x7f6c540a3550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c197140 tx=0x7f6c5c005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.586+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 msgr2=0x7f6c6c196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.586+0000 7f6c726d0640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c196200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000d950 tx=0x7f6c6000de10 comp rx=0 tx=0).stop 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.586+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 shutdown_connections 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.586+0000 7f6c726d0640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f6c540a1090 0x7f6c540a3550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.587+0000 7f6c726d0640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c6c100810 0x7f6c6c19d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.587+0000 7f6c726d0640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c6c0ffe50 0x7f6c6c196200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.587+0000 7f6c726d0640 1 --2- 172.21.15.44:0/3459642197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6c0fec60 0x7f6c6c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.587+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 >> 172.21.15.44:0/3459642197 conn(0x7f6c6c0fa3e0 msgr2=0x7f6c6c067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.587+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 shutdown_connections 2024-09-13T17:24:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.587+0000 7f6c726d0640 1 -- 172.21.15.44:0/3459642197 wait complete. 2024-09-13T17:24:45.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-09-13T17:24:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 -- 172.21.15.44:0/2780854424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 msgr2=0x7fabd0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/2780854424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0101410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009900 tx=0x7fabc402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 -- 172.21.15.44:0/2780854424 shutdown_connections 2024-09-13T17:24:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/2780854424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd0101950 0x7fabd01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/2780854424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0101410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/2780854424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd00ffda0 0x7fabd01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 -- 172.21.15.44:0/2780854424 >> 172.21.15.44:0/2780854424 conn(0x7fabd00fb520 msgr2=0x7fabd00fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 -- 172.21.15.44:0/2780854424 shutdown_connections 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 -- 172.21.15.44:0/2780854424 wait complete. 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 Processor -- start 2024-09-13T17:24:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.708+0000 7fabd5ed0640 1 -- start start 2024-09-13T17:24:45.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabd5ed0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00ffda0 0x7fabd0195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabd5ed0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabd5ed0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd0101950 0x7fabd019d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabd5ed0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019d730 con 0x7fabd0100f90 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabd5ed0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019d8a0 con 0x7fabd0101950 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabd5ed0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019da60 con 0x7fabd00ffda0 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45508/0 (socket says 172.21.15.44:45508) 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 -- 172.21.15.44:0/1529749234 learned_addr learned my addr 172.21.15.44:0/1529749234 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabcf7fe640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00ffda0 0x7fabd0195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabcffff640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd0101950 0x7fabd019d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:45.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 -- 172.21.15.44:0/1529749234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00ffda0 msgr2=0x7fabd0195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00ffda0 0x7fabd0195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 -- 172.21.15.44:0/1529749234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd0101950 msgr2=0x7fabd019d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd0101950 0x7fabd019d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 -- 172.21.15.44:0/1529749234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4009560 con 0x7fabd0100f90 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabcffff640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd0101950 0x7fabd019d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabceffd640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0196190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009230 tx=0x7fabc4004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.709+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4031d70 con 0x7fabd0100f90 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.710+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabc4004b50 con 0x7fabd0100f90 2024-09-13T17:24:45.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.710+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd019dce0 con 0x7fabd0100f90 2024-09-13T17:24:45.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.710+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4003bf0 con 0x7fabd0100f90 2024-09-13T17:24:45.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.710+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd019e190 con 0x7fabd0100f90 2024-09-13T17:24:45.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.710+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0107200 con 0x7fabd0100f90 2024-09-13T17:24:45.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.712+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fabc40046c0 con 0x7fabd0100f90 2024-09-13T17:24:45.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.712+0000 7fabccff9640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fabb80a1110 0x7fabb80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:45.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.712+0000 7fabcf7fe640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fabb80a1110 0x7fabb80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:45.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.712+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fabc40ee1e0 con 0x7fabd0100f90 2024-09-13T17:24:45.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.713+0000 7fabcf7fe640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fabb80a1110 0x7fabb80a35d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabd0197120 tx=0x7fabc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:45.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.716+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fabc40b75d0 con 0x7fabd0100f90 2024-09-13T17:24:45.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:45 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/498618284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T17:24:45.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:45 smithi044 ceph-mon[31175]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:45.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:45 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3459642197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-13T17:24:45.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:45 smithi044 ceph-mon[31175]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:45.927+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fabd010b570 con 0x7fabd0100f90 2024-09-13T17:24:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:45 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/498618284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T17:24:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:45 smithi070 ceph-mon[34502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:45 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3459642197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-13T17:24:45.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:45 smithi070 ceph-mon[34502]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:45 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/498618284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T17:24:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:45 smithi100 ceph-mon[35055]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:45 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3459642197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-13T17:24:46.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:45 smithi100 ceph-mon[35055]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:46.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.140+0000 7fabccff9640 1 -- 172.21.15.44:0/1529749234 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fabc40b9500 con 0x7fabd0100f90 2024-09-13T17:24:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fabb80a1110 msgr2=0x7fabb80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fabb80a1110 0x7fabb80a35d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabd0197120 tx=0x7fabc0005f90 comp rx=0 tx=0).stop 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 msgr2=0x7fabd0196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0196190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009230 tx=0x7fabc4004380 comp rx=0 tx=0).stop 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 shutdown_connections 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fabb80a1110 0x7fabb80a35d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabd0101950 0x7fabd019d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabd0100f90 0x7fabd0196190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 --2- 172.21.15.44:0/1529749234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00ffda0 0x7fabd0195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 >> 172.21.15.44:0/1529749234 conn(0x7fabd00fb520 msgr2=0x7fabd01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 shutdown_connections 2024-09-13T17:24:46.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.142+0000 7fabd5ed0640 1 -- 172.21.15.44:0/1529749234 wait complete. 2024-09-13T17:24:46.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-13T17:24:46.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.262+0000 7f009535a640 1 -- 172.21.15.44:0/3903802997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 msgr2=0x7f0090105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:46.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.262+0000 7f009535a640 1 --2- 172.21.15.44:0/3903802997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090105530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0084009900 tx=0x7f008402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:46.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 -- 172.21.15.44:0/3903802997 shutdown_connections 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 --2- 172.21.15.44:0/3903802997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090105a70 0x7f0090107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 --2- 172.21.15.44:0/3903802997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090105530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 --2- 172.21.15.44:0/3903802997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0090100810 0x7f0090102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 -- 172.21.15.44:0/3903802997 >> 172.21.15.44:0/3903802997 conn(0x7f00900fa3f0 msgr2=0x7f00900fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 -- 172.21.15.44:0/3903802997 shutdown_connections 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 -- 172.21.15.44:0/3903802997 wait complete. 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 Processor -- start 2024-09-13T17:24:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.263+0000 7f009535a640 1 -- start start 2024-09-13T17:24:46.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 0x7f0090195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0090105a70 0x7f009019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009019d830 con 0x7f0090100810 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009019d9a0 con 0x7f0090103140 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009019db60 con 0x7f0090105a70 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 0x7f0090195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 0x7f0090195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45518/0 (socket says 172.21.15.44:45518) 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 -- 172.21.15.44:0/89130 learned_addr learned my addr 172.21.15.44:0/89130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008e7fc640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008f7fe640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0090105a70 0x7f009019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 -- 172.21.15.44:0/89130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0090105a70 msgr2=0x7f009019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0090105a70 0x7f009019d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 -- 172.21.15.44:0/89130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 msgr2=0x7f0090196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 -- 172.21.15.44:0/89130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088009840 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008e7fc640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f008effd640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 0x7f0090195d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9f0 tx=0x7f008000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080009700 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 -- 172.21.15.44:0/89130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0084009560 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f008000cd80 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800187e0 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.264+0000 7f009535a640 1 -- 172.21.15.44:0/89130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009019e190 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.265+0000 7f009535a640 1 -- 172.21.15.44:0/89130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00900fbee0 con 0x7f0090100810 2024-09-13T17:24:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.267+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0080010430 con 0x7f0090100810 2024-09-13T17:24:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.267+0000 7f0077fff640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00780a1090 0x7f00780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.267+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0080014070 con 0x7f0090100810 2024-09-13T17:24:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.267+0000 7f008e7fc640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00780a1090 0x7f00780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.268+0000 7f008e7fc640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00780a1090 0x7f00780a3550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0084006010 tx=0x7f0084005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.271+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0080052e20 con 0x7f0090100810 2024-09-13T17:24:46.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:46.480+0000 7f009535a640 1 -- 172.21.15.44:0/89130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f009002c480 con 0x7f0090100810 2024-09-13T17:24:46.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3459642197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1529749234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] up:active 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1529749234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:24:46.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 2 up:standby-replay 8 up:standby 2024-09-13T17:24:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2024-09-13T17:24:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-13T17:24:46.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/89130' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T17:24:46.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T17:24:46.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3459642197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-13T17:24:46.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T17:24:46.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:46.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-13T17:24:46.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1529749234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] up:active 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1529749234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 2 up:standby-replay 8 up:standby 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-13T17:24:46.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/89130' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T17:24:47.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3459642197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1529749234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] up:active 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1529749234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:24:47.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 2 up:standby-replay 8 up:standby 2024-09-13T17:24:47.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2024-09-13T17:24:47.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-13T17:24:47.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/89130' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T17:24:47.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.147+0000 7f0077fff640 1 -- 172.21.15.44:0/89130 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f00800964e0 con 0x7f0090100810 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 -- 172.21.15.44:0/89130 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00780a1090 msgr2=0x7f00780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00780a1090 0x7f00780a3550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0084006010 tx=0x7f0084005e00 comp rx=0 tx=0).stop 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 -- 172.21.15.44:0/89130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 msgr2=0x7f0090195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 0x7f0090195d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9f0 tx=0x7f008000eeb0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 -- 172.21.15.44:0/89130 shutdown_connections 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f00780a1090 0x7f00780a3550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0090105a70 0x7f009019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0090103140 0x7f0090196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 --2- 172.21.15.44:0/89130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0090100810 0x7f0090195d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.149+0000 7f009535a640 1 -- 172.21.15.44:0/89130 >> 172.21.15.44:0/89130 conn(0x7f00900fa3f0 msgr2=0x7f00901010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:47.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.150+0000 7f009535a640 1 -- 172.21.15.44:0/89130 shutdown_connections 2024-09-13T17:24:47.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.150+0000 7f009535a640 1 -- 172.21.15.44:0/89130 wait complete. 2024-09-13T17:24:47.164 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-13T17:24:47.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 -- 172.21.15.44:0/1393228645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 msgr2=0x7f22580ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 --2- 172.21.15.44:0/1393228645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f22580ff060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2254009900 tx=0x7f225402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 -- 172.21.15.44:0/1393228645 shutdown_connections 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 --2- 172.21.15.44:0/1393228645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22580ffe50 0x7f22581002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 --2- 172.21.15.44:0/1393228645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2258100810 0x7f22581070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 --2- 172.21.15.44:0/1393228645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f22580ff060 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 -- 172.21.15.44:0/1393228645 >> 172.21.15.44:0/1393228645 conn(0x7f22580fa3e0 msgr2=0x7f22580fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 -- 172.21.15.44:0/1393228645 shutdown_connections 2024-09-13T17:24:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.277+0000 7f226042b640 1 -- 172.21.15.44:0/1393228645 wait complete. 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 Processor -- start 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 -- start start 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f2258195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22580ffe50 0x7f22581961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2258100810 0x7f225819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225819d760 con 0x7f22580fec60 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225819d8d0 con 0x7f22580ffe50 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f226042b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225819da90 con 0x7f2258100810 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f225e1c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f2258195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f225e1c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f2258195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45536/0 (socket says 172.21.15.44:45536) 2024-09-13T17:24:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f225e1c7640 1 -- 172.21.15.44:0/3522992780 learned_addr learned my addr 172.21.15.44:0/3522992780 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.278+0000 7f225e9c8640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2258100810 0x7f225819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225d9c6640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22580ffe50 0x7f22581961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225e1c7640 1 -- 172.21.15.44:0/3522992780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2258100810 msgr2=0x7f225819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225e1c7640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2258100810 0x7f225819d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225e1c7640 1 -- 172.21.15.44:0/3522992780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22580ffe50 msgr2=0x7f22581961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225e1c7640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22580ffe50 0x7f22581961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225e1c7640 1 -- 172.21.15.44:0/3522992780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254009560 con 0x7f22580fec60 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f225e1c7640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f2258195c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2254005ab0 tx=0x7f22540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254031d70 con 0x7f22580fec60 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22540044e0 con 0x7f22580fec60 2024-09-13T17:24:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225819dd10 con 0x7f22580fec60 2024-09-13T17:24:47.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254003b00 con 0x7f22580fec60 2024-09-13T17:24:47.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.279+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225819e1c0 con 0x7f22580fec60 2024-09-13T17:24:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.280+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225810a3b0 con 0x7f22580fec60 2024-09-13T17:24:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.281+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2254004c50 con 0x7f22580fec60 2024-09-13T17:24:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.282+0000 7f224f7fe640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f22440a1090 0x7f22440a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.282+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f22540edcc0 con 0x7f22580fec60 2024-09-13T17:24:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.282+0000 7f225d9c6640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f22440a1090 0x7f22440a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.282+0000 7f225d9c6640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f22440a1090 0x7f22440a3550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2248005d00 tx=0x7f224800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.286+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f22540b7160 con 0x7f22580fec60 2024-09-13T17:24:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.487+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2258068d30 con 0x7f22440a1090 2024-09-13T17:24:47.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.510+0000 7f224f7fe640 1 -- 172.21.15.44:0/3522992780 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2258068d30 con 0x7f22440a1090 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f22440a1090 msgr2=0x7f22440a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f22440a1090 0x7f22440a3550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2248005d00 tx=0x7f224800b410 comp rx=0 tx=0).stop 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 msgr2=0x7f2258195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f2258195c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2254005ab0 tx=0x7f22540042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 shutdown_connections 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f22440a1090 0x7f22440a3550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2258100810 0x7f225819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22580ffe50 0x7f22581961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 --2- 172.21.15.44:0/3522992780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22580fec60 0x7f2258195c80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 >> 172.21.15.44:0/3522992780 conn(0x7f22580fa3e0 msgr2=0x7f2258068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 shutdown_connections 2024-09-13T17:24:47.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.512+0000 7f226042b640 1 -- 172.21.15.44:0/3522992780 wait complete. 2024-09-13T17:24:47.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 -- 172.21.15.44:0/2278076526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc100840 msgr2=0x7f7fdc1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 --2- 172.21.15.44:0/2278076526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc100840 0x7f7fdc1070d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009ca0 tx=0x7f7fd402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 -- 172.21.15.44:0/2278076526 shutdown_connections 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 --2- 172.21.15.44:0/2278076526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 --2- 172.21.15.44:0/2278076526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc100840 0x7f7fdc1070d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 --2- 172.21.15.44:0/2278076526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc0fec90 0x7f7fdc0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 -- 172.21.15.44:0/2278076526 >> 172.21.15.44:0/2278076526 conn(0x7f7fdc0fa3f0 msgr2=0x7f7fdc0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 -- 172.21.15.44:0/2278076526 shutdown_connections 2024-09-13T17:24:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.631+0000 7f7fe1873640 1 -- 172.21.15.44:0/2278076526 wait complete. 2024-09-13T17:24:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 Processor -- start 2024-09-13T17:24:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 -- start start 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fdc0fec90 0x7f7fdc195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc1961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc100840 0x7f7fdc19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc19d740 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc19d8b0 con 0x7f7fdc100840 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fe1873640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc19da70 con 0x7f7fdc0fec90 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fda7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc1961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fda7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc1961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45572/0 (socket says 172.21.15.44:45572) 2024-09-13T17:24:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fda7fc640 1 -- 172.21.15.44:0/1658351248 learned_addr learned my addr 172.21.15.44:0/1658351248 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.632+0000 7f7fdaffd640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fdc0fec90 0x7f7fdc195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fdb7fe640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc100840 0x7f7fdc19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fda7fc640 1 -- 172.21.15.44:0/1658351248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fdc0fec90 msgr2=0x7f7fdc195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fda7fc640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fdc0fec90 0x7f7fdc195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fda7fc640 1 -- 172.21.15.44:0/1658351248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc100840 msgr2=0x7f7fdc19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fda7fc640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc100840 0x7f7fdc19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fda7fc640 1 -- 172.21.15.44:0/1658351248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4009900 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fdb7fe640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc100840 0x7f7fdc19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fda7fc640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc1961a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d770 tx=0x7f7fd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0009c30 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc19dd50 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd000de80 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0004d40 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.633+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc19e1b0 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.634+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc10a400 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.635+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7fd0004ea0 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.636+0000 7f7fe0871640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7fc40a1090 0x7f7fc40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.636+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7fd00ccb90 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.636+0000 7f7fdaffd640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7fc40a1090 0x7f7fc40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.636+0000 7f7fdaffd640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7fc40a1090 0x7f7fc40a3550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc197130 tx=0x7f7fcc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.639+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7fd0095f20 con 0x7f7fdc0ffe80 2024-09-13T17:24:47.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.839+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fdc068cd0 con 0x7f7fc40a1090 2024-09-13T17:24:47.856 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:24:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:24:47.491+0000 7f67d2766640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:24:47.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:24:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:24:47.491+0000 7f67d2766640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:24:47.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:24:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:24:47.491+0000 7f67d2766640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:24:47.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:24:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:24:47.491+0000 7f67d2766640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:24:47.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:24:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:24:47.491+0000 7f67d2766640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:24:47.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.864+0000 7f7fe0871640 1 -- 172.21.15.44:0/1658351248 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7fdc068cd0 con 0x7f7fc40a1090 2024-09-13T17:24:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7fc40a1090 msgr2=0x7f7fc40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7fc40a1090 0x7f7fc40a3550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc197130 tx=0x7f7fcc00b390 comp rx=0 tx=0).stop 2024-09-13T17:24:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 msgr2=0x7f7fdc1961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc1961a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d770 tx=0x7f7fd000dc30 comp rx=0 tx=0).stop 2024-09-13T17:24:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 shutdown_connections 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7fc40a1090 0x7f7fc40a3550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc100840 0x7f7fdc19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fdc0ffe80 0x7f7fdc1961a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 --2- 172.21.15.44:0/1658351248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fdc0fec90 0x7f7fdc195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 >> 172.21.15.44:0/1658351248 conn(0x7f7fdc0fa3f0 msgr2=0x7f7fdc0685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 shutdown_connections 2024-09-13T17:24:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.866+0000 7f7fe1873640 1 -- 172.21.15.44:0/1658351248 wait complete. 2024-09-13T17:24:47.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-13T17:24:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 -- 172.21.15.44:0/3471457422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f80ffda0 msgr2=0x7f13f81001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 --2- 172.21.15.44:0/3471457422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f80ffda0 0x7f13f81001a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009900 tx=0x7f13e802a410 comp rx=0 tx=0).stop 2024-09-13T17:24:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 -- 172.21.15.44:0/3471457422 shutdown_connections 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 --2- 172.21.15.44:0/3471457422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13f8101950 0x7f13f81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 --2- 172.21.15.44:0/3471457422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8100f90 0x7f13f8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 --2- 172.21.15.44:0/3471457422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f80ffda0 0x7f13f81001a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 -- 172.21.15.44:0/3471457422 >> 172.21.15.44:0/3471457422 conn(0x7f13f80fb520 msgr2=0x7f13f80fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 -- 172.21.15.44:0/3471457422 shutdown_connections 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.985+0000 7f1400290640 1 -- 172.21.15.44:0/3471457422 wait complete. 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 Processor -- start 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 -- start start 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13f80ffda0 0x7f13f8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 0x7f13f8196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8101950 0x7f13f819d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f819d800 con 0x7f13f8100f90 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f819d970 con 0x7f13f8101950 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f1400290640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f819db30 con 0x7f13f80ffda0 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 0x7f13f8196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 0x7f13f8196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45592/0 (socket says 172.21.15.44:45592) 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 -- 172.21.15.44:0/1748541629 learned_addr learned my addr 172.21.15.44:0/1748541629 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fe02c640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13f80ffda0 0x7f13f8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 -- 172.21.15.44:0/1748541629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13f80ffda0 msgr2=0x7f13f8195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13f80ffda0 0x7f13f8195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 -- 172.21.15.44:0/1748541629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8101950 msgr2=0x7f13f819d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8101950 0x7f13f819d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.986+0000 7f13fd82b640 1 -- 172.21.15.44:0/1748541629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8009560 con 0x7f13f8100f90 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.987+0000 7f13fd82b640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 0x7f13f8196260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c890 tx=0x7f13f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.987+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4004220 con 0x7f13f8100f90 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.987+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13f4007e40 con 0x7f13f8100f90 2024-09-13T17:24:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.987+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f819de10 con 0x7f13f8100f90 2024-09-13T17:24:47.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.987+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4004a00 con 0x7f13f8100f90 2024-09-13T17:24:47.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.987+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f819e2b0 con 0x7f13f8100f90 2024-09-13T17:24:47.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.988+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f81072c0 con 0x7f13f8100f90 2024-09-13T17:24:47.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.989+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f13f40079b0 con 0x7f13f8100f90 2024-09-13T17:24:47.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.990+0000 7f13ef7fe640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f13e40a1110 0x7f13e40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:47.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.990+0000 7f13fe02c640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f13e40a1110 0x7f13e40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:47.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.990+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f13f40ccd60 con 0x7f13f8100f90 2024-09-13T17:24:47.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.990+0000 7f13fe02c640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f13e40a1110 0x7f13e40a35d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13e802aa00 tx=0x7f13e8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:47.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:47.993+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f13f40961d0 con 0x7f13f8100f90 2024-09-13T17:24:48.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.163+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f13f40965a0 con 0x7f13f8100f90 2024-09-13T17:24:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.196+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f13f802c4f0 con 0x7f13e40a1110 2024-09-13T17:24:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.202+0000 7f13ef7fe640 1 -- 172.21.15.44:0/1748541629 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f13f802c4f0 con 0x7f13e40a1110 2024-09-13T17:24:48.203 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/qa/sv_0/b119cd80-e9f3-469f-ae00-550e1071ac52 2024-09-13T17:24:48.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f13e40a1110 msgr2=0x7f13e40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f13e40a1110 0x7f13e40a35d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13e802aa00 tx=0x7f13e8005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:48.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 msgr2=0x7f13f8196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 0x7f13f8196260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c890 tx=0x7f13f400cd50 comp rx=0 tx=0).stop 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 shutdown_connections 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f13e40a1110 0x7f13e40a35d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8101950 0x7f13f819d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13f8100f90 0x7f13f8196260 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 --2- 172.21.15.44:0/1748541629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13f80ffda0 0x7f13f8195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 >> 172.21.15.44:0/1748541629 conn(0x7f13f80fb520 msgr2=0x7f13f8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 shutdown_connections 2024-09-13T17:24:48.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.204+0000 7f1400290640 1 -- 172.21.15.44:0/1748541629 wait complete. 2024-09-13T17:24:48.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 -- 172.21.15.44:0/2432768791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e00fec60 msgr2=0x7fd1e00ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/2432768791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e00fec60 0x7fd1e00ff060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 -- 172.21.15.44:0/2432768791 shutdown_connections 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/2432768791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e0100810 0x7fd1e01070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/2432768791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e00ffe50 0x7fd1e01002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/2432768791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e00fec60 0x7fd1e00ff060 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 -- 172.21.15.44:0/2432768791 >> 172.21.15.44:0/2432768791 conn(0x7fd1e00fa3e0 msgr2=0x7fd1e00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 -- 172.21.15.44:0/2432768791 shutdown_connections 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 -- 172.21.15.44:0/2432768791 wait complete. 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 Processor -- start 2024-09-13T17:24:48.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.323+0000 7fd1e75e4640 1 -- start start 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e00fec60 0x7fd1e00680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e00ffe50 0x7fd1e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 0x7fd1e019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019d7f0 con 0x7fd1e0100810 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019d960 con 0x7fd1e00fec60 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019db20 con 0x7fd1e00ffe50 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 0x7fd1e019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 0x7fd1e019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45598/0 (socket says 172.21.15.44:45598) 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 -- 172.21.15.44:0/956298471 learned_addr learned my addr 172.21.15.44:0/956298471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e4b7f640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e00ffe50 0x7fd1e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5380640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e00fec60 0x7fd1e00680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 -- 172.21.15.44:0/956298471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e00ffe50 msgr2=0x7fd1e019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e00ffe50 0x7fd1e019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 -- 172.21.15.44:0/956298471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e00fec60 msgr2=0x7fd1e00680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e00fec60 0x7fd1e00680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 -- 172.21.15.44:0/956298471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009560 con 0x7fd1e0100810 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e5b81640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 0x7fd1e019d060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800eb10 tx=0x7fd1d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004b20 con 0x7fd1e0100810 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1d8004c80 con 0x7fd1e0100810 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e019de00 con 0x7fd1e0100810 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8005a10 con 0x7fd1e0100810 2024-09-13T17:24:48.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.324+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e019e280 con 0x7fd1e0100810 2024-09-13T17:24:48.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.325+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e0106180 con 0x7fd1e0100810 2024-09-13T17:24:48.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.326+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd1d8004fd0 con 0x7fd1e0100810 2024-09-13T17:24:48.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.327+0000 7fd1d67fc640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1cc0a1120 0x7fd1cc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.327+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1d8014070 con 0x7fd1e0100810 2024-09-13T17:24:48.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.327+0000 7fd1e5380640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1cc0a1120 0x7fd1cc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.328+0000 7fd1e5380640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1cc0a1120 0x7fd1cc0a35e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc006010 tx=0x7fd1dc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:48.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.330+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd1d8096940 con 0x7fd1e0100810 2024-09-13T17:24:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:48 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] up:active 2024-09-13T17:24:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:48 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/89130' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T17:24:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:48 smithi070 ceph-mon[34502]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-13T17:24:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:48 smithi070 ceph-mon[34502]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T17:24:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:48 smithi070 ceph-mon[34502]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:48 smithi070 ceph-mon[34502]: pgmap v293: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 1.1 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:48 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] up:active 2024-09-13T17:24:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:48 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/89130' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T17:24:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:48 smithi100 ceph-mon[35055]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-13T17:24:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:48 smithi100 ceph-mon[35055]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T17:24:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:48 smithi100 ceph-mon[35055]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:48 smithi100 ceph-mon[35055]: pgmap v293: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 1.1 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:48.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.528+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e002c4f0 con 0x7fd1cc0a1120 2024-09-13T17:24:48.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.552+0000 7fd1d67fc640 1 -- 172.21.15.44:0/956298471 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd1e002c4f0 con 0x7fd1cc0a1120 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1cc0a1120 msgr2=0x7fd1cc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1cc0a1120 0x7fd1cc0a35e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc006010 tx=0x7fd1dc005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 msgr2=0x7fd1e019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 0x7fd1e019d060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800eb10 tx=0x7fd1d800c2d0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 shutdown_connections 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1cc0a1120 0x7fd1cc0a35e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e00ffe50 0x7fd1e019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e0100810 0x7fd1e019d060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 --2- 172.21.15.44:0/956298471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e00fec60 0x7fd1e00680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 >> 172.21.15.44:0/956298471 conn(0x7fd1e00fa3e0 msgr2=0x7fd1e0103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:48.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 shutdown_connections 2024-09-13T17:24:48.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.554+0000 7fd1e75e4640 1 -- 172.21.15.44:0/956298471 wait complete. 2024-09-13T17:24:48.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:48 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] up:active 2024-09-13T17:24:48.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:48 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/89130' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T17:24:48.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:48 smithi044 ceph-mon[31175]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-13T17:24:48.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:48 smithi044 ceph-mon[31175]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T17:24:48.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:48 smithi044 ceph-mon[31175]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:48.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:48 smithi044 ceph-mon[31175]: pgmap v293: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 1.1 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:48.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-13T17:24:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 -- 172.21.15.44:0/3589098544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 msgr2=0x7f2dec105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 --2- 172.21.15.44:0/3589098544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec105530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009900 tx=0x7f2dd402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 -- 172.21.15.44:0/3589098544 shutdown_connections 2024-09-13T17:24:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 --2- 172.21.15.44:0/3589098544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dec105a70 0x7f2dec107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 --2- 172.21.15.44:0/3589098544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec105530 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 --2- 172.21.15.44:0/3589098544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dec100810 0x7f2dec102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 -- 172.21.15.44:0/3589098544 >> 172.21.15.44:0/3589098544 conn(0x7f2dec0fa3f0 msgr2=0x7f2dec0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 -- 172.21.15.44:0/3589098544 shutdown_connections 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 -- 172.21.15.44:0/3589098544 wait complete. 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.673+0000 7f2df1301640 1 Processor -- start 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 -- start start 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dec100810 0x7f2dec195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dec105a70 0x7f2dec19d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec19d790 con 0x7f2dec103140 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec19d900 con 0x7f2dec105a70 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2df1301640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec19dac0 con 0x7f2dec100810 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec1961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45620/0 (socket says 172.21.15.44:45620) 2024-09-13T17:24:48.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 -- 172.21.15.44:0/1468883254 learned_addr learned my addr 172.21.15.44:0/1468883254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2deaffd640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dec100810 0x7f2dec195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2deb7fe640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dec105a70 0x7f2dec19d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 -- 172.21.15.44:0/1468883254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dec100810 msgr2=0x7f2dec195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dec100810 0x7f2dec195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 -- 172.21.15.44:0/1468883254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dec105a70 msgr2=0x7f2dec19d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dec105a70 0x7f2dec19d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 -- 172.21.15.44:0/1468883254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd4009560 con 0x7f2dec103140 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.674+0000 7f2de27fc640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec1961c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4005ab0 tx=0x7f2dd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.675+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4031d70 con 0x7f2dec103140 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.675+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec19dd40 con 0x7f2dec103140 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.675+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dd40044e0 con 0x7f2dec103140 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.675+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4003a00 con 0x7f2dec103140 2024-09-13T17:24:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.675+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec19e1a0 con 0x7f2dec103140 2024-09-13T17:24:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.676+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec0fbee0 con 0x7f2dec103140 2024-09-13T17:24:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.677+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2dd4003b60 con 0x7f2dec103140 2024-09-13T17:24:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.678+0000 7f2de8ff9640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2dd80a1120 0x7f2dd80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.678+0000 7f2deaffd640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2dd80a1120 0x7f2dd80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.678+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2dd40ee1e0 con 0x7f2dec103140 2024-09-13T17:24:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.678+0000 7f2deaffd640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2dd80a1120 0x7f2dd80a35e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2dec196b50 tx=0x7f2ddc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:48.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.681+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2dd40b7500 con 0x7f2dec103140 2024-09-13T17:24:48.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.881+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dec02c450 con 0x7f2dd80a1120 2024-09-13T17:24:48.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.886+0000 7f2de8ff9640 1 -- 172.21.15.44:0/1468883254 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f2dec02c450 con 0x7f2dd80a1120 2024-09-13T17:24:48.887 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/qa/sv_1/9f24dc6c-f4cb-478a-9fbe-d7bccf2dde59 2024-09-13T17:24:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2dd80a1120 msgr2=0x7f2dd80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2dd80a1120 0x7f2dd80a35e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2dec196b50 tx=0x7f2ddc00a2d0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 msgr2=0x7f2dec1961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec1961c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4005ab0 tx=0x7f2dd40042f0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 shutdown_connections 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2dd80a1120 0x7f2dd80a35e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dec105a70 0x7f2dec19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dec103140 0x7f2dec1961c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 --2- 172.21.15.44:0/1468883254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dec100810 0x7f2dec195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 >> 172.21.15.44:0/1468883254 conn(0x7f2dec0fa3f0 msgr2=0x7f2dec1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 shutdown_connections 2024-09-13T17:24:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:48.888+0000 7f2df1301640 1 -- 172.21.15.44:0/1468883254 wait complete. 2024-09-13T17:24:48.900 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:24:49.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 -- 172.21.15.44:0/1914959880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c105ab0 msgr2=0x7fe32c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 --2- 172.21.15.44:0/1914959880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c105ab0 0x7fe32c107f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe324009ca0 tx=0x7fe32402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 -- 172.21.15.44:0/1914959880 shutdown_connections 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 --2- 172.21.15.44:0/1914959880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe32c103180 0x7fe32c105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 --2- 172.21.15.44:0/1914959880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c105ab0 0x7fe32c107f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 --2- 172.21.15.44:0/1914959880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c100850 0x7fe32c102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 -- 172.21.15.44:0/1914959880 >> 172.21.15.44:0/1914959880 conn(0x7fe32c0fa430 msgr2=0x7fe32c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 -- 172.21.15.44:0/1914959880 shutdown_connections 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.001+0000 7fe33105a640 1 -- 172.21.15.44:0/1914959880 wait complete. 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 Processor -- start 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 -- start start 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 0x7fe32c195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c103180 0x7fe32c1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe32c105ab0 0x7fe32c19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c19d880 con 0x7fe32c100850 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c19d9f0 con 0x7fe32c103180 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.002+0000 7fe33105a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c19dbb0 con 0x7fe32c105ab0 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 0x7fe32c195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 0x7fe32c195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45628/0 (socket says 172.21.15.44:45628) 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 -- 172.21.15.44:0/1774112824 learned_addr learned my addr 172.21.15.44:0/1774112824 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32a59c640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c103180 0x7fe32c1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32b59e640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe32c105ab0 0x7fe32c19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 -- 172.21.15.44:0/1774112824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe32c105ab0 msgr2=0x7fe32c19d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe32c105ab0 0x7fe32c19d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 -- 172.21.15.44:0/1774112824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c103180 msgr2=0x7fe32c1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c103180 0x7fe32c1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 -- 172.21.15.44:0/1774112824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324009900 con 0x7fe32c100850 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32a59c640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c103180 0x7fe32c1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe32ad9d640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 0x7fe32c195da0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d950 tx=0x7fe31c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c009a30 con 0x7fe32c100850 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe31c0044b0 con 0x7fe32c100850 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c19de90 con 0x7fe32c100850 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0047a0 con 0x7fe32c100850 2024-09-13T17:24:49.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.003+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c19e2b0 con 0x7fe32c100850 2024-09-13T17:24:49.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.004+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0fbf20 con 0x7fe32c100850 2024-09-13T17:24:49.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.005+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe31c004020 con 0x7fe32c100850 2024-09-13T17:24:49.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.006+0000 7fe313fff640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.006+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe31c0cc950 con 0x7fe32c100850 2024-09-13T17:24:49.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.006+0000 7fe32a59c640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.006+0000 7fe32a59c640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe3140a1120 0x7fe3140a35e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe32c069150 tx=0x7fe320008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.010+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe31c095be0 con 0x7fe32c100850 2024-09-13T17:24:49.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.218+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe32c10b5d0 con 0x7fe32c100850 2024-09-13T17:24:49.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.218+0000 7fe313fff640 1 -- 172.21.15.44:0/1774112824 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fe31c097b10 con 0x7fe32c100850 2024-09-13T17:24:49.219 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:49.220 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:47.147746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:24:49.220 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:24:49.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe3140a1120 msgr2=0x7fe3140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe3140a1120 0x7fe3140a35e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe32c069150 tx=0x7fe320008040 comp rx=0 tx=0).stop 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 msgr2=0x7fe32c195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 0x7fe32c195da0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d950 tx=0x7fe31c00de10 comp rx=0 tx=0).stop 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 shutdown_connections 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe32c105ab0 0x7fe32c19d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c103180 0x7fe32c1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 --2- 172.21.15.44:0/1774112824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe32c100850 0x7fe32c195da0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 >> 172.21.15.44:0/1774112824 conn(0x7fe32c0fa430 msgr2=0x7fe32c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 shutdown_connections 2024-09-13T17:24:49.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.220+0000 7fe33105a640 1 -- 172.21.15.44:0/1774112824 wait complete. 2024-09-13T17:24:49.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.333+0000 7fb35bbce640 1 -- 172.21.15.44:0/2348669844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 msgr2=0x7fb354102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.333+0000 7fb35bbce640 1 --2- 172.21.15.44:0/2348669844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354102c40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb350009900 tx=0x7fb35002a410 comp rx=0 tx=0).stop 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- 172.21.15.44:0/2348669844 shutdown_connections 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 --2- 172.21.15.44:0/2348669844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb354103180 0x7fb354105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 --2- 172.21.15.44:0/2348669844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb354105ab0 0x7fb354107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 --2- 172.21.15.44:0/2348669844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354102c40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- 172.21.15.44:0/2348669844 >> 172.21.15.44:0/2348669844 conn(0x7fb3540fa430 msgr2=0x7fb3540fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- 172.21.15.44:0/2348669844 shutdown_connections 2024-09-13T17:24:49.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- 172.21.15.44:0/2348669844 wait complete. 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 Processor -- start 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- start start 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb354103180 0x7fb3541962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb354105ab0 0x7fb35419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35419d8a0 con 0x7fb354100850 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35419da10 con 0x7fb354105ab0 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.334+0000 7fb35bbce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35419dbd0 con 0x7fb354103180 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45640/0 (socket says 172.21.15.44:45640) 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 -- 172.21.15.44:0/4114881839 learned_addr learned my addr 172.21.15.44:0/4114881839 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb359169640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb354103180 0x7fb3541962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35a16b640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb354105ab0 0x7fb35419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 -- 172.21.15.44:0/4114881839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb354103180 msgr2=0x7fb3541962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb354103180 0x7fb3541962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 -- 172.21.15.44:0/4114881839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb354105ab0 msgr2=0x7fb35419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb354105ab0 0x7fb35419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 -- 172.21.15.44:0/4114881839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb350009560 con 0x7fb354100850 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35996a640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354195db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb3500057c0 tx=0x7fb3500052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350031d70 con 0x7fb354100850 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3500054a0 con 0x7fb354100850 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35419de50 con 0x7fb354100850 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350003b10 con 0x7fb354100850 2024-09-13T17:24:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.335+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35419e270 con 0x7fb354100850 2024-09-13T17:24:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.336+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3540fbf20 con 0x7fb354100850 2024-09-13T17:24:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.338+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb350003c70 con 0x7fb354100850 2024-09-13T17:24:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.338+0000 7fb34affd640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb3400a11a0 0x7fb3400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.338+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb3500ee2d0 con 0x7fb354100850 2024-09-13T17:24:49.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.338+0000 7fb359169640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb3400a11a0 0x7fb3400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.339+0000 7fb359169640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb3400a11a0 0x7fb3400a3660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb3541972c0 tx=0x7fb34400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.341+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb3500b75f0 con 0x7fb354100850 2024-09-13T17:24:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:49 smithi070 ceph-mon[34502]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:49 smithi070 ceph-mon[34502]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-09-13T17:24:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:49 smithi070 ceph-mon[34502]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:49 smithi070 ceph-mon[34502]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.515+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb35410b5a0 con 0x7fb354100850 2024-09-13T17:24:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.516+0000 7fb34affd640 1 -- 172.21.15.44:0/4114881839 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24427 (secure 0 0 0) 0x7fb35004a090 con 0x7fb354100850 2024-09-13T17:24:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:49.518 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:42.549640+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb3400a11a0 msgr2=0x7fb3400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb3400a11a0 0x7fb3400a3660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb3541972c0 tx=0x7fb34400b390 comp rx=0 tx=0).stop 2024-09-13T17:24:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 msgr2=0x7fb354195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354195db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb3500057c0 tx=0x7fb3500052b0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 shutdown_connections 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb3400a11a0 0x7fb3400a3660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb354103180 0x7fb3541962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb354105ab0 0x7fb35419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 --2- 172.21.15.44:0/4114881839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb354100850 0x7fb354195db0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 >> 172.21.15.44:0/4114881839 conn(0x7fb3540fa430 msgr2=0x7fb3541012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 shutdown_connections 2024-09-13T17:24:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.518+0000 7fb35bbce640 1 -- 172.21.15.44:0/4114881839 wait complete. 2024-09-13T17:24:49.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:49 smithi100 ceph-mon[35055]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:49 smithi100 ceph-mon[35055]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-09-13T17:24:49.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:49 smithi100 ceph-mon[35055]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:49 smithi100 ceph-mon[35055]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.529 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:49 smithi044 ceph-mon[31175]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.529 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:49 smithi044 ceph-mon[31175]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-09-13T17:24:49.529 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:49 smithi044 ceph-mon[31175]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.529 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:49 smithi044 ceph-mon[31175]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:49.531 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:24:49.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:24:49.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.632+0000 7f52fae41640 1 -- 172.21.15.44:0/3417401048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4100f90 msgr2=0x7f52f4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.632+0000 7f52fae41640 1 --2- 172.21.15.44:0/3417401048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4100f90 0x7f52f4101410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.632+0000 7f52fae41640 1 -- 172.21.15.44:0/3417401048 shutdown_connections 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.632+0000 7f52fae41640 1 --2- 172.21.15.44:0/3417401048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f4101950 0x7f52f41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 --2- 172.21.15.44:0/3417401048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4100f90 0x7f52f4101410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 --2- 172.21.15.44:0/3417401048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f40ffda0 0x7f52f41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- 172.21.15.44:0/3417401048 >> 172.21.15.44:0/3417401048 conn(0x7f52f40fb520 msgr2=0x7f52f40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- 172.21.15.44:0/3417401048 shutdown_connections 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- 172.21.15.44:0/3417401048 wait complete. 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 Processor -- start 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- start start 2024-09-13T17:24:49.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f40ffda0 0x7f52f4195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f4100f90 0x7f52f41961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 0x7f52f419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f419d7b0 con 0x7f52f4101950 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f419d920 con 0x7f52f4100f90 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.633+0000 7f52fae41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f419dae0 con 0x7f52f40ffda0 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 0x7f52f419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 0x7f52f419d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34084/0 (socket says 172.21.15.44:34084) 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 -- 172.21.15.44:0/880580567 learned_addr learned my addr 172.21.15.44:0/880580567 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52ebfff640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f4100f90 0x7f52f41961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f8bdd640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f40ffda0 0x7f52f4195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 -- 172.21.15.44:0/880580567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f40ffda0 msgr2=0x7f52f4195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f40ffda0 0x7f52f4195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 -- 172.21.15.44:0/880580567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f4100f90 msgr2=0x7f52f41961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f4100f90 0x7f52f41961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 -- 172.21.15.44:0/880580567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009560 con 0x7f52f4101950 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f8bdd640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f40ffda0 0x7f52f4195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52f93de640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 0x7f52f419d240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00eaa0 tx=0x7f52ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00cdb0 con 0x7f52f4101950 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52ec005020 con 0x7f52f4101950 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0116d0 con 0x7f52f4101950 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.634+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f419ddc0 con 0x7f52f4101950 2024-09-13T17:24:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.635+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f419e1f0 con 0x7f52f4101950 2024-09-13T17:24:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.636+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f41072c0 con 0x7f52f4101950 2024-09-13T17:24:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.636+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f52ec005190 con 0x7f52f4101950 2024-09-13T17:24:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.637+0000 7f52e9ffb640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52dc0a1120 0x7f52dc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.637+0000 7f52f8bdd640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52dc0a1120 0x7f52dc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.637+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52ec0ccfb0 con 0x7f52f4101950 2024-09-13T17:24:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.638+0000 7f52f8bdd640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52dc0a1120 0x7f52dc0a35e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52f0005fd0 tx=0x7f52f0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.641+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f52ec096270 con 0x7f52f4101950 2024-09-13T17:24:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.811+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52f402c450 con 0x7f52dc0a1120 2024-09-13T17:24:49.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.823+0000 7f52e9ffb640 1 -- 172.21.15.44:0/880580567 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423385 (secure 0 0 0) 0x7f52f402c450 con 0x7f52dc0a1120 2024-09-13T17:24:49.829 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:49.832 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52dc0a1120 msgr2=0x7f52dc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52dc0a1120 0x7f52dc0a35e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52f0005fd0 tx=0x7f52f0005850 comp rx=0 tx=0).stop 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 msgr2=0x7f52f419d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 0x7f52f419d240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00eaa0 tx=0x7f52ec00ef60 comp rx=0 tx=0).stop 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 shutdown_connections 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f52dc0a1120 0x7f52dc0a35e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f4101950 0x7f52f419d240 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f4100f90 0x7f52f41961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 --2- 172.21.15.44:0/880580567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f40ffda0 0x7f52f4195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.827+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 >> 172.21.15.44:0/880580567 conn(0x7f52f40fb520 msgr2=0x7f52f4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.828+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 shutdown_connections 2024-09-13T17:24:49.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.828+0000 7f52fae41640 1 -- 172.21.15.44:0/880580567 wait complete. 2024-09-13T17:24:49.839 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-13T17:24:49.558721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595849,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":275,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":424,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385152,"kb_used_data":11772,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124556672,"statfs":{"total":1151940427776,"available":1151546032128,"internally_reserved":0,"allocated":12054528,"data_stored":7716460,"data_compressed":319998,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":888,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.008621"},"pg_stats":[{"pgid":"2.1f","version":"83'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553006+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:24:41.553006+0000","last_peered":"2024-09-13T17:24:41.553006+0000","last_clean":"2024-09-13T17:24:41.553006+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:24:41.553006+0000","last_undegraded":"2024-09-13T17:24:41.553006+0000","last_fullsized":"2024-09-13T17:24:41.553006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560804+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:24:41.560804+0000","last_peered":"2024-09-13T17:24:41.560804+0000","last_clean":"2024-09-13T17:24:41.560804+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:24:41.560804+0000","last_undegraded":"2024-09-13T17:24:41.560804+0000","last_fullsized":"2024-09-13T17:24:41.560804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552435+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:24:41.552435+0000","last_peered":"2024-09-13T17:24:41.552435+0000","last_clean":"2024-09-13T17:24:41.552435+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:24:41.552435+0000","last_undegraded":"2024-09-13T17:24:41.552435+0000","last_fullsized":"2024-09-13T17:24:41.552435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.779362+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:24:42.779362+0000","last_peered":"2024-09-13T17:24:42.779362+0000","last_clean":"2024-09-13T17:24:42.779362+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:24:42.779362+0000","last_undegraded":"2024-09-13T17:24:42.779362+0000","last_fullsized":"2024-09-13T17:24:42.779362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562952+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:24:41.562952+0000","last_peered":"2024-09-13T17:24:41.562952+0000","last_clean":"2024-09-13T17:24:41.562952+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:24:41.562952+0000","last_undegraded":"2024-09-13T17:24:41.562952+0000","last_fullsized":"2024-09-13T17:24:41.562952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.554744+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:24:41.554744+0000","last_peered":"2024-09-13T17:24:41.554744+0000","last_clean":"2024-09-13T17:24:41.554744+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:24:41.554744+0000","last_undegraded":"2024-09-13T17:24:41.554744+0000","last_fullsized":"2024-09-13T17:24:41.554744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.152617+0000","last_change":"2024-09-13T17:24:42.568012+0000","last_active":"2024-09-13T17:24:46.152617+0000","last_peered":"2024-09-13T17:24:46.152617+0000","last_clean":"2024-09-13T17:24:46.152617+0000","last_became_active":"2024-09-13T17:24:42.563885+0000","last_became_peered":"2024-09-13T17:24:42.563885+0000","last_unstale":"2024-09-13T17:24:46.152617+0000","last_undegraded":"2024-09-13T17:24:46.152617+0000","last_fullsized":"2024-09-13T17:24:46.152617+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:17.333114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682396+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:24:41.682396+0000","last_peered":"2024-09-13T17:24:41.682396+0000","last_clean":"2024-09-13T17:24:41.682396+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:24:41.682396+0000","last_undegraded":"2024-09-13T17:24:41.682396+0000","last_fullsized":"2024-09-13T17:24:41.682396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555308+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:24:41.555308+0000","last_peered":"2024-09-13T17:24:41.555308+0000","last_clean":"2024-09-13T17:24:41.555308+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:24:41.555308+0000","last_undegraded":"2024-09-13T17:24:41.555308+0000","last_fullsized":"2024-09-13T17:24:41.555308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557127+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:24:41.557127+0000","last_peered":"2024-09-13T17:24:41.557127+0000","last_clean":"2024-09-13T17:24:41.557127+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:24:41.557127+0000","last_undegraded":"2024-09-13T17:24:41.557127+0000","last_fullsized":"2024-09-13T17:24:41.557127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560045+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:24:41.560045+0000","last_peered":"2024-09-13T17:24:41.560045+0000","last_clean":"2024-09-13T17:24:41.560045+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:24:41.560045+0000","last_undegraded":"2024-09-13T17:24:41.560045+0000","last_fullsized":"2024-09-13T17:24:41.560045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.554898+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:24:41.554898+0000","last_peered":"2024-09-13T17:24:41.554898+0000","last_clean":"2024-09-13T17:24:41.554898+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:24:41.554898+0000","last_undegraded":"2024-09-13T17:24:41.554898+0000","last_fullsized":"2024-09-13T17:24:41.554898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552705+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:24:41.552705+0000","last_peered":"2024-09-13T17:24:41.552705+0000","last_clean":"2024-09-13T17:24:41.552705+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:24:41.552705+0000","last_undegraded":"2024-09-13T17:24:41.552705+0000","last_fullsized":"2024-09-13T17:24:41.552705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.597825+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:24:45.597825+0000","last_peered":"2024-09-13T17:24:45.597825+0000","last_clean":"2024-09-13T17:24:45.597825+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:24:45.597825+0000","last_undegraded":"2024-09-13T17:24:45.597825+0000","last_fullsized":"2024-09-13T17:24:45.597825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555004+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:24:41.555004+0000","last_peered":"2024-09-13T17:24:41.555004+0000","last_clean":"2024-09-13T17:24:41.555004+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:24:41.555004+0000","last_undegraded":"2024-09-13T17:24:41.555004+0000","last_fullsized":"2024-09-13T17:24:41.555004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559987+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:24:41.559987+0000","last_peered":"2024-09-13T17:24:41.559987+0000","last_clean":"2024-09-13T17:24:41.559987+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:24:41.559987+0000","last_undegraded":"2024-09-13T17:24:41.559987+0000","last_fullsized":"2024-09-13T17:24:41.559987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552714+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:24:41.552714+0000","last_peered":"2024-09-13T17:24:41.552714+0000","last_clean":"2024-09-13T17:24:41.552714+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:24:41.552714+0000","last_undegraded":"2024-09-13T17:24:41.552714+0000","last_fullsized":"2024-09-13T17:24:41.552714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.564232+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:24:42.564232+0000","last_peered":"2024-09-13T17:24:42.564232+0000","last_clean":"2024-09-13T17:24:42.564232+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:24:42.564232+0000","last_undegraded":"2024-09-13T17:24:42.564232+0000","last_fullsized":"2024-09-13T17:24:42.564232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556338+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:24:41.556338+0000","last_peered":"2024-09-13T17:24:41.556338+0000","last_clean":"2024-09-13T17:24:41.556338+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:24:41.556338+0000","last_undegraded":"2024-09-13T17:24:41.556338+0000","last_fullsized":"2024-09-13T17:24:41.556338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.597159+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:24:45.597159+0000","last_peered":"2024-09-13T17:24:45.597159+0000","last_clean":"2024-09-13T17:24:45.597159+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:24:45.597159+0000","last_undegraded":"2024-09-13T17:24:45.597159+0000","last_fullsized":"2024-09-13T17:24:45.597159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552695+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:24:41.552695+0000","last_peered":"2024-09-13T17:24:41.552695+0000","last_clean":"2024-09-13T17:24:41.552695+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:24:41.552695+0000","last_undegraded":"2024-09-13T17:24:41.552695+0000","last_fullsized":"2024-09-13T17:24:41.552695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556372+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:24:41.556372+0000","last_peered":"2024-09-13T17:24:41.556372+0000","last_clean":"2024-09-13T17:24:41.556372+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:24:41.556372+0000","last_undegraded":"2024-09-13T17:24:41.556372+0000","last_fullsized":"2024-09-13T17:24:41.556372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.596903+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:24:45.596903+0000","last_peered":"2024-09-13T17:24:45.596903+0000","last_clean":"2024-09-13T17:24:45.596903+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:24:45.596903+0000","last_undegraded":"2024-09-13T17:24:45.596903+0000","last_fullsized":"2024-09-13T17:24:45.596903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.562373+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:24:42.562373+0000","last_peered":"2024-09-13T17:24:42.562373+0000","last_clean":"2024-09-13T17:24:42.562373+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:24:42.562373+0000","last_undegraded":"2024-09-13T17:24:42.562373+0000","last_fullsized":"2024-09-13T17:24:42.562373+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562512+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:24:41.562512+0000","last_peered":"2024-09-13T17:24:41.562512+0000","last_clean":"2024-09-13T17:24:41.562512+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:24:41.562512+0000","last_undegraded":"2024-09-13T17:24:41.562512+0000","last_fullsized":"2024-09-13T17:24:41.562512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.593921+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:24:45.593921+0000","last_peered":"2024-09-13T17:24:45.593921+0000","last_clean":"2024-09-13T17:24:45.593921+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:24:45.593921+0000","last_undegraded":"2024-09-13T17:24:45.593921+0000","last_fullsized":"2024-09-13T17:24:45.593921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552392+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:24:41.552392+0000","last_peered":"2024-09-13T17:24:41.552392+0000","last_clean":"2024-09-13T17:24:41.552392+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:24:41.552392+0000","last_undegraded":"2024-09-13T17:24:41.552392+0000","last_fullsized":"2024-09-13T17:24:41.552392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561223+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:24:41.561223+0000","last_peered":"2024-09-13T17:24:41.561223+0000","last_clean":"2024-09-13T17:24:41.561223+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:24:41.561223+0000","last_undegraded":"2024-09-13T17:24:41.561223+0000","last_fullsized":"2024-09-13T17:24:41.561223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553081+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:24:41.553081+0000","last_peered":"2024-09-13T17:24:41.553081+0000","last_clean":"2024-09-13T17:24:41.553081+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:24:41.553081+0000","last_undegraded":"2024-09-13T17:24:41.553081+0000","last_fullsized":"2024-09-13T17:24:41.553081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559476+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:24:41.559476+0000","last_peered":"2024-09-13T17:24:41.559476+0000","last_clean":"2024-09-13T17:24:41.559476+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:24:41.559476+0000","last_undegraded":"2024-09-13T17:24:41.559476+0000","last_fullsized":"2024-09-13T17:24:41.559476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555689+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:24:41.555689+0000","last_peered":"2024-09-13T17:24:41.555689+0000","last_clean":"2024-09-13T17:24:41.555689+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:24:41.555689+0000","last_undegraded":"2024-09-13T17:24:41.555689+0000","last_fullsized":"2024-09-13T17:24:41.555689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.594502+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:24:45.594502+0000","last_peered":"2024-09-13T17:24:45.594502+0000","last_clean":"2024-09-13T17:24:45.594502+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:24:45.594502+0000","last_undegraded":"2024-09-13T17:24:45.594502+0000","last_fullsized":"2024-09-13T17:24:45.594502+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557540+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:24:41.557540+0000","last_peered":"2024-09-13T17:24:41.557540+0000","last_clean":"2024-09-13T17:24:41.557540+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:24:41.557540+0000","last_undegraded":"2024-09-13T17:24:41.557540+0000","last_fullsized":"2024-09-13T17:24:41.557540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557355+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:24:41.557355+0000","last_peered":"2024-09-13T17:24:41.557355+0000","last_clean":"2024-09-13T17:24:41.557355+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:24:41.557355+0000","last_undegraded":"2024-09-13T17:24:41.557355+0000","last_fullsized":"2024-09-13T17:24:41.557355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552367+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:24:41.552367+0000","last_peered":"2024-09-13T17:24:41.552367+0000","last_clean":"2024-09-13T17:24:41.552367+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:24:41.552367+0000","last_undegraded":"2024-09-13T17:24:41.552367+0000","last_fullsized":"2024-09-13T17:24:41.552367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556502+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:24:41.556502+0000","last_peered":"2024-09-13T17:24:41.556502+0000","last_clean":"2024-09-13T17:24:41.556502+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:24:41.556502+0000","last_undegraded":"2024-09-13T17:24:41.556502+0000","last_fullsized":"2024-09-13T17:24:41.556502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556724+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:24:41.556724+0000","last_peered":"2024-09-13T17:24:41.556724+0000","last_clean":"2024-09-13T17:24:41.556724+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:24:41.556724+0000","last_undegraded":"2024-09-13T17:24:41.556724+0000","last_fullsized":"2024-09-13T17:24:41.556724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.563321+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:24:42.563321+0000","last_peered":"2024-09-13T17:24:42.563321+0000","last_clean":"2024-09-13T17:24:42.563321+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:24:42.563321+0000","last_undegraded":"2024-09-13T17:24:42.563321+0000","last_fullsized":"2024-09-13T17:24:42.563321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555440+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:24:41.555440+0000","last_peered":"2024-09-13T17:24:41.555440+0000","last_clean":"2024-09-13T17:24:41.555440+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:24:41.555440+0000","last_undegraded":"2024-09-13T17:24:41.555440+0000","last_fullsized":"2024-09-13T17:24:41.555440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559595+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:24:41.559595+0000","last_peered":"2024-09-13T17:24:41.559595+0000","last_clean":"2024-09-13T17:24:41.559595+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:24:41.559595+0000","last_undegraded":"2024-09-13T17:24:41.559595+0000","last_fullsized":"2024-09-13T17:24:41.559595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.153863+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:24:46.153863+0000","last_peered":"2024-09-13T17:24:46.153863+0000","last_clean":"2024-09-13T17:24:46.153863+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:24:46.153863+0000","last_undegraded":"2024-09-13T17:24:46.153863+0000","last_fullsized":"2024-09-13T17:24:46.153863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.558700+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:24:41.558700+0000","last_peered":"2024-09-13T17:24:41.558700+0000","last_clean":"2024-09-13T17:24:41.558700+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:24:41.558700+0000","last_undegraded":"2024-09-13T17:24:41.558700+0000","last_fullsized":"2024-09-13T17:24:41.558700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555011+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:24:41.555011+0000","last_peered":"2024-09-13T17:24:41.555011+0000","last_clean":"2024-09-13T17:24:41.555011+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:24:41.555011+0000","last_undegraded":"2024-09-13T17:24:41.555011+0000","last_fullsized":"2024-09-13T17:24:41.555011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.152458+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:24:46.152458+0000","last_peered":"2024-09-13T17:24:46.152458+0000","last_clean":"2024-09-13T17:24:46.152458+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:24:46.152458+0000","last_undegraded":"2024-09-13T17:24:46.152458+0000","last_fullsized":"2024-09-13T17:24:46.152458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559792+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:24:41.559792+0000","last_peered":"2024-09-13T17:24:41.559792+0000","last_clean":"2024-09-13T17:24:41.559792+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:24:41.559792+0000","last_undegraded":"2024-09-13T17:24:41.559792+0000","last_fullsized":"2024-09-13T17:24:41.559792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560096+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:24:41.560096+0000","last_peered":"2024-09-13T17:24:41.560096+0000","last_clean":"2024-09-13T17:24:41.560096+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:24:41.560096+0000","last_undegraded":"2024-09-13T17:24:41.560096+0000","last_fullsized":"2024-09-13T17:24:41.560096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.143457+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:24:46.143457+0000","last_peered":"2024-09-13T17:24:46.143457+0000","last_clean":"2024-09-13T17:24:46.143457+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:24:46.143457+0000","last_undegraded":"2024-09-13T17:24:46.143457+0000","last_fullsized":"2024-09-13T17:24:46.143457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552655+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:24:41.552655+0000","last_peered":"2024-09-13T17:24:41.552655+0000","last_clean":"2024-09-13T17:24:41.552655+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:24:41.552655+0000","last_undegraded":"2024-09-13T17:24:41.552655+0000","last_fullsized":"2024-09-13T17:24:41.552655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552736+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:24:41.552736+0000","last_peered":"2024-09-13T17:24:41.552736+0000","last_clean":"2024-09-13T17:24:41.552736+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:24:41.552736+0000","last_undegraded":"2024-09-13T17:24:41.552736+0000","last_fullsized":"2024-09-13T17:24:41.552736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552963+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:24:41.552963+0000","last_peered":"2024-09-13T17:24:41.552963+0000","last_clean":"2024-09-13T17:24:41.552963+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:24:41.552963+0000","last_undegraded":"2024-09-13T17:24:41.552963+0000","last_fullsized":"2024-09-13T17:24:41.552963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553171+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:24:41.553171+0000","last_peered":"2024-09-13T17:24:41.553171+0000","last_clean":"2024-09-13T17:24:41.553171+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:24:41.553171+0000","last_undegraded":"2024-09-13T17:24:41.553171+0000","last_fullsized":"2024-09-13T17:24:41.553171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682144+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:24:41.682144+0000","last_peered":"2024-09-13T17:24:41.682144+0000","last_clean":"2024-09-13T17:24:41.682144+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:24:41.682144+0000","last_undegraded":"2024-09-13T17:24:41.682144+0000","last_fullsized":"2024-09-13T17:24:41.682144+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553045+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:24:41.553045+0000","last_peered":"2024-09-13T17:24:41.553045+0000","last_clean":"2024-09-13T17:24:41.553045+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:24:41.553045+0000","last_undegraded":"2024-09-13T17:24:41.553045+0000","last_fullsized":"2024-09-13T17:24:41.553045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560111+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:24:41.560111+0000","last_peered":"2024-09-13T17:24:41.560111+0000","last_clean":"2024-09-13T17:24:41.560111+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:24:41.560111+0000","last_undegraded":"2024-09-13T17:24:41.560111+0000","last_fullsized":"2024-09-13T17:24:41.560111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561446+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:24:41.561446+0000","last_peered":"2024-09-13T17:24:41.561446+0000","last_clean":"2024-09-13T17:24:41.561446+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:24:41.561446+0000","last_undegraded":"2024-09-13T17:24:41.561446+0000","last_fullsized":"2024-09-13T17:24:41.561446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553254+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:24:41.553254+0000","last_peered":"2024-09-13T17:24:41.553254+0000","last_clean":"2024-09-13T17:24:41.553254+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:24:41.553254+0000","last_undegraded":"2024-09-13T17:24:41.553254+0000","last_fullsized":"2024-09-13T17:24:41.553254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559041+0000","last_change":"2024-09-13T17:24:21.384590+0000","last_active":"2024-09-13T17:24:41.559041+0000","last_peered":"2024-09-13T17:24:41.559041+0000","last_clean":"2024-09-13T17:24:41.559041+0000","last_became_active":"2024-09-13T17:24:21.383582+0000","last_became_peered":"2024-09-13T17:24:21.383582+0000","last_unstale":"2024-09-13T17:24:41.559041+0000","last_undegraded":"2024-09-13T17:24:41.559041+0000","last_fullsized":"2024-09-13T17:24:41.559041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:01:46.302548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559133+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:24:41.559133+0000","last_peered":"2024-09-13T17:24:41.559133+0000","last_clean":"2024-09-13T17:24:41.559133+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:24:41.559133+0000","last_undegraded":"2024-09-13T17:24:41.559133+0000","last_fullsized":"2024-09-13T17:24:41.559133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557587+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:24:41.557587+0000","last_peered":"2024-09-13T17:24:41.557587+0000","last_clean":"2024-09-13T17:24:41.557587+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:24:41.557587+0000","last_undegraded":"2024-09-13T17:24:41.557587+0000","last_fullsized":"2024-09-13T17:24:41.557587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.560416+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:24:42.560416+0000","last_peered":"2024-09-13T17:24:42.560416+0000","last_clean":"2024-09-13T17:24:42.560416+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:24:42.560416+0000","last_undegraded":"2024-09-13T17:24:42.560416+0000","last_fullsized":"2024-09-13T17:24:42.560416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559644+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:24:41.559644+0000","last_peered":"2024-09-13T17:24:41.559644+0000","last_clean":"2024-09-13T17:24:41.559644+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:24:41.559644+0000","last_undegraded":"2024-09-13T17:24:41.559644+0000","last_fullsized":"2024-09-13T17:24:41.559644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559556+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:24:41.559556+0000","last_peered":"2024-09-13T17:24:41.559556+0000","last_clean":"2024-09-13T17:24:41.559556+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:24:41.559556+0000","last_undegraded":"2024-09-13T17:24:41.559556+0000","last_fullsized":"2024-09-13T17:24:41.559556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.154312+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:24:46.154312+0000","last_peered":"2024-09-13T17:24:46.154312+0000","last_clean":"2024-09-13T17:24:46.154312+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:24:46.154312+0000","last_undegraded":"2024-09-13T17:24:46.154312+0000","last_fullsized":"2024-09-13T17:24:46.154312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.780454+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:24:42.780454+0000","last_peered":"2024-09-13T17:24:42.780454+0000","last_clean":"2024-09-13T17:24:42.780454+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:24:42.780454+0000","last_undegraded":"2024-09-13T17:24:42.780454+0000","last_fullsized":"2024-09-13T17:24:42.780454+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552420+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:24:41.552420+0000","last_peered":"2024-09-13T17:24:41.552420+0000","last_clean":"2024-09-13T17:24:41.552420+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:24:41.552420+0000","last_undegraded":"2024-09-13T17:24:41.552420+0000","last_fullsized":"2024-09-13T17:24:41.552420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.153360+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:24:46.153360+0000","last_peered":"2024-09-13T17:24:46.153360+0000","last_clean":"2024-09-13T17:24:46.153360+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:24:46.153360+0000","last_undegraded":"2024-09-13T17:24:46.153360+0000","last_fullsized":"2024-09-13T17:24:46.153360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.781366+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:24:42.781366+0000","last_peered":"2024-09-13T17:24:42.781366+0000","last_clean":"2024-09-13T17:24:42.781366+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:24:42.781366+0000","last_undegraded":"2024-09-13T17:24:42.781366+0000","last_fullsized":"2024-09-13T17:24:42.781366+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552908+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:24:41.552908+0000","last_peered":"2024-09-13T17:24:41.552908+0000","last_clean":"2024-09-13T17:24:41.552908+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:24:41.552908+0000","last_undegraded":"2024-09-13T17:24:41.552908+0000","last_fullsized":"2024-09-13T17:24:41.552908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559490+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:24:41.559490+0000","last_peered":"2024-09-13T17:24:41.559490+0000","last_clean":"2024-09-13T17:24:41.559490+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:24:41.559490+0000","last_undegraded":"2024-09-13T17:24:41.559490+0000","last_fullsized":"2024-09-13T17:24:41.559490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556439+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:24:41.556439+0000","last_peered":"2024-09-13T17:24:41.556439+0000","last_clean":"2024-09-13T17:24:41.556439+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:24:41.556439+0000","last_undegraded":"2024-09-13T17:24:41.556439+0000","last_fullsized":"2024-09-13T17:24:41.556439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555546+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:24:41.555546+0000","last_peered":"2024-09-13T17:24:41.555546+0000","last_clean":"2024-09-13T17:24:41.555546+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:24:41.555546+0000","last_undegraded":"2024-09-13T17:24:41.555546+0000","last_fullsized":"2024-09-13T17:24:41.555546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.595473+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:24:45.595473+0000","last_peered":"2024-09-13T17:24:45.595473+0000","last_clean":"2024-09-13T17:24:45.595473+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:24:45.595473+0000","last_undegraded":"2024-09-13T17:24:45.595473+0000","last_fullsized":"2024-09-13T17:24:45.595473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559797+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:24:41.559797+0000","last_peered":"2024-09-13T17:24:41.559797+0000","last_clean":"2024-09-13T17:24:41.559797+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:24:41.559797+0000","last_undegraded":"2024-09-13T17:24:41.559797+0000","last_fullsized":"2024-09-13T17:24:41.559797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556842+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:24:41.556842+0000","last_peered":"2024-09-13T17:24:41.556842+0000","last_clean":"2024-09-13T17:24:41.556842+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:24:41.556842+0000","last_undegraded":"2024-09-13T17:24:41.556842+0000","last_fullsized":"2024-09-13T17:24:41.556842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557035+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:24:41.557035+0000","last_peered":"2024-09-13T17:24:41.557035+0000","last_clean":"2024-09-13T17:24:41.557035+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:24:41.557035+0000","last_undegraded":"2024-09-13T17:24:41.557035+0000","last_fullsized":"2024-09-13T17:24:41.557035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552092+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:24:41.552092+0000","last_peered":"2024-09-13T17:24:41.552092+0000","last_clean":"2024-09-13T17:24:41.552092+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:24:41.552092+0000","last_undegraded":"2024-09-13T17:24:41.552092+0000","last_fullsized":"2024-09-13T17:24:41.552092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682480+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:24:41.682480+0000","last_peered":"2024-09-13T17:24:41.682480+0000","last_clean":"2024-09-13T17:24:41.682480+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:24:41.682480+0000","last_undegraded":"2024-09-13T17:24:41.682480+0000","last_fullsized":"2024-09-13T17:24:41.682480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560013+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:24:41.560013+0000","last_peered":"2024-09-13T17:24:41.560013+0000","last_clean":"2024-09-13T17:24:41.560013+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:24:41.560013+0000","last_undegraded":"2024-09-13T17:24:41.560013+0000","last_fullsized":"2024-09-13T17:24:41.560013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555127+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:24:41.555127+0000","last_peered":"2024-09-13T17:24:41.555127+0000","last_clean":"2024-09-13T17:24:41.555127+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:24:41.555127+0000","last_undegraded":"2024-09-13T17:24:41.555127+0000","last_fullsized":"2024-09-13T17:24:41.555127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.563023+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:24:41.563023+0000","last_peered":"2024-09-13T17:24:41.563023+0000","last_clean":"2024-09-13T17:24:41.563023+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:24:41.563023+0000","last_undegraded":"2024-09-13T17:24:41.563023+0000","last_fullsized":"2024-09-13T17:24:41.563023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.596999+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:24:45.596999+0000","last_peered":"2024-09-13T17:24:45.596999+0000","last_clean":"2024-09-13T17:24:45.596999+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:24:45.596999+0000","last_undegraded":"2024-09-13T17:24:45.596999+0000","last_fullsized":"2024-09-13T17:24:45.596999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.551888+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:24:41.551888+0000","last_peered":"2024-09-13T17:24:41.551888+0000","last_clean":"2024-09-13T17:24:41.551888+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:24:41.551888+0000","last_undegraded":"2024-09-13T17:24:41.551888+0000","last_fullsized":"2024-09-13T17:24:41.551888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682557+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:24:41.682557+0000","last_peered":"2024-09-13T17:24:41.682557+0000","last_clean":"2024-09-13T17:24:41.682557+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:24:41.682557+0000","last_undegraded":"2024-09-13T17:24:41.682557+0000","last_fullsized":"2024-09-13T17:24:41.682557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557523+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:24:41.557523+0000","last_peered":"2024-09-13T17:24:41.557523+0000","last_clean":"2024-09-13T17:24:41.557523+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:24:41.557523+0000","last_undegraded":"2024-09-13T17:24:41.557523+0000","last_fullsized":"2024-09-13T17:24:41.557523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559758+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:24:41.559758+0000","last_peered":"2024-09-13T17:24:41.559758+0000","last_clean":"2024-09-13T17:24:41.559758+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:24:41.559758+0000","last_undegraded":"2024-09-13T17:24:41.559758+0000","last_fullsized":"2024-09-13T17:24:41.559758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562405+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:24:41.562405+0000","last_peered":"2024-09-13T17:24:41.562405+0000","last_clean":"2024-09-13T17:24:41.562405+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:24:41.562405+0000","last_undegraded":"2024-09-13T17:24:41.562405+0000","last_fullsized":"2024-09-13T17:24:41.562405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.551991+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:24:41.551991+0000","last_peered":"2024-09-13T17:24:41.551991+0000","last_clean":"2024-09-13T17:24:41.551991+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:24:41.551991+0000","last_undegraded":"2024-09-13T17:24:41.551991+0000","last_fullsized":"2024-09-13T17:24:41.551991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559619+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:24:41.559619+0000","last_peered":"2024-09-13T17:24:41.559619+0000","last_clean":"2024-09-13T17:24:41.559619+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:24:41.559619+0000","last_undegraded":"2024-09-13T17:24:41.559619+0000","last_fullsized":"2024-09-13T17:24:41.559619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557770+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:24:41.557770+0000","last_peered":"2024-09-13T17:24:41.557770+0000","last_clean":"2024-09-13T17:24:41.557770+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:24:41.557770+0000","last_undegraded":"2024-09-13T17:24:41.557770+0000","last_fullsized":"2024-09-13T17:24:41.557770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559655+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:24:41.559655+0000","last_peered":"2024-09-13T17:24:41.559655+0000","last_clean":"2024-09-13T17:24:41.559655+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:24:41.559655+0000","last_undegraded":"2024-09-13T17:24:41.559655+0000","last_fullsized":"2024-09-13T17:24:41.559655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.152790+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:24:46.152790+0000","last_peered":"2024-09-13T17:24:46.152790+0000","last_clean":"2024-09-13T17:24:46.152790+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:24:46.152790+0000","last_undegraded":"2024-09-13T17:24:46.152790+0000","last_fullsized":"2024-09-13T17:24:46.152790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557213+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:24:41.557213+0000","last_peered":"2024-09-13T17:24:41.557213+0000","last_clean":"2024-09-13T17:24:41.557213+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:24:41.557213+0000","last_undegraded":"2024-09-13T17:24:41.557213+0000","last_fullsized":"2024-09-13T17:24:41.557213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559395+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:24:41.559395+0000","last_peered":"2024-09-13T17:24:41.559395+0000","last_clean":"2024-09-13T17:24:41.559395+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:24:41.559395+0000","last_undegraded":"2024-09-13T17:24:41.559395+0000","last_fullsized":"2024-09-13T17:24:41.559395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552625+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:24:41.552625+0000","last_peered":"2024-09-13T17:24:41.552625+0000","last_clean":"2024-09-13T17:24:41.552625+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:24:41.552625+0000","last_undegraded":"2024-09-13T17:24:41.552625+0000","last_fullsized":"2024-09-13T17:24:41.552625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559471+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:24:41.559471+0000","last_peered":"2024-09-13T17:24:41.559471+0000","last_clean":"2024-09-13T17:24:41.559471+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:24:41.559471+0000","last_undegraded":"2024-09-13T17:24:41.559471+0000","last_fullsized":"2024-09-13T17:24:41.559471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556788+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:24:41.556788+0000","last_peered":"2024-09-13T17:24:41.556788+0000","last_clean":"2024-09-13T17:24:41.556788+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:24:41.556788+0000","last_undegraded":"2024-09-13T17:24:41.556788+0000","last_fullsized":"2024-09-13T17:24:41.556788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552663+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:24:41.552663+0000","last_peered":"2024-09-13T17:24:41.552663+0000","last_clean":"2024-09-13T17:24:41.552663+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:24:41.552663+0000","last_undegraded":"2024-09-13T17:24:41.552663+0000","last_fullsized":"2024-09-13T17:24:41.552663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556248+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:24:41.556248+0000","last_peered":"2024-09-13T17:24:41.556248+0000","last_clean":"2024-09-13T17:24:41.556248+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:24:41.556248+0000","last_undegraded":"2024-09-13T17:24:41.556248+0000","last_fullsized":"2024-09-13T17:24:41.556248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552916+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:24:41.552916+0000","last_peered":"2024-09-13T17:24:41.552916+0000","last_clean":"2024-09-13T17:24:41.552916+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:24:41.552916+0000","last_undegraded":"2024-09-13T17:24:41.552916+0000","last_fullsized":"2024-09-13T17:24:41.552916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552444+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:24:41.552444+0000","last_peered":"2024-09-13T17:24:41.552444+0000","last_clean":"2024-09-13T17:24:41.552444+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:24:41.552444+0000","last_undegraded":"2024-09-13T17:24:41.552444+0000","last_fullsized":"2024-09-13T17:24:41.552444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.562202+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:24:42.562202+0000","last_peered":"2024-09-13T17:24:42.562202+0000","last_clean":"2024-09-13T17:24:42.562202+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:24:42.562202+0000","last_undegraded":"2024-09-13T17:24:42.562202+0000","last_fullsized":"2024-09-13T17:24:42.562202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555590+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:24:41.555590+0000","last_peered":"2024-09-13T17:24:41.555590+0000","last_clean":"2024-09-13T17:24:41.555590+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:24:41.555590+0000","last_undegraded":"2024-09-13T17:24:41.555590+0000","last_fullsized":"2024-09-13T17:24:41.555590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.564605+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:24:42.564605+0000","last_peered":"2024-09-13T17:24:42.564605+0000","last_clean":"2024-09-13T17:24:42.564605+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:24:42.564605+0000","last_undegraded":"2024-09-13T17:24:42.564605+0000","last_fullsized":"2024-09-13T17:24:42.564605+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552935+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:24:41.552935+0000","last_peered":"2024-09-13T17:24:41.552935+0000","last_clean":"2024-09-13T17:24:41.552935+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:24:41.552935+0000","last_undegraded":"2024-09-13T17:24:41.552935+0000","last_fullsized":"2024-09-13T17:24:41.552935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561183+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:24:41.561183+0000","last_peered":"2024-09-13T17:24:41.561183+0000","last_clean":"2024-09-13T17:24:41.561183+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:24:41.561183+0000","last_undegraded":"2024-09-13T17:24:41.561183+0000","last_fullsized":"2024-09-13T17:24:41.561183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559386+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:24:41.559386+0000","last_peered":"2024-09-13T17:24:41.559386+0000","last_clean":"2024-09-13T17:24:41.559386+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:24:41.559386+0000","last_undegraded":"2024-09-13T17:24:41.559386+0000","last_fullsized":"2024-09-13T17:24:41.559386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560368+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:24:41.560368+0000","last_peered":"2024-09-13T17:24:41.560368+0000","last_clean":"2024-09-13T17:24:41.560368+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:24:41.560368+0000","last_undegraded":"2024-09-13T17:24:41.560368+0000","last_fullsized":"2024-09-13T17:24:41.560368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682352+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:24:41.682352+0000","last_peered":"2024-09-13T17:24:41.682352+0000","last_clean":"2024-09-13T17:24:41.682352+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:24:41.682352+0000","last_undegraded":"2024-09-13T17:24:41.682352+0000","last_fullsized":"2024-09-13T17:24:41.682352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.558058+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:24:41.558058+0000","last_peered":"2024-09-13T17:24:41.558058+0000","last_clean":"2024-09-13T17:24:41.558058+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:24:41.558058+0000","last_undegraded":"2024-09-13T17:24:41.558058+0000","last_fullsized":"2024-09-13T17:24:41.558058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556586+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:24:41.556586+0000","last_peered":"2024-09-13T17:24:41.556586+0000","last_clean":"2024-09-13T17:24:41.556586+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:24:41.556586+0000","last_undegraded":"2024-09-13T17:24:41.556586+0000","last_fullsized":"2024-09-13T17:24:41.556586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553193+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:24:41.553193+0000","last_peered":"2024-09-13T17:24:41.553193+0000","last_clean":"2024-09-13T17:24:41.553193+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:24:41.553193+0000","last_undegraded":"2024-09-13T17:24:41.553193+0000","last_fullsized":"2024-09-13T17:24:41.553193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553038+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:24:41.553038+0000","last_peered":"2024-09-13T17:24:41.553038+0000","last_clean":"2024-09-13T17:24:41.553038+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:24:41.553038+0000","last_undegraded":"2024-09-13T17:24:41.553038+0000","last_fullsized":"2024-09-13T17:24:41.553038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.558152+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:24:41.558152+0000","last_peered":"2024-09-13T17:24:41.558152+0000","last_clean":"2024-09-13T17:24:41.558152+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:24:41.558152+0000","last_undegraded":"2024-09-13T17:24:41.558152+0000","last_fullsized":"2024-09-13T17:24:41.558152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682055+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:24:41.682055+0000","last_peered":"2024-09-13T17:24:41.682055+0000","last_clean":"2024-09-13T17:24:41.682055+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:24:41.682055+0000","last_undegraded":"2024-09-13T17:24:41.682055+0000","last_fullsized":"2024-09-13T17:24:41.682055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562032+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:24:41.562032+0000","last_peered":"2024-09-13T17:24:41.562032+0000","last_clean":"2024-09-13T17:24:41.562032+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:24:41.562032+0000","last_undegraded":"2024-09-13T17:24:41.562032+0000","last_fullsized":"2024-09-13T17:24:41.562032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561452+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:24:41.561452+0000","last_peered":"2024-09-13T17:24:41.561452+0000","last_clean":"2024-09-13T17:24:41.561452+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:24:41.561452+0000","last_undegraded":"2024-09-13T17:24:41.561452+0000","last_fullsized":"2024-09-13T17:24:41.561452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557870+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:24:41.557870+0000","last_peered":"2024-09-13T17:24:41.557870+0000","last_clean":"2024-09-13T17:24:41.557870+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:24:41.557870+0000","last_undegraded":"2024-09-13T17:24:41.557870+0000","last_fullsized":"2024-09-13T17:24:41.557870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556273+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:24:41.556273+0000","last_peered":"2024-09-13T17:24:41.556273+0000","last_clean":"2024-09-13T17:24:41.556273+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:24:41.556273+0000","last_undegraded":"2024-09-13T17:24:41.556273+0000","last_fullsized":"2024-09-13T17:24:41.556273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552838+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:24:41.552838+0000","last_peered":"2024-09-13T17:24:41.552838+0000","last_clean":"2024-09-13T17:24:41.552838+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:24:41.552838+0000","last_undegraded":"2024-09-13T17:24:41.552838+0000","last_fullsized":"2024-09-13T17:24:41.552838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.565853+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:24:42.565853+0000","last_peered":"2024-09-13T17:24:42.565853+0000","last_clean":"2024-09-13T17:24:42.565853+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:24:42.565853+0000","last_undegraded":"2024-09-13T17:24:42.565853+0000","last_fullsized":"2024-09-13T17:24:42.565853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682582+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:24:41.682582+0000","last_peered":"2024-09-13T17:24:41.682582+0000","last_clean":"2024-09-13T17:24:41.682582+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:24:41.682582+0000","last_undegraded":"2024-09-13T17:24:41.682582+0000","last_fullsized":"2024-09-13T17:24:41.682582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559634+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:24:41.559634+0000","last_peered":"2024-09-13T17:24:41.559634+0000","last_clean":"2024-09-13T17:24:41.559634+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:24:41.559634+0000","last_undegraded":"2024-09-13T17:24:41.559634+0000","last_fullsized":"2024-09-13T17:24:41.559634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552492+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:24:41.552492+0000","last_peered":"2024-09-13T17:24:41.552492+0000","last_clean":"2024-09-13T17:24:41.552492+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:24:41.552492+0000","last_undegraded":"2024-09-13T17:24:41.552492+0000","last_fullsized":"2024-09-13T17:24:41.552492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557109+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:24:41.557109+0000","last_peered":"2024-09-13T17:24:41.557109+0000","last_clean":"2024-09-13T17:24:41.557109+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:24:41.557109+0000","last_undegraded":"2024-09-13T17:24:41.557109+0000","last_fullsized":"2024-09-13T17:24:41.557109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556565+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:24:41.556565+0000","last_peered":"2024-09-13T17:24:41.556565+0000","last_clean":"2024-09-13T17:24:41.556565+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:24:41.556565+0000","last_undegraded":"2024-09-13T17:24:41.556565+0000","last_fullsized":"2024-09-13T17:24:41.556565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.565009+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:24:42.565009+0000","last_peered":"2024-09-13T17:24:42.565009+0000","last_clean":"2024-09-13T17:24:42.565009+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:24:42.565009+0000","last_undegraded":"2024-09-13T17:24:42.565009+0000","last_fullsized":"2024-09-13T17:24:42.565009+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562287+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:24:41.562287+0000","last_peered":"2024-09-13T17:24:41.562287+0000","last_clean":"2024-09-13T17:24:41.562287+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:24:41.562287+0000","last_undegraded":"2024-09-13T17:24:41.562287+0000","last_fullsized":"2024-09-13T17:24:41.562287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562420+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:24:41.562420+0000","last_peered":"2024-09-13T17:24:41.562420+0000","last_clean":"2024-09-13T17:24:41.562420+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:24:41.562420+0000","last_undegraded":"2024-09-13T17:24:41.562420+0000","last_fullsized":"2024-09-13T17:24:41.562420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557971+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:24:41.557971+0000","last_peered":"2024-09-13T17:24:41.557971+0000","last_clean":"2024-09-13T17:24:41.557971+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:24:41.557971+0000","last_undegraded":"2024-09-13T17:24:41.557971+0000","last_fullsized":"2024-09-13T17:24:41.557971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5481,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":94,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":59461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285499,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":496612,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57799999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":9,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76500000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.68000000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":1048576,"data_stored":1085938,"data_compressed":53893,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.308},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.33500000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.438},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.59599999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67300000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":492494,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.99399999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.87},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.95499999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.73799999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":496393,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.93600000000000005,"15min":1.0069999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68500000000000005,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71899999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.54800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65400000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68799999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58399999999999996,"5min":0.746,"15min":0.746},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.501,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.61199999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41399999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.73299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.627}]},{"osd":7,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.74299999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.433,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64700000000000002}]},{"osd":9,"last update":"Fri Sep 13 17:23:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.67800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1032192,"data_stored":1082862,"data_compressed":53893,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.374,"15min":0.36299999999999999},"min":{"1min":0.28100000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.27000000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63}]},{"osd":3,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.442}]},{"osd":5,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55700000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66100000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492002,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64900000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.372},{"interface":"front","average":{"1min":0.371,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.51600000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.749},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.72999999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59499999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67200000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.628},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76400000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":1.018,"15min":1.018},"last":0.78700000000000003}]},{"osd":8,"last update":"Fri Sep 13 17:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 17:23:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.76300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1082462,"data_compressed":53893,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.47499999999999998,"15min":0.46800000000000003},"min":{"1min":0.371,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.35699999999999998,"15min":0.34599999999999997},"min":{"1min":0.26500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.317},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.434,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.627}]},{"osd":6,"last update":"Fri Sep 13 17:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.53000000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.55000000000000004,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.52800000000000002,"15min":0.51200000000000001},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.442}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69299999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":991232,"data_stored":492508,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.33300000000000002},"min":{"1min":0.21199999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.34100000000000003},"min":{"1min":0.249,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.315}]},{"osd":2,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.48299999999999998},"min":{"1min":0.40899999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51200000000000001,"15min":0.49099999999999999},"min":{"1min":0.35499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81799999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41499999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.501,"15min":0.495},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.879,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Sep 13 17:23:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.496},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Sep 13 17:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":492196,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.377,"15min":0.35199999999999998},"min":{"1min":0.25700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.39200000000000002,"15min":0.36199999999999999},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.378}]},{"osd":1,"last update":"Fri Sep 13 17:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86399999999999999,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77600000000000002,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:23:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.83199999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.622}]},{"osd":5,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.32400000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.501}]},{"osd":7,"last update":"Fri Sep 13 17:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.53900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:24:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.069,"15min":1.069},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65700000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":999424,"data_stored":495984,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68100000000000005,"5min":0.752,"15min":0.752},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.35299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:24:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.496,"15min":0.47799999999999998},"min":{"1min":0.34000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.46500000000000002},"min":{"1min":0.39500000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.72899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70399999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:23:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45200000000000001,"15min":0.439},"min":{"1min":0.33900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74299999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:24:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.69099999999999995}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1011712,"data_stored":515007,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48599999999999999,"15min":0.47799999999999998},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52800000000000002,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Sep 13 17:23:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.628,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.375},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.26700000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.34599999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.52500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53100000000000003}]},{"osd":8,"last update":"Fri Sep 13 17:24:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63900000000000001}]},{"osd":9,"last update":"Fri Sep 13 17:24:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.753},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.60799999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":492002,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.435},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.49099999999999999,"15min":0.438},"min":{"1min":0.42099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:24:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.425,"5min":0.69699999999999995,"15min":0.55400000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":15.816000000000001,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:23:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.35899999999999999},"min":{"1min":0.29199999999999998,"5min":0.214,"15min":0.214},"max":{"1min":0.70499999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39800000000000002,"15min":0.372},"min":{"1min":0.28799999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:23:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.379}]},{"osd":5,"last update":"Fri Sep 13 17:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.472,"5min":0.755,"15min":0.622},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":15.723000000000001,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.29899999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.48699999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54400000000000004}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55900000000000005}]},{"osd":11,"last update":"Fri Sep 13 17:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":45625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:24:49.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:24:49.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 -- 172.21.15.44:0/4120937690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23681008b0 msgr2=0x7f2368102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4120937690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23681008b0 0x7f2368102ca0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f235c009900 tx=0x7f235c02a410 comp rx=0 tx=0).stop 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 -- 172.21.15.44:0/4120937690 shutdown_connections 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4120937690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2368105b10 0x7f2368107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4120937690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681031e0 0x7f23681055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4120937690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23681008b0 0x7f2368102ca0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.980+0000 7f236fcf7640 1 -- 172.21.15.44:0/4120937690 >> 172.21.15.44:0/4120937690 conn(0x7f23680fa430 msgr2=0x7f23680fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 -- 172.21.15.44:0/4120937690 shutdown_connections 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 -- 172.21.15.44:0/4120937690 wait complete. 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 Processor -- start 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 -- start start 2024-09-13T17:24:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681008b0 0x7f2368100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23681031e0 0x7f23680fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 0x7f23680feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680ff500 con 0x7f2368105b10 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680ff670 con 0x7f23681008b0 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236fcf7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819fe00 con 0x7f23681031e0 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236e294640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 0x7f23680feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.981+0000 7f236e294640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 0x7f23680feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34098/0 (socket says 172.21.15.44:34098) 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 -- 172.21.15.44:0/4194601177 learned_addr learned my addr 172.21.15.44:0/4194601177 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236da93640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681008b0 0x7f2368100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236d292640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23681031e0 0x7f23680fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 -- 172.21.15.44:0/4194601177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23681031e0 msgr2=0x7f23680fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23681031e0 0x7f23680fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 -- 172.21.15.44:0/4194601177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681008b0 msgr2=0x7f2368100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681008b0 0x7f2368100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 -- 172.21.15.44:0/4194601177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009560 con 0x7f2368105b10 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236da93640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681008b0 0x7f2368100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236d292640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23681031e0 0x7f23680fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236e294640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 0x7f23680feda0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f236000cae0 tx=0x7f236000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000ae30 con 0x7f2368105b10 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2360005020 con 0x7f2368105b10 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23681a00b0 con 0x7f2368105b10 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000f6b0 con 0x7f2368105b10 2024-09-13T17:24:49.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.982+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681a0610 con 0x7f2368105b10 2024-09-13T17:24:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.983+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23680fbf20 con 0x7f2368105b10 2024-09-13T17:24:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.985+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2360004a40 con 0x7f2368105b10 2024-09-13T17:24:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.985+0000 7f235affd640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f23540a11a0 0x7f23540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.985+0000 7f236da93640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f23540a11a0 0x7f23540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.985+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f23600cd070 con 0x7f2368105b10 2024-09-13T17:24:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.986+0000 7f236da93640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f23540a11a0 0x7f23540a3660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f235c0057c0 tx=0x7f235c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:49.989+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2360096390 con 0x7f2368105b10 2024-09-13T17:24:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.158+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f236802c480 con 0x7f23540a11a0 2024-09-13T17:24:50.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.172+0000 7f235affd640 1 -- 172.21.15.44:0/4194601177 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423385 (secure 0 0 0) 0x7f236802c480 con 0x7f23540a11a0 2024-09-13T17:24:50.177 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:50.180 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:24:50.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f23540a11a0 msgr2=0x7f23540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:50.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f23540a11a0 0x7f23540a3660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f235c0057c0 tx=0x7f235c005a20 comp rx=0 tx=0).stop 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 msgr2=0x7f23680feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 0x7f23680feda0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f236000cae0 tx=0x7f236000a250 comp rx=0 tx=0).stop 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 shutdown_connections 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f23540a11a0 0x7f23540a3660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23681031e0 0x7f23680fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2368105b10 0x7f23680feda0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 --2- 172.21.15.44:0/4194601177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23681008b0 0x7f2368100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 >> 172.21.15.44:0/4194601177 conn(0x7f23680fa430 msgr2=0x7f2368101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 shutdown_connections 2024-09-13T17:24:50.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:50.176+0000 7f236fcf7640 1 -- 172.21.15.44:0/4194601177 wait complete. 2024-09-13T17:24:50.188 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-13T17:24:49.558721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595849,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":275,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":424,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385152,"kb_used_data":11772,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124556672,"statfs":{"total":1151940427776,"available":1151546032128,"internally_reserved":0,"allocated":12054528,"data_stored":7716460,"data_compressed":319998,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":888,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.008621"},"pg_stats":[{"pgid":"2.1f","version":"83'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553006+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:24:41.553006+0000","last_peered":"2024-09-13T17:24:41.553006+0000","last_clean":"2024-09-13T17:24:41.553006+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:24:41.553006+0000","last_undegraded":"2024-09-13T17:24:41.553006+0000","last_fullsized":"2024-09-13T17:24:41.553006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560804+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:24:41.560804+0000","last_peered":"2024-09-13T17:24:41.560804+0000","last_clean":"2024-09-13T17:24:41.560804+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:24:41.560804+0000","last_undegraded":"2024-09-13T17:24:41.560804+0000","last_fullsized":"2024-09-13T17:24:41.560804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552435+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:24:41.552435+0000","last_peered":"2024-09-13T17:24:41.552435+0000","last_clean":"2024-09-13T17:24:41.552435+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:24:41.552435+0000","last_undegraded":"2024-09-13T17:24:41.552435+0000","last_fullsized":"2024-09-13T17:24:41.552435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.779362+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:24:42.779362+0000","last_peered":"2024-09-13T17:24:42.779362+0000","last_clean":"2024-09-13T17:24:42.779362+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:24:42.779362+0000","last_undegraded":"2024-09-13T17:24:42.779362+0000","last_fullsized":"2024-09-13T17:24:42.779362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562952+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:24:41.562952+0000","last_peered":"2024-09-13T17:24:41.562952+0000","last_clean":"2024-09-13T17:24:41.562952+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:24:41.562952+0000","last_undegraded":"2024-09-13T17:24:41.562952+0000","last_fullsized":"2024-09-13T17:24:41.562952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.554744+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:24:41.554744+0000","last_peered":"2024-09-13T17:24:41.554744+0000","last_clean":"2024-09-13T17:24:41.554744+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:24:41.554744+0000","last_undegraded":"2024-09-13T17:24:41.554744+0000","last_fullsized":"2024-09-13T17:24:41.554744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.152617+0000","last_change":"2024-09-13T17:24:42.568012+0000","last_active":"2024-09-13T17:24:46.152617+0000","last_peered":"2024-09-13T17:24:46.152617+0000","last_clean":"2024-09-13T17:24:46.152617+0000","last_became_active":"2024-09-13T17:24:42.563885+0000","last_became_peered":"2024-09-13T17:24:42.563885+0000","last_unstale":"2024-09-13T17:24:46.152617+0000","last_undegraded":"2024-09-13T17:24:46.152617+0000","last_fullsized":"2024-09-13T17:24:46.152617+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:17.333114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682396+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:24:41.682396+0000","last_peered":"2024-09-13T17:24:41.682396+0000","last_clean":"2024-09-13T17:24:41.682396+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:24:41.682396+0000","last_undegraded":"2024-09-13T17:24:41.682396+0000","last_fullsized":"2024-09-13T17:24:41.682396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555308+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:24:41.555308+0000","last_peered":"2024-09-13T17:24:41.555308+0000","last_clean":"2024-09-13T17:24:41.555308+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:24:41.555308+0000","last_undegraded":"2024-09-13T17:24:41.555308+0000","last_fullsized":"2024-09-13T17:24:41.555308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557127+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:24:41.557127+0000","last_peered":"2024-09-13T17:24:41.557127+0000","last_clean":"2024-09-13T17:24:41.557127+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:24:41.557127+0000","last_undegraded":"2024-09-13T17:24:41.557127+0000","last_fullsized":"2024-09-13T17:24:41.557127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560045+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:24:41.560045+0000","last_peered":"2024-09-13T17:24:41.560045+0000","last_clean":"2024-09-13T17:24:41.560045+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:24:41.560045+0000","last_undegraded":"2024-09-13T17:24:41.560045+0000","last_fullsized":"2024-09-13T17:24:41.560045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.554898+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:24:41.554898+0000","last_peered":"2024-09-13T17:24:41.554898+0000","last_clean":"2024-09-13T17:24:41.554898+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:24:41.554898+0000","last_undegraded":"2024-09-13T17:24:41.554898+0000","last_fullsized":"2024-09-13T17:24:41.554898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552705+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:24:41.552705+0000","last_peered":"2024-09-13T17:24:41.552705+0000","last_clean":"2024-09-13T17:24:41.552705+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:24:41.552705+0000","last_undegraded":"2024-09-13T17:24:41.552705+0000","last_fullsized":"2024-09-13T17:24:41.552705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.597825+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:24:45.597825+0000","last_peered":"2024-09-13T17:24:45.597825+0000","last_clean":"2024-09-13T17:24:45.597825+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:24:45.597825+0000","last_undegraded":"2024-09-13T17:24:45.597825+0000","last_fullsized":"2024-09-13T17:24:45.597825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555004+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:24:41.555004+0000","last_peered":"2024-09-13T17:24:41.555004+0000","last_clean":"2024-09-13T17:24:41.555004+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:24:41.555004+0000","last_undegraded":"2024-09-13T17:24:41.555004+0000","last_fullsized":"2024-09-13T17:24:41.555004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559987+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:24:41.559987+0000","last_peered":"2024-09-13T17:24:41.559987+0000","last_clean":"2024-09-13T17:24:41.559987+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:24:41.559987+0000","last_undegraded":"2024-09-13T17:24:41.559987+0000","last_fullsized":"2024-09-13T17:24:41.559987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552714+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:24:41.552714+0000","last_peered":"2024-09-13T17:24:41.552714+0000","last_clean":"2024-09-13T17:24:41.552714+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:24:41.552714+0000","last_undegraded":"2024-09-13T17:24:41.552714+0000","last_fullsized":"2024-09-13T17:24:41.552714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.564232+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:24:42.564232+0000","last_peered":"2024-09-13T17:24:42.564232+0000","last_clean":"2024-09-13T17:24:42.564232+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:24:42.564232+0000","last_undegraded":"2024-09-13T17:24:42.564232+0000","last_fullsized":"2024-09-13T17:24:42.564232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556338+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:24:41.556338+0000","last_peered":"2024-09-13T17:24:41.556338+0000","last_clean":"2024-09-13T17:24:41.556338+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:24:41.556338+0000","last_undegraded":"2024-09-13T17:24:41.556338+0000","last_fullsized":"2024-09-13T17:24:41.556338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.597159+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:24:45.597159+0000","last_peered":"2024-09-13T17:24:45.597159+0000","last_clean":"2024-09-13T17:24:45.597159+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:24:45.597159+0000","last_undegraded":"2024-09-13T17:24:45.597159+0000","last_fullsized":"2024-09-13T17:24:45.597159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552695+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:24:41.552695+0000","last_peered":"2024-09-13T17:24:41.552695+0000","last_clean":"2024-09-13T17:24:41.552695+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:24:41.552695+0000","last_undegraded":"2024-09-13T17:24:41.552695+0000","last_fullsized":"2024-09-13T17:24:41.552695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556372+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:24:41.556372+0000","last_peered":"2024-09-13T17:24:41.556372+0000","last_clean":"2024-09-13T17:24:41.556372+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:24:41.556372+0000","last_undegraded":"2024-09-13T17:24:41.556372+0000","last_fullsized":"2024-09-13T17:24:41.556372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.596903+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:24:45.596903+0000","last_peered":"2024-09-13T17:24:45.596903+0000","last_clean":"2024-09-13T17:24:45.596903+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:24:45.596903+0000","last_undegraded":"2024-09-13T17:24:45.596903+0000","last_fullsized":"2024-09-13T17:24:45.596903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.562373+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:24:42.562373+0000","last_peered":"2024-09-13T17:24:42.562373+0000","last_clean":"2024-09-13T17:24:42.562373+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:24:42.562373+0000","last_undegraded":"2024-09-13T17:24:42.562373+0000","last_fullsized":"2024-09-13T17:24:42.562373+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562512+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:24:41.562512+0000","last_peered":"2024-09-13T17:24:41.562512+0000","last_clean":"2024-09-13T17:24:41.562512+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:24:41.562512+0000","last_undegraded":"2024-09-13T17:24:41.562512+0000","last_fullsized":"2024-09-13T17:24:41.562512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.593921+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:24:45.593921+0000","last_peered":"2024-09-13T17:24:45.593921+0000","last_clean":"2024-09-13T17:24:45.593921+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:24:45.593921+0000","last_undegraded":"2024-09-13T17:24:45.593921+0000","last_fullsized":"2024-09-13T17:24:45.593921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552392+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:24:41.552392+0000","last_peered":"2024-09-13T17:24:41.552392+0000","last_clean":"2024-09-13T17:24:41.552392+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:24:41.552392+0000","last_undegraded":"2024-09-13T17:24:41.552392+0000","last_fullsized":"2024-09-13T17:24:41.552392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561223+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:24:41.561223+0000","last_peered":"2024-09-13T17:24:41.561223+0000","last_clean":"2024-09-13T17:24:41.561223+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:24:41.561223+0000","last_undegraded":"2024-09-13T17:24:41.561223+0000","last_fullsized":"2024-09-13T17:24:41.561223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553081+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:24:41.553081+0000","last_peered":"2024-09-13T17:24:41.553081+0000","last_clean":"2024-09-13T17:24:41.553081+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:24:41.553081+0000","last_undegraded":"2024-09-13T17:24:41.553081+0000","last_fullsized":"2024-09-13T17:24:41.553081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559476+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:24:41.559476+0000","last_peered":"2024-09-13T17:24:41.559476+0000","last_clean":"2024-09-13T17:24:41.559476+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:24:41.559476+0000","last_undegraded":"2024-09-13T17:24:41.559476+0000","last_fullsized":"2024-09-13T17:24:41.559476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555689+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:24:41.555689+0000","last_peered":"2024-09-13T17:24:41.555689+0000","last_clean":"2024-09-13T17:24:41.555689+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:24:41.555689+0000","last_undegraded":"2024-09-13T17:24:41.555689+0000","last_fullsized":"2024-09-13T17:24:41.555689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.594502+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:24:45.594502+0000","last_peered":"2024-09-13T17:24:45.594502+0000","last_clean":"2024-09-13T17:24:45.594502+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:24:45.594502+0000","last_undegraded":"2024-09-13T17:24:45.594502+0000","last_fullsized":"2024-09-13T17:24:45.594502+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557540+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:24:41.557540+0000","last_peered":"2024-09-13T17:24:41.557540+0000","last_clean":"2024-09-13T17:24:41.557540+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:24:41.557540+0000","last_undegraded":"2024-09-13T17:24:41.557540+0000","last_fullsized":"2024-09-13T17:24:41.557540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557355+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:24:41.557355+0000","last_peered":"2024-09-13T17:24:41.557355+0000","last_clean":"2024-09-13T17:24:41.557355+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:24:41.557355+0000","last_undegraded":"2024-09-13T17:24:41.557355+0000","last_fullsized":"2024-09-13T17:24:41.557355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552367+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:24:41.552367+0000","last_peered":"2024-09-13T17:24:41.552367+0000","last_clean":"2024-09-13T17:24:41.552367+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:24:41.552367+0000","last_undegraded":"2024-09-13T17:24:41.552367+0000","last_fullsized":"2024-09-13T17:24:41.552367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556502+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:24:41.556502+0000","last_peered":"2024-09-13T17:24:41.556502+0000","last_clean":"2024-09-13T17:24:41.556502+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:24:41.556502+0000","last_undegraded":"2024-09-13T17:24:41.556502+0000","last_fullsized":"2024-09-13T17:24:41.556502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556724+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:24:41.556724+0000","last_peered":"2024-09-13T17:24:41.556724+0000","last_clean":"2024-09-13T17:24:41.556724+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:24:41.556724+0000","last_undegraded":"2024-09-13T17:24:41.556724+0000","last_fullsized":"2024-09-13T17:24:41.556724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.563321+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:24:42.563321+0000","last_peered":"2024-09-13T17:24:42.563321+0000","last_clean":"2024-09-13T17:24:42.563321+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:24:42.563321+0000","last_undegraded":"2024-09-13T17:24:42.563321+0000","last_fullsized":"2024-09-13T17:24:42.563321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555440+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:24:41.555440+0000","last_peered":"2024-09-13T17:24:41.555440+0000","last_clean":"2024-09-13T17:24:41.555440+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:24:41.555440+0000","last_undegraded":"2024-09-13T17:24:41.555440+0000","last_fullsized":"2024-09-13T17:24:41.555440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559595+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:24:41.559595+0000","last_peered":"2024-09-13T17:24:41.559595+0000","last_clean":"2024-09-13T17:24:41.559595+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:24:41.559595+0000","last_undegraded":"2024-09-13T17:24:41.559595+0000","last_fullsized":"2024-09-13T17:24:41.559595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.153863+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:24:46.153863+0000","last_peered":"2024-09-13T17:24:46.153863+0000","last_clean":"2024-09-13T17:24:46.153863+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:24:46.153863+0000","last_undegraded":"2024-09-13T17:24:46.153863+0000","last_fullsized":"2024-09-13T17:24:46.153863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.558700+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:24:41.558700+0000","last_peered":"2024-09-13T17:24:41.558700+0000","last_clean":"2024-09-13T17:24:41.558700+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:24:41.558700+0000","last_undegraded":"2024-09-13T17:24:41.558700+0000","last_fullsized":"2024-09-13T17:24:41.558700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555011+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:24:41.555011+0000","last_peered":"2024-09-13T17:24:41.555011+0000","last_clean":"2024-09-13T17:24:41.555011+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:24:41.555011+0000","last_undegraded":"2024-09-13T17:24:41.555011+0000","last_fullsized":"2024-09-13T17:24:41.555011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.152458+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:24:46.152458+0000","last_peered":"2024-09-13T17:24:46.152458+0000","last_clean":"2024-09-13T17:24:46.152458+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:24:46.152458+0000","last_undegraded":"2024-09-13T17:24:46.152458+0000","last_fullsized":"2024-09-13T17:24:46.152458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559792+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:24:41.559792+0000","last_peered":"2024-09-13T17:24:41.559792+0000","last_clean":"2024-09-13T17:24:41.559792+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:24:41.559792+0000","last_undegraded":"2024-09-13T17:24:41.559792+0000","last_fullsized":"2024-09-13T17:24:41.559792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560096+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:24:41.560096+0000","last_peered":"2024-09-13T17:24:41.560096+0000","last_clean":"2024-09-13T17:24:41.560096+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:24:41.560096+0000","last_undegraded":"2024-09-13T17:24:41.560096+0000","last_fullsized":"2024-09-13T17:24:41.560096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.143457+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:24:46.143457+0000","last_peered":"2024-09-13T17:24:46.143457+0000","last_clean":"2024-09-13T17:24:46.143457+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:24:46.143457+0000","last_undegraded":"2024-09-13T17:24:46.143457+0000","last_fullsized":"2024-09-13T17:24:46.143457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552655+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:24:41.552655+0000","last_peered":"2024-09-13T17:24:41.552655+0000","last_clean":"2024-09-13T17:24:41.552655+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:24:41.552655+0000","last_undegraded":"2024-09-13T17:24:41.552655+0000","last_fullsized":"2024-09-13T17:24:41.552655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552736+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:24:41.552736+0000","last_peered":"2024-09-13T17:24:41.552736+0000","last_clean":"2024-09-13T17:24:41.552736+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:24:41.552736+0000","last_undegraded":"2024-09-13T17:24:41.552736+0000","last_fullsized":"2024-09-13T17:24:41.552736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552963+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:24:41.552963+0000","last_peered":"2024-09-13T17:24:41.552963+0000","last_clean":"2024-09-13T17:24:41.552963+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:24:41.552963+0000","last_undegraded":"2024-09-13T17:24:41.552963+0000","last_fullsized":"2024-09-13T17:24:41.552963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553171+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:24:41.553171+0000","last_peered":"2024-09-13T17:24:41.553171+0000","last_clean":"2024-09-13T17:24:41.553171+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:24:41.553171+0000","last_undegraded":"2024-09-13T17:24:41.553171+0000","last_fullsized":"2024-09-13T17:24:41.553171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682144+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:24:41.682144+0000","last_peered":"2024-09-13T17:24:41.682144+0000","last_clean":"2024-09-13T17:24:41.682144+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:24:41.682144+0000","last_undegraded":"2024-09-13T17:24:41.682144+0000","last_fullsized":"2024-09-13T17:24:41.682144+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553045+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:24:41.553045+0000","last_peered":"2024-09-13T17:24:41.553045+0000","last_clean":"2024-09-13T17:24:41.553045+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:24:41.553045+0000","last_undegraded":"2024-09-13T17:24:41.553045+0000","last_fullsized":"2024-09-13T17:24:41.553045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560111+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:24:41.560111+0000","last_peered":"2024-09-13T17:24:41.560111+0000","last_clean":"2024-09-13T17:24:41.560111+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:24:41.560111+0000","last_undegraded":"2024-09-13T17:24:41.560111+0000","last_fullsized":"2024-09-13T17:24:41.560111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561446+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:24:41.561446+0000","last_peered":"2024-09-13T17:24:41.561446+0000","last_clean":"2024-09-13T17:24:41.561446+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:24:41.561446+0000","last_undegraded":"2024-09-13T17:24:41.561446+0000","last_fullsized":"2024-09-13T17:24:41.561446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553254+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:24:41.553254+0000","last_peered":"2024-09-13T17:24:41.553254+0000","last_clean":"2024-09-13T17:24:41.553254+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:24:41.553254+0000","last_undegraded":"2024-09-13T17:24:41.553254+0000","last_fullsized":"2024-09-13T17:24:41.553254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559041+0000","last_change":"2024-09-13T17:24:21.384590+0000","last_active":"2024-09-13T17:24:41.559041+0000","last_peered":"2024-09-13T17:24:41.559041+0000","last_clean":"2024-09-13T17:24:41.559041+0000","last_became_active":"2024-09-13T17:24:21.383582+0000","last_became_peered":"2024-09-13T17:24:21.383582+0000","last_unstale":"2024-09-13T17:24:41.559041+0000","last_undegraded":"2024-09-13T17:24:41.559041+0000","last_fullsized":"2024-09-13T17:24:41.559041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:01:46.302548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559133+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:24:41.559133+0000","last_peered":"2024-09-13T17:24:41.559133+0000","last_clean":"2024-09-13T17:24:41.559133+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:24:41.559133+0000","last_undegraded":"2024-09-13T17:24:41.559133+0000","last_fullsized":"2024-09-13T17:24:41.559133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557587+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:24:41.557587+0000","last_peered":"2024-09-13T17:24:41.557587+0000","last_clean":"2024-09-13T17:24:41.557587+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:24:41.557587+0000","last_undegraded":"2024-09-13T17:24:41.557587+0000","last_fullsized":"2024-09-13T17:24:41.557587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.560416+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:24:42.560416+0000","last_peered":"2024-09-13T17:24:42.560416+0000","last_clean":"2024-09-13T17:24:42.560416+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:24:42.560416+0000","last_undegraded":"2024-09-13T17:24:42.560416+0000","last_fullsized":"2024-09-13T17:24:42.560416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559644+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:24:41.559644+0000","last_peered":"2024-09-13T17:24:41.559644+0000","last_clean":"2024-09-13T17:24:41.559644+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:24:41.559644+0000","last_undegraded":"2024-09-13T17:24:41.559644+0000","last_fullsized":"2024-09-13T17:24:41.559644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559556+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:24:41.559556+0000","last_peered":"2024-09-13T17:24:41.559556+0000","last_clean":"2024-09-13T17:24:41.559556+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:24:41.559556+0000","last_undegraded":"2024-09-13T17:24:41.559556+0000","last_fullsized":"2024-09-13T17:24:41.559556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.154312+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:24:46.154312+0000","last_peered":"2024-09-13T17:24:46.154312+0000","last_clean":"2024-09-13T17:24:46.154312+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:24:46.154312+0000","last_undegraded":"2024-09-13T17:24:46.154312+0000","last_fullsized":"2024-09-13T17:24:46.154312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.780454+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:24:42.780454+0000","last_peered":"2024-09-13T17:24:42.780454+0000","last_clean":"2024-09-13T17:24:42.780454+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:24:42.780454+0000","last_undegraded":"2024-09-13T17:24:42.780454+0000","last_fullsized":"2024-09-13T17:24:42.780454+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552420+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:24:41.552420+0000","last_peered":"2024-09-13T17:24:41.552420+0000","last_clean":"2024-09-13T17:24:41.552420+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:24:41.552420+0000","last_undegraded":"2024-09-13T17:24:41.552420+0000","last_fullsized":"2024-09-13T17:24:41.552420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.153360+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:24:46.153360+0000","last_peered":"2024-09-13T17:24:46.153360+0000","last_clean":"2024-09-13T17:24:46.153360+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:24:46.153360+0000","last_undegraded":"2024-09-13T17:24:46.153360+0000","last_fullsized":"2024-09-13T17:24:46.153360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.781366+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:24:42.781366+0000","last_peered":"2024-09-13T17:24:42.781366+0000","last_clean":"2024-09-13T17:24:42.781366+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:24:42.781366+0000","last_undegraded":"2024-09-13T17:24:42.781366+0000","last_fullsized":"2024-09-13T17:24:42.781366+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552908+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:24:41.552908+0000","last_peered":"2024-09-13T17:24:41.552908+0000","last_clean":"2024-09-13T17:24:41.552908+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:24:41.552908+0000","last_undegraded":"2024-09-13T17:24:41.552908+0000","last_fullsized":"2024-09-13T17:24:41.552908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559490+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:24:41.559490+0000","last_peered":"2024-09-13T17:24:41.559490+0000","last_clean":"2024-09-13T17:24:41.559490+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:24:41.559490+0000","last_undegraded":"2024-09-13T17:24:41.559490+0000","last_fullsized":"2024-09-13T17:24:41.559490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556439+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:24:41.556439+0000","last_peered":"2024-09-13T17:24:41.556439+0000","last_clean":"2024-09-13T17:24:41.556439+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:24:41.556439+0000","last_undegraded":"2024-09-13T17:24:41.556439+0000","last_fullsized":"2024-09-13T17:24:41.556439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555546+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:24:41.555546+0000","last_peered":"2024-09-13T17:24:41.555546+0000","last_clean":"2024-09-13T17:24:41.555546+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:24:41.555546+0000","last_undegraded":"2024-09-13T17:24:41.555546+0000","last_fullsized":"2024-09-13T17:24:41.555546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.595473+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:24:45.595473+0000","last_peered":"2024-09-13T17:24:45.595473+0000","last_clean":"2024-09-13T17:24:45.595473+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:24:45.595473+0000","last_undegraded":"2024-09-13T17:24:45.595473+0000","last_fullsized":"2024-09-13T17:24:45.595473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559797+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:24:41.559797+0000","last_peered":"2024-09-13T17:24:41.559797+0000","last_clean":"2024-09-13T17:24:41.559797+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:24:41.559797+0000","last_undegraded":"2024-09-13T17:24:41.559797+0000","last_fullsized":"2024-09-13T17:24:41.559797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556842+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:24:41.556842+0000","last_peered":"2024-09-13T17:24:41.556842+0000","last_clean":"2024-09-13T17:24:41.556842+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:24:41.556842+0000","last_undegraded":"2024-09-13T17:24:41.556842+0000","last_fullsized":"2024-09-13T17:24:41.556842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557035+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:24:41.557035+0000","last_peered":"2024-09-13T17:24:41.557035+0000","last_clean":"2024-09-13T17:24:41.557035+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:24:41.557035+0000","last_undegraded":"2024-09-13T17:24:41.557035+0000","last_fullsized":"2024-09-13T17:24:41.557035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552092+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:24:41.552092+0000","last_peered":"2024-09-13T17:24:41.552092+0000","last_clean":"2024-09-13T17:24:41.552092+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:24:41.552092+0000","last_undegraded":"2024-09-13T17:24:41.552092+0000","last_fullsized":"2024-09-13T17:24:41.552092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682480+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:24:41.682480+0000","last_peered":"2024-09-13T17:24:41.682480+0000","last_clean":"2024-09-13T17:24:41.682480+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:24:41.682480+0000","last_undegraded":"2024-09-13T17:24:41.682480+0000","last_fullsized":"2024-09-13T17:24:41.682480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560013+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:24:41.560013+0000","last_peered":"2024-09-13T17:24:41.560013+0000","last_clean":"2024-09-13T17:24:41.560013+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:24:41.560013+0000","last_undegraded":"2024-09-13T17:24:41.560013+0000","last_fullsized":"2024-09-13T17:24:41.560013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555127+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:24:41.555127+0000","last_peered":"2024-09-13T17:24:41.555127+0000","last_clean":"2024-09-13T17:24:41.555127+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:24:41.555127+0000","last_undegraded":"2024-09-13T17:24:41.555127+0000","last_fullsized":"2024-09-13T17:24:41.555127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.563023+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:24:41.563023+0000","last_peered":"2024-09-13T17:24:41.563023+0000","last_clean":"2024-09-13T17:24:41.563023+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:24:41.563023+0000","last_undegraded":"2024-09-13T17:24:41.563023+0000","last_fullsized":"2024-09-13T17:24:41.563023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:45.596999+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:24:45.596999+0000","last_peered":"2024-09-13T17:24:45.596999+0000","last_clean":"2024-09-13T17:24:45.596999+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:24:45.596999+0000","last_undegraded":"2024-09-13T17:24:45.596999+0000","last_fullsized":"2024-09-13T17:24:45.596999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.551888+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:24:41.551888+0000","last_peered":"2024-09-13T17:24:41.551888+0000","last_clean":"2024-09-13T17:24:41.551888+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:24:41.551888+0000","last_undegraded":"2024-09-13T17:24:41.551888+0000","last_fullsized":"2024-09-13T17:24:41.551888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682557+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:24:41.682557+0000","last_peered":"2024-09-13T17:24:41.682557+0000","last_clean":"2024-09-13T17:24:41.682557+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:24:41.682557+0000","last_undegraded":"2024-09-13T17:24:41.682557+0000","last_fullsized":"2024-09-13T17:24:41.682557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557523+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:24:41.557523+0000","last_peered":"2024-09-13T17:24:41.557523+0000","last_clean":"2024-09-13T17:24:41.557523+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:24:41.557523+0000","last_undegraded":"2024-09-13T17:24:41.557523+0000","last_fullsized":"2024-09-13T17:24:41.557523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559758+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:24:41.559758+0000","last_peered":"2024-09-13T17:24:41.559758+0000","last_clean":"2024-09-13T17:24:41.559758+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:24:41.559758+0000","last_undegraded":"2024-09-13T17:24:41.559758+0000","last_fullsized":"2024-09-13T17:24:41.559758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562405+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:24:41.562405+0000","last_peered":"2024-09-13T17:24:41.562405+0000","last_clean":"2024-09-13T17:24:41.562405+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:24:41.562405+0000","last_undegraded":"2024-09-13T17:24:41.562405+0000","last_fullsized":"2024-09-13T17:24:41.562405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.551991+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:24:41.551991+0000","last_peered":"2024-09-13T17:24:41.551991+0000","last_clean":"2024-09-13T17:24:41.551991+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:24:41.551991+0000","last_undegraded":"2024-09-13T17:24:41.551991+0000","last_fullsized":"2024-09-13T17:24:41.551991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559619+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:24:41.559619+0000","last_peered":"2024-09-13T17:24:41.559619+0000","last_clean":"2024-09-13T17:24:41.559619+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:24:41.559619+0000","last_undegraded":"2024-09-13T17:24:41.559619+0000","last_fullsized":"2024-09-13T17:24:41.559619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557770+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:24:41.557770+0000","last_peered":"2024-09-13T17:24:41.557770+0000","last_clean":"2024-09-13T17:24:41.557770+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:24:41.557770+0000","last_undegraded":"2024-09-13T17:24:41.557770+0000","last_fullsized":"2024-09-13T17:24:41.557770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559655+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:24:41.559655+0000","last_peered":"2024-09-13T17:24:41.559655+0000","last_clean":"2024-09-13T17:24:41.559655+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:24:41.559655+0000","last_undegraded":"2024-09-13T17:24:41.559655+0000","last_fullsized":"2024-09-13T17:24:41.559655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:46.152790+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:24:46.152790+0000","last_peered":"2024-09-13T17:24:46.152790+0000","last_clean":"2024-09-13T17:24:46.152790+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:24:46.152790+0000","last_undegraded":"2024-09-13T17:24:46.152790+0000","last_fullsized":"2024-09-13T17:24:46.152790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557213+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:24:41.557213+0000","last_peered":"2024-09-13T17:24:41.557213+0000","last_clean":"2024-09-13T17:24:41.557213+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:24:41.557213+0000","last_undegraded":"2024-09-13T17:24:41.557213+0000","last_fullsized":"2024-09-13T17:24:41.557213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559395+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:24:41.559395+0000","last_peered":"2024-09-13T17:24:41.559395+0000","last_clean":"2024-09-13T17:24:41.559395+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:24:41.559395+0000","last_undegraded":"2024-09-13T17:24:41.559395+0000","last_fullsized":"2024-09-13T17:24:41.559395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552625+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:24:41.552625+0000","last_peered":"2024-09-13T17:24:41.552625+0000","last_clean":"2024-09-13T17:24:41.552625+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:24:41.552625+0000","last_undegraded":"2024-09-13T17:24:41.552625+0000","last_fullsized":"2024-09-13T17:24:41.552625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559471+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:24:41.559471+0000","last_peered":"2024-09-13T17:24:41.559471+0000","last_clean":"2024-09-13T17:24:41.559471+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:24:41.559471+0000","last_undegraded":"2024-09-13T17:24:41.559471+0000","last_fullsized":"2024-09-13T17:24:41.559471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556788+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:24:41.556788+0000","last_peered":"2024-09-13T17:24:41.556788+0000","last_clean":"2024-09-13T17:24:41.556788+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:24:41.556788+0000","last_undegraded":"2024-09-13T17:24:41.556788+0000","last_fullsized":"2024-09-13T17:24:41.556788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552663+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:24:41.552663+0000","last_peered":"2024-09-13T17:24:41.552663+0000","last_clean":"2024-09-13T17:24:41.552663+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:24:41.552663+0000","last_undegraded":"2024-09-13T17:24:41.552663+0000","last_fullsized":"2024-09-13T17:24:41.552663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556248+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:24:41.556248+0000","last_peered":"2024-09-13T17:24:41.556248+0000","last_clean":"2024-09-13T17:24:41.556248+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:24:41.556248+0000","last_undegraded":"2024-09-13T17:24:41.556248+0000","last_fullsized":"2024-09-13T17:24:41.556248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552916+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:24:41.552916+0000","last_peered":"2024-09-13T17:24:41.552916+0000","last_clean":"2024-09-13T17:24:41.552916+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:24:41.552916+0000","last_undegraded":"2024-09-13T17:24:41.552916+0000","last_fullsized":"2024-09-13T17:24:41.552916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552444+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:24:41.552444+0000","last_peered":"2024-09-13T17:24:41.552444+0000","last_clean":"2024-09-13T17:24:41.552444+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:24:41.552444+0000","last_undegraded":"2024-09-13T17:24:41.552444+0000","last_fullsized":"2024-09-13T17:24:41.552444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.562202+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:24:42.562202+0000","last_peered":"2024-09-13T17:24:42.562202+0000","last_clean":"2024-09-13T17:24:42.562202+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:24:42.562202+0000","last_undegraded":"2024-09-13T17:24:42.562202+0000","last_fullsized":"2024-09-13T17:24:42.562202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.555590+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:24:41.555590+0000","last_peered":"2024-09-13T17:24:41.555590+0000","last_clean":"2024-09-13T17:24:41.555590+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:24:41.555590+0000","last_undegraded":"2024-09-13T17:24:41.555590+0000","last_fullsized":"2024-09-13T17:24:41.555590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.564605+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:24:42.564605+0000","last_peered":"2024-09-13T17:24:42.564605+0000","last_clean":"2024-09-13T17:24:42.564605+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:24:42.564605+0000","last_undegraded":"2024-09-13T17:24:42.564605+0000","last_fullsized":"2024-09-13T17:24:42.564605+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552935+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:24:41.552935+0000","last_peered":"2024-09-13T17:24:41.552935+0000","last_clean":"2024-09-13T17:24:41.552935+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:24:41.552935+0000","last_undegraded":"2024-09-13T17:24:41.552935+0000","last_fullsized":"2024-09-13T17:24:41.552935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561183+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:24:41.561183+0000","last_peered":"2024-09-13T17:24:41.561183+0000","last_clean":"2024-09-13T17:24:41.561183+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:24:41.561183+0000","last_undegraded":"2024-09-13T17:24:41.561183+0000","last_fullsized":"2024-09-13T17:24:41.561183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559386+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:24:41.559386+0000","last_peered":"2024-09-13T17:24:41.559386+0000","last_clean":"2024-09-13T17:24:41.559386+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:24:41.559386+0000","last_undegraded":"2024-09-13T17:24:41.559386+0000","last_fullsized":"2024-09-13T17:24:41.559386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.560368+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:24:41.560368+0000","last_peered":"2024-09-13T17:24:41.560368+0000","last_clean":"2024-09-13T17:24:41.560368+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:24:41.560368+0000","last_undegraded":"2024-09-13T17:24:41.560368+0000","last_fullsized":"2024-09-13T17:24:41.560368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682352+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:24:41.682352+0000","last_peered":"2024-09-13T17:24:41.682352+0000","last_clean":"2024-09-13T17:24:41.682352+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:24:41.682352+0000","last_undegraded":"2024-09-13T17:24:41.682352+0000","last_fullsized":"2024-09-13T17:24:41.682352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.558058+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:24:41.558058+0000","last_peered":"2024-09-13T17:24:41.558058+0000","last_clean":"2024-09-13T17:24:41.558058+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:24:41.558058+0000","last_undegraded":"2024-09-13T17:24:41.558058+0000","last_fullsized":"2024-09-13T17:24:41.558058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556586+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:24:41.556586+0000","last_peered":"2024-09-13T17:24:41.556586+0000","last_clean":"2024-09-13T17:24:41.556586+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:24:41.556586+0000","last_undegraded":"2024-09-13T17:24:41.556586+0000","last_fullsized":"2024-09-13T17:24:41.556586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553193+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:24:41.553193+0000","last_peered":"2024-09-13T17:24:41.553193+0000","last_clean":"2024-09-13T17:24:41.553193+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:24:41.553193+0000","last_undegraded":"2024-09-13T17:24:41.553193+0000","last_fullsized":"2024-09-13T17:24:41.553193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.553038+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:24:41.553038+0000","last_peered":"2024-09-13T17:24:41.553038+0000","last_clean":"2024-09-13T17:24:41.553038+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:24:41.553038+0000","last_undegraded":"2024-09-13T17:24:41.553038+0000","last_fullsized":"2024-09-13T17:24:41.553038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.558152+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:24:41.558152+0000","last_peered":"2024-09-13T17:24:41.558152+0000","last_clean":"2024-09-13T17:24:41.558152+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:24:41.558152+0000","last_undegraded":"2024-09-13T17:24:41.558152+0000","last_fullsized":"2024-09-13T17:24:41.558152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682055+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:24:41.682055+0000","last_peered":"2024-09-13T17:24:41.682055+0000","last_clean":"2024-09-13T17:24:41.682055+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:24:41.682055+0000","last_undegraded":"2024-09-13T17:24:41.682055+0000","last_fullsized":"2024-09-13T17:24:41.682055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562032+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:24:41.562032+0000","last_peered":"2024-09-13T17:24:41.562032+0000","last_clean":"2024-09-13T17:24:41.562032+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:24:41.562032+0000","last_undegraded":"2024-09-13T17:24:41.562032+0000","last_fullsized":"2024-09-13T17:24:41.562032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.561452+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:24:41.561452+0000","last_peered":"2024-09-13T17:24:41.561452+0000","last_clean":"2024-09-13T17:24:41.561452+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:24:41.561452+0000","last_undegraded":"2024-09-13T17:24:41.561452+0000","last_fullsized":"2024-09-13T17:24:41.561452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557870+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:24:41.557870+0000","last_peered":"2024-09-13T17:24:41.557870+0000","last_clean":"2024-09-13T17:24:41.557870+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:24:41.557870+0000","last_undegraded":"2024-09-13T17:24:41.557870+0000","last_fullsized":"2024-09-13T17:24:41.557870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556273+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:24:41.556273+0000","last_peered":"2024-09-13T17:24:41.556273+0000","last_clean":"2024-09-13T17:24:41.556273+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:24:41.556273+0000","last_undegraded":"2024-09-13T17:24:41.556273+0000","last_fullsized":"2024-09-13T17:24:41.556273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552838+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:24:41.552838+0000","last_peered":"2024-09-13T17:24:41.552838+0000","last_clean":"2024-09-13T17:24:41.552838+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:24:41.552838+0000","last_undegraded":"2024-09-13T17:24:41.552838+0000","last_fullsized":"2024-09-13T17:24:41.552838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.565853+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:24:42.565853+0000","last_peered":"2024-09-13T17:24:42.565853+0000","last_clean":"2024-09-13T17:24:42.565853+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:24:42.565853+0000","last_undegraded":"2024-09-13T17:24:42.565853+0000","last_fullsized":"2024-09-13T17:24:42.565853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.682582+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:24:41.682582+0000","last_peered":"2024-09-13T17:24:41.682582+0000","last_clean":"2024-09-13T17:24:41.682582+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:24:41.682582+0000","last_undegraded":"2024-09-13T17:24:41.682582+0000","last_fullsized":"2024-09-13T17:24:41.682582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.559634+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:24:41.559634+0000","last_peered":"2024-09-13T17:24:41.559634+0000","last_clean":"2024-09-13T17:24:41.559634+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:24:41.559634+0000","last_undegraded":"2024-09-13T17:24:41.559634+0000","last_fullsized":"2024-09-13T17:24:41.559634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.552492+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:24:41.552492+0000","last_peered":"2024-09-13T17:24:41.552492+0000","last_clean":"2024-09-13T17:24:41.552492+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:24:41.552492+0000","last_undegraded":"2024-09-13T17:24:41.552492+0000","last_fullsized":"2024-09-13T17:24:41.552492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557109+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:24:41.557109+0000","last_peered":"2024-09-13T17:24:41.557109+0000","last_clean":"2024-09-13T17:24:41.557109+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:24:41.557109+0000","last_undegraded":"2024-09-13T17:24:41.557109+0000","last_fullsized":"2024-09-13T17:24:41.557109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.556565+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:24:41.556565+0000","last_peered":"2024-09-13T17:24:41.556565+0000","last_clean":"2024-09-13T17:24:41.556565+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:24:41.556565+0000","last_undegraded":"2024-09-13T17:24:41.556565+0000","last_fullsized":"2024-09-13T17:24:41.556565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T17:24:42.565009+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:24:42.565009+0000","last_peered":"2024-09-13T17:24:42.565009+0000","last_clean":"2024-09-13T17:24:42.565009+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:24:42.565009+0000","last_undegraded":"2024-09-13T17:24:42.565009+0000","last_fullsized":"2024-09-13T17:24:42.565009+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562287+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:24:41.562287+0000","last_peered":"2024-09-13T17:24:41.562287+0000","last_clean":"2024-09-13T17:24:41.562287+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:24:41.562287+0000","last_undegraded":"2024-09-13T17:24:41.562287+0000","last_fullsized":"2024-09-13T17:24:41.562287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.562420+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:24:41.562420+0000","last_peered":"2024-09-13T17:24:41.562420+0000","last_clean":"2024-09-13T17:24:41.562420+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:24:41.562420+0000","last_undegraded":"2024-09-13T17:24:41.562420+0000","last_fullsized":"2024-09-13T17:24:41.562420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T17:24:41.557971+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:24:41.557971+0000","last_peered":"2024-09-13T17:24:41.557971+0000","last_clean":"2024-09-13T17:24:41.557971+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:24:41.557971+0000","last_undegraded":"2024-09-13T17:24:41.557971+0000","last_fullsized":"2024-09-13T17:24:41.557971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5481,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":94,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":59461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285499,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":496612,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57799999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":9,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76500000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.68000000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":1048576,"data_stored":1085938,"data_compressed":53893,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.308},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.33500000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.438},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.59599999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67300000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":492494,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.99399999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.87},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.95499999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.73799999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":496393,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.93600000000000005,"15min":1.0069999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68500000000000005,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71899999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.54800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65400000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68799999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58399999999999996,"5min":0.746,"15min":0.746},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.501,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.61199999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41399999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.73299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.627}]},{"osd":7,"last update":"Fri Sep 13 17:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.74299999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.433,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64700000000000002}]},{"osd":9,"last update":"Fri Sep 13 17:23:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.67800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1032192,"data_stored":1082862,"data_compressed":53893,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.374,"15min":0.36299999999999999},"min":{"1min":0.28100000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.27000000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63}]},{"osd":3,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.442}]},{"osd":5,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55700000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66100000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492002,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64900000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.372},{"interface":"front","average":{"1min":0.371,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.51600000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.749},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.72999999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59499999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.498,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67200000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.628},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76400000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":1.018,"15min":1.018},"last":0.78700000000000003}]},{"osd":8,"last update":"Fri Sep 13 17:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 17:23:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.76300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1082462,"data_compressed":53893,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.47499999999999998,"15min":0.46800000000000003},"min":{"1min":0.371,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.35699999999999998,"15min":0.34599999999999997},"min":{"1min":0.26500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.317},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.434,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.627}]},{"osd":6,"last update":"Fri Sep 13 17:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.53000000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.55000000000000004,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.52800000000000002,"15min":0.51200000000000001},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.442}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69299999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":991232,"data_stored":492508,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.33300000000000002},"min":{"1min":0.21199999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.34100000000000003},"min":{"1min":0.249,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.315}]},{"osd":2,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.48299999999999998},"min":{"1min":0.40899999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51200000000000001,"15min":0.49099999999999999},"min":{"1min":0.35499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81799999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41499999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.501,"15min":0.495},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.879,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Sep 13 17:23:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.496},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Sep 13 17:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":492196,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.377,"15min":0.35199999999999998},"min":{"1min":0.25700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.39200000000000002,"15min":0.36199999999999999},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.378}]},{"osd":1,"last update":"Fri Sep 13 17:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86399999999999999,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77600000000000002,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:23:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.83199999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.622}]},{"osd":5,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.32400000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.501}]},{"osd":7,"last update":"Fri Sep 13 17:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.53900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:24:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.069,"15min":1.069},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65700000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":999424,"data_stored":495984,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68100000000000005,"5min":0.752,"15min":0.752},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.35299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:24:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.496,"15min":0.47799999999999998},"min":{"1min":0.34000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.46500000000000002},"min":{"1min":0.39500000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.72899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70399999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:23:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45200000000000001,"15min":0.439},"min":{"1min":0.33900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74299999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:24:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.69099999999999995}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1011712,"data_stored":515007,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48599999999999999,"15min":0.47799999999999998},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52800000000000002,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Sep 13 17:23:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.628,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.375},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.26700000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.34599999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.52500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53100000000000003}]},{"osd":8,"last update":"Fri Sep 13 17:24:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63900000000000001}]},{"osd":9,"last update":"Fri Sep 13 17:24:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.753},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.60799999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":492002,"data_compressed":17591,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.435},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.49099999999999999,"15min":0.438},"min":{"1min":0.42099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:24:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.425,"5min":0.69699999999999995,"15min":0.55400000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":15.816000000000001,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:23:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.35899999999999999},"min":{"1min":0.29199999999999998,"5min":0.214,"15min":0.214},"max":{"1min":0.70499999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39800000000000002,"15min":0.372},"min":{"1min":0.28799999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:23:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.379}]},{"osd":5,"last update":"Fri Sep 13 17:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.472,"5min":0.755,"15min":0.622},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":15.723000000000001,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.29899999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.48699999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54400000000000004}]},{"osd":10,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55900000000000005}]},{"osd":11,"last update":"Fri Sep 13 17:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":45625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:24:50.195 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:24:50.195 INFO:tasks.cephadm:Setting up client nodes... 2024-09-13T17:24:50.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T17:24:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:24:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:50 smithi070 ceph-mon[34502]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1774112824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4114881839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:50 smithi070 ceph-mon[34502]: pgmap v294: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 2.4 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:50 smithi100 ceph-mon[35055]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1774112824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4114881839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:50 smithi100 ceph-mon[35055]: pgmap v294: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 2.4 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:50.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:50 smithi044 ceph-mon[31175]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:24:50.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1774112824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:24:50.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4114881839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:50.553 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:50 smithi044 ceph-mon[31175]: pgmap v294: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 2.4 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:51 smithi070 ceph-mon[34502]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:24:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:51 smithi070 ceph-mon[34502]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:24:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:51 smithi100 ceph-mon[35055]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:24:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:51 smithi100 ceph-mon[35055]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:24:51.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:51 smithi044 ceph-mon[31175]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:24:51.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:51 smithi044 ceph-mon[31175]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 -- 172.21.15.44:0/1361782056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c105b50 msgr2=0x7f693c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 --2- 172.21.15.44:0/1361782056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c105b50 0x7f693c107f40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6934009900 tx=0x7f693402a410 comp rx=0 tx=0).stop 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 -- 172.21.15.44:0/1361782056 shutdown_connections 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 --2- 172.21.15.44:0/1361782056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f693c103220 0x7f693c105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 --2- 172.21.15.44:0/1361782056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c105b50 0x7f693c107f40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 --2- 172.21.15.44:0/1361782056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c1008f0 0x7f693c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 -- 172.21.15.44:0/1361782056 >> 172.21.15.44:0/1361782056 conn(0x7f693c0fa490 msgr2=0x7f693c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:52.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 -- 172.21.15.44:0/1361782056 shutdown_connections 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.192+0000 7f69412a5640 1 -- 172.21.15.44:0/1361782056 wait complete. 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 Processor -- start 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 -- start start 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f693c1008f0 0x7f693c195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 0x7f693c1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c105b50 0x7f693c19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c19d770 con 0x7f693c103220 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c19d8e0 con 0x7f693c105b50 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f69412a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c19daa0 con 0x7f693c1008f0 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 0x7f693c1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 0x7f693c1961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34118/0 (socket says 172.21.15.44:34118) 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 -- 172.21.15.44:0/3590773343 learned_addr learned my addr 172.21.15.44:0/3590773343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693affd640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f693c1008f0 0x7f693c195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693b7fe640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c105b50 0x7f693c19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 -- 172.21.15.44:0/3590773343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f693c1008f0 msgr2=0x7f693c195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f693c1008f0 0x7f693c195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 -- 172.21.15.44:0/3590773343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c105b50 msgr2=0x7f693c19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c105b50 0x7f693c19d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.193+0000 7f693a7fc640 1 -- 172.21.15.44:0/3590773343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6934009560 con 0x7f693c103220 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.194+0000 7f693a7fc640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 0x7f693c1961c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f693000ebc0 tx=0x7f693000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.194+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930004d20 con 0x7f693c103220 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.194+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c19dd80 con 0x7f693c103220 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.194+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f693000eec0 con 0x7f693c103220 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.194+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930010690 con 0x7f693c103220 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.194+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c19e220 con 0x7f693c103220 2024-09-13T17:24:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.195+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693c0fbf80 con 0x7f693c103220 2024-09-13T17:24:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.196+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6930005020 con 0x7f693c103220 2024-09-13T17:24:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.197+0000 7f6923fff640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f69240a1190 0x7f69240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.197+0000 7f693affd640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f69240a1190 0x7f69240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.197+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6930014070 con 0x7f693c103220 2024-09-13T17:24:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.197+0000 7f693affd640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f69240a1190 0x7f69240a3650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f692c005fd0 tx=0x7f692c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.201+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6930096980 con 0x7f693c103220 2024-09-13T17:24:52.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.439+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f693c02c480 con 0x7f693c103220 2024-09-13T17:24:52.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.446+0000 7f6923fff640 1 -- 172.21.15.44:0/3590773343 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6930099fa0 con 0x7f693c103220 2024-09-13T17:24:52.447 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T17:24:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBkdeRmu9NIGhAAkSZ5Rrg6+kWuIpGEjCe+xg== 2024-09-13T17:24:52.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f69240a1190 msgr2=0x7f69240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f69240a1190 0x7f69240a3650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f692c005fd0 tx=0x7f692c005850 comp rx=0 tx=0).stop 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 msgr2=0x7f693c1961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 0x7f693c1961c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f693000ebc0 tx=0x7f693000c6a0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 shutdown_connections 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f69240a1190 0x7f69240a3650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c105b50 0x7f693c19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f693c103220 0x7f693c1961c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 --2- 172.21.15.44:0/3590773343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f693c1008f0 0x7f693c195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.448+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 >> 172.21.15.44:0/3590773343 conn(0x7f693c0fa490 msgr2=0x7f693c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.449+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 shutdown_connections 2024-09-13T17:24:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:52.449+0000 7f69412a5640 1 -- 172.21.15.44:0/3590773343 wait complete. 2024-09-13T17:24:53.139 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:24:53.140 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-13T17:24:53.140 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T17:24:53.182 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T17:24:53.182 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-13T17:24:53.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-13T17:24:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:24:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:53 smithi070 ceph-mon[34502]: pgmap v295: 129 pgs: 129 active+clean; 625 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:53 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3590773343' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T17:24:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:53 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3590773343' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T17:24:53.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:53 smithi100 ceph-mon[35055]: pgmap v295: 129 pgs: 129 active+clean; 625 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:53.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:53 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3590773343' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T17:24:53.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:53 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3590773343' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T17:24:53.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:53 smithi044 ceph-mon[31175]: pgmap v295: 129 pgs: 129 active+clean; 625 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:53.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:53 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3590773343' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T17:24:53.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:53 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3590773343' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T17:24:55.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 -- 172.21.15.44:0/2948070824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb41050a0 msgr2=0x7f4cb410cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/2948070824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb41050a0 0x7f4cb410cc20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009cd0 tx=0x7f4cac02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 -- 172.21.15.44:0/2948070824 shutdown_connections 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/2948070824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb41050a0 0x7f4cb410cc20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/2948070824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb4100920 0x7f4cb4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/2948070824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb40fff70 0x7f4cb4100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 -- 172.21.15.44:0/2948070824 >> 172.21.15.44:0/2948070824 conn(0x7f4cb40fba30 msgr2=0x7f4cb40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 -- 172.21.15.44:0/2948070824 shutdown_connections 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.122+0000 7f4cb9eef640 1 -- 172.21.15.44:0/2948070824 wait complete. 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 Processor -- start 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 -- start start 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 0x7f4cb40727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:55.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb4100920 0x7f4cb4072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb41050a0 0x7f4cb4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb41a4640 con 0x7f4cb40fff70 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb41a47b0 con 0x7f4cb41050a0 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb9eef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb41a4970 con 0x7f4cb4100920 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 0x7f4cb40727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.123+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 0x7f4cb40727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34144/0 (socket says 172.21.15.44:34144) 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 -- 172.21.15.44:0/3195922248 learned_addr learned my addr 172.21.15.44:0/3195922248 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb2ffd640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb4100920 0x7f4cb4072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb3fff640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb41050a0 0x7f4cb4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 -- 172.21.15.44:0/3195922248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb4100920 msgr2=0x7f4cb4072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb4100920 0x7f4cb4072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 -- 172.21.15.44:0/3195922248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb41050a0 msgr2=0x7f4cb4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb41050a0 0x7f4cb4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 -- 172.21.15.44:0/3195922248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac009900 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb37fe640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 0x7f4cb40727b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400d770 tx=0x7f4ca400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4005220 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb41a4c50 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ca4009cb0 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4004c30 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.124+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb41a5150 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.125+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb4101450 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.126+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4ca4005380 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.127+0000 7f4cb0ff9640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4c9c0a1110 0x7f4c9c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.127+0000 7f4cb2ffd640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4c9c0a1110 0x7f4c9c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.127+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ca40cc810 con 0x7f4cb40fff70 2024-09-13T17:24:55.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.128+0000 7f4cb2ffd640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4c9c0a1110 0x7f4c9c0a35d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40742b0 tx=0x7f4ca8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:55.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.132+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4ca4095b30 con 0x7f4cb40fff70 2024-09-13T17:24:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.351+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4cb4108280 con 0x7f4cb40fff70 2024-09-13T17:24:55.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.355+0000 7f4cb0ff9640 1 -- 172.21.15.44:0/3195922248 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260283 (secure 0 0 0) 0x7f4ca4017090 con 0x7f4cb40fff70 2024-09-13T17:24:55.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4c9c0a1110 msgr2=0x7f4c9c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4c9c0a1110 0x7f4c9c0a35d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40742b0 tx=0x7f4ca8009110 comp rx=0 tx=0).stop 2024-09-13T17:24:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 msgr2=0x7f4cb40727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 0x7f4cb40727b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400d770 tx=0x7f4ca400db70 comp rx=0 tx=0).stop 2024-09-13T17:24:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 shutdown_connections 2024-09-13T17:24:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4c9c0a1110 0x7f4c9c0a35d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb4100920 0x7f4cb4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb41050a0 0x7f4cb4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 --2- 172.21.15.44:0/3195922248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb40fff70 0x7f4cb40727b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:55.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 >> 172.21.15.44:0/3195922248 conn(0x7f4cb40fba30 msgr2=0x7f4cb40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:55.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 shutdown_connections 2024-09-13T17:24:55.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:55.360+0000 7f4cb9eef640 1 -- 172.21.15.44:0/3195922248 wait complete. 2024-09-13T17:24:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:55 smithi070 ceph-mon[34502]: pgmap v296: 129 pgs: 129 active+clean; 625 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.8 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:55.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:55 smithi100 ceph-mon[35055]: pgmap v296: 129 pgs: 129 active+clean; 625 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.8 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:55.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:55 smithi044 ceph-mon[31175]: pgmap v296: 129 pgs: 129 active+clean; 625 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.8 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:56.024 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":322597156},{"type":"v1","addr":"172.21.15.44:6801","nonce":322597156}]},"active_addr":"172.21.15.44:6801/322597156","active_change":"2024-09-13T17:17:41.472900+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24118,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.44:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":3278871357}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2025177182}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":1409924258}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":4032970337}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":3372003479}]}]} 2024-09-13T17:24:56.030 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-13T17:24:56.030 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-13T17:24:56.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T17:24:56.242 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:24:56.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:56 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3195922248' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T17:24:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:56 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3195922248' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T17:24:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:56 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3195922248' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T17:24:57.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:57 smithi070 ceph-mon[34502]: pgmap v297: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:57.481 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:57 smithi044 ceph-mon[31175]: pgmap v297: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:57.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:57 smithi100 ceph-mon[35055]: pgmap v297: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:24:58.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 -- 172.21.15.44:0/3443139460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c100f30 msgr2=0x7f858c101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:58.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 --2- 172.21.15.44:0/3443139460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c100f30 0x7f858c101310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8584009ca0 tx=0x7f858402a950 comp rx=0 tx=0).stop 2024-09-13T17:24:58.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 -- 172.21.15.44:0/3443139460 shutdown_connections 2024-09-13T17:24:58.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 --2- 172.21.15.44:0/3443139460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c101850 0x7f858c109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 --2- 172.21.15.44:0/3443139460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c10a360 0x7f858c10c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 --2- 172.21.15.44:0/3443139460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c100f30 0x7f858c101310 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 -- 172.21.15.44:0/3443139460 >> 172.21.15.44:0/3443139460 conn(0x7f858c0fa9b0 msgr2=0x7f858c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 -- 172.21.15.44:0/3443139460 shutdown_connections 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.114+0000 7f859089e640 1 -- 172.21.15.44:0/3443139460 wait complete. 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 Processor -- start 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 -- start start 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c100f30 0x7f858c1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 0x7f858c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c10a360 0x7f858c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c19b000 con 0x7f858c101850 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c19b170 con 0x7f858c100f30 2024-09-13T17:24:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f859089e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c19b2e0 con 0x7f858c10a360 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f8589d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 0x7f858c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f8589d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 0x7f858c196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34176/0 (socket says 172.21.15.44:34176) 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.115+0000 7f8589d9b640 1 -- 172.21.15.44:0/1506788111 learned_addr learned my addr 172.21.15.44:0/1506788111 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f858ad9d640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c10a360 0x7f858c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f858a59c640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c100f30 0x7f858c1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f8589d9b640 1 -- 172.21.15.44:0/1506788111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c10a360 msgr2=0x7f858c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f8589d9b640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c10a360 0x7f858c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f8589d9b640 1 -- 172.21.15.44:0/1506788111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c100f30 msgr2=0x7f858c1960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f8589d9b640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c100f30 0x7f858c1960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f8589d9b640 1 -- 172.21.15.44:0/1506788111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584009900 con 0x7f858c101850 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f858a59c640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c100f30 0x7f858c1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f858ad9d640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c10a360 0x7f858c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f8589d9b640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 0x7f858c196600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f857400c970 tx=0x7f857400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574007bb0 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c19b5c0 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8574007d10 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574018660 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.116+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c1a2e80 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.117+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c0687e0 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.119+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f857400f450 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.119+0000 7f85837fe640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85780a1110 0x7f85780a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.120+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f85740ccec0 con 0x7f858c101850 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.120+0000 7f858a59c640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85780a1110 0x7f85780a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:24:58.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.120+0000 7f858a59c640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85780a1110 0x7f85780a35d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8584006010 tx=0x7f8584005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:24:58.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.124+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f85740961e0 con 0x7f858c101850 2024-09-13T17:24:58.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.299+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f858c101310 con 0x7f858c101850 2024-09-13T17:24:58.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.300+0000 7f85837fe640 1 -- 172.21.15.44:0/1506788111 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24427 (secure 0 0 0) 0x7f8574099800 con 0x7f858c101850 2024-09-13T17:24:58.303 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:24:58.303 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:42.549640+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85780a1110 msgr2=0x7f85780a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85780a1110 0x7f85780a35d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8584006010 tx=0x7f8584005d80 comp rx=0 tx=0).stop 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 msgr2=0x7f858c196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 0x7f858c196600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f857400c970 tx=0x7f857400ce30 comp rx=0 tx=0).stop 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 shutdown_connections 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f85780a1110 0x7f85780a35d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f858c10a360 0x7f858c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f858c101850 0x7f858c196600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 --2- 172.21.15.44:0/1506788111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858c100f30 0x7f858c1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.302+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 >> 172.21.15.44:0/1506788111 conn(0x7f858c0fa9b0 msgr2=0x7f858c0fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:24:58.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.303+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 shutdown_connections 2024-09-13T17:24:58.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:24:58.303+0000 7f859089e640 1 -- 172.21.15.44:0/1506788111 wait complete. 2024-09-13T17:24:59.120 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-13T17:24:59.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T17:24:59.331 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:24:59.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:59 smithi044 ceph-mon[31175]: pgmap v298: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-09-13T17:24:59.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:24:59 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1506788111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:59 smithi070 ceph-mon[34502]: pgmap v298: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-09-13T17:24:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:24:59 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1506788111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:24:59.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:59 smithi100 ceph-mon[35055]: pgmap v298: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-09-13T17:24:59.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:24:59 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1506788111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 -- 172.21.15.44:0/1794770880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b80ff4d0 msgr2=0x7f26b810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 --2- 172.21.15.44:0/1794770880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b80ff4d0 0x7f26b810c190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009900 tx=0x7f26a802a410 comp rx=0 tx=0).stop 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 -- 172.21.15.44:0/1794770880 shutdown_connections 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 --2- 172.21.15.44:0/1794770880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26b810c6d0 0x7f26b810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 --2- 172.21.15.44:0/1794770880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b80ff4d0 0x7f26b810c190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 --2- 172.21.15.44:0/1794770880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80feb20 0x7f26b80fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 -- 172.21.15.44:0/1794770880 >> 172.21.15.44:0/1794770880 conn(0x7f26b80fa9b0 msgr2=0x7f26b80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 -- 172.21.15.44:0/1794770880 shutdown_connections 2024-09-13T17:25:01.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.148+0000 7f26bf768640 1 -- 172.21.15.44:0/1794770880 wait complete. 2024-09-13T17:25:01.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 Processor -- start 2024-09-13T17:25:01.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 -- start start 2024-09-13T17:25:01.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26b80feb20 0x7f26b81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:01.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80ff4d0 0x7f26b8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:01.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 0x7f26b819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:01.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b819d310 con 0x7f26b810c6d0 2024-09-13T17:25:01.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b819d480 con 0x7f26b80ff4d0 2024-09-13T17:25:01.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bf768640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b819d5f0 con 0x7f26b80feb20 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bdd05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 0x7f26b819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.149+0000 7f26bdd05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 0x7f26b819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60468/0 (socket says 172.21.15.44:60468) 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 -- 172.21.15.44:0/424633756 learned_addr learned my addr 172.21.15.44:0/424633756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bcd03640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80ff4d0 0x7f26b8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bd504640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26b80feb20 0x7f26b81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 -- 172.21.15.44:0/424633756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26b80feb20 msgr2=0x7f26b81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26b80feb20 0x7f26b81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 -- 172.21.15.44:0/424633756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80ff4d0 msgr2=0x7f26b8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80ff4d0 0x7f26b8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 -- 172.21.15.44:0/424633756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8009560 con 0x7f26b810c6d0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bdd05640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 0x7f26b819cd10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b3f0 tx=0x7f26b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0011830 con 0x7f26b810c6d0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26b0011e20 con 0x7f26b810c6d0 2024-09-13T17:25:01.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b819d8d0 con 0x7f26b810c6d0 2024-09-13T17:25:01.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0010df0 con 0x7f26b810c6d0 2024-09-13T17:25:01.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.150+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b81a5190 con 0x7f26b810c6d0 2024-09-13T17:25:01.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.151+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b8068780 con 0x7f26b810c6d0 2024-09-13T17:25:01.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.153+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f26b0011990 con 0x7f26b810c6d0 2024-09-13T17:25:01.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.153+0000 7f26ae7fc640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26a40a1190 0x7f26a40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:01.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.153+0000 7f26bd504640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26a40a1190 0x7f26a40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:01.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.153+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26b00ccc10 con 0x7f26b810c6d0 2024-09-13T17:25:01.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.154+0000 7f26bd504640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26a40a1190 0x7f26a40a3650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26b4007350 tx=0x7f26b40072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:01.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.156+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f26b0096040 con 0x7f26b810c6d0 2024-09-13T17:25:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.332+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26b8199730 con 0x7f26b810c6d0 2024-09-13T17:25:01.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.333+0000 7f26ae7fc640 1 -- 172.21.15.44:0/424633756 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24427 (secure 0 0 0) 0x7f26b0099660 con 0x7f26b810c6d0 2024-09-13T17:25:01.336 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:25:01.336 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:24:42.549640+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26a40a1190 msgr2=0x7f26a40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26a40a1190 0x7f26a40a3650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26b4007350 tx=0x7f26b40072e0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 msgr2=0x7f26b819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 0x7f26b819cd10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b3f0 tx=0x7f26b000b8b0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 shutdown_connections 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26a40a1190 0x7f26a40a3650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b810c6d0 0x7f26b819cd10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80ff4d0 0x7f26b8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 --2- 172.21.15.44:0/424633756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26b80feb20 0x7f26b81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 >> 172.21.15.44:0/424633756 conn(0x7f26b80fa9b0 msgr2=0x7f26b80fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 shutdown_connections 2024-09-13T17:25:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:01.336+0000 7f26bf768640 1 -- 172.21.15.44:0/424633756 wait complete. 2024-09-13T17:25:01.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:01 smithi070 ceph-mon[34502]: pgmap v299: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-09-13T17:25:01.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:01 smithi100 ceph-mon[35055]: pgmap v299: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-09-13T17:25:01.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:01 smithi044 ceph-mon[31175]: pgmap v299: 129 pgs: 129 active+clean; 633 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-09-13T17:25:02.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-13T17:25:02.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-13T17:25:02.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-13T17:25:02.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-13T17:25:02.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-13T17:25:02.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-13T17:25:02.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-13T17:25:02.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-13T17:25:02.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-13T17:25:02.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-13T17:25:02.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-13T17:25:02.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-13T17:25:02.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/424633756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:25:02.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:02 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/424633756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:25:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:02 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/424633756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:25:02.665 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.678 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.973 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:02.987 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:03.442 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:03.444 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:03.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:03 smithi100 ceph-mon[35055]: pgmap v300: 129 pgs: 129 active+clean; 637 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-09-13T17:25:03.539 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:03 smithi044 ceph-mon[31175]: pgmap v300: 129 pgs: 129 active+clean; 637 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-09-13T17:25:03.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:03 smithi070 ceph-mon[34502]: pgmap v300: 129 pgs: 129 active+clean; 637 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-09-13T17:25:05.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:05 smithi100 ceph-mon[35055]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-13T17:25:05.550 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:05 smithi044 ceph-mon[31175]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-13T17:25:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:05 smithi070 ceph-mon[34502]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-13T17:25:07.514 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:07 smithi044 ceph-mon[31175]: pgmap v302: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-13T17:25:07.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:07 smithi100 ceph-mon[35055]: pgmap v302: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-13T17:25:07.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:07 smithi070 ceph-mon[34502]: pgmap v302: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-13T17:25:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:09.311 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:09 smithi044 ceph-mon[31175]: pgmap v303: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-13T17:25:09.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:09 smithi100 ceph-mon[35055]: pgmap v303: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-13T17:25:09.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:09 smithi070 ceph-mon[34502]: pgmap v303: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-13T17:25:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:11.506 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:11 smithi044 ceph-mon[31175]: pgmap v304: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-13T17:25:11.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:11 smithi100 ceph-mon[35055]: pgmap v304: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-13T17:25:11.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:11 smithi070 ceph-mon[34502]: pgmap v304: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-13T17:25:12.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.046+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/4017892742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c24100f60 msgr2=0x7f0c24101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/4017892742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c24100f60 0x7f0c24101340 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009900 tx=0x7f0c1402a480 comp rx=0 tx=0).stop 2024-09-13T17:25:12.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/4017892742 shutdown_connections 2024-09-13T17:25:12.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/4017892742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c2410a390 0x7f0c2410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/4017892742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c24101880 0x7f0c24109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/4017892742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c24100f60 0x7f0c24101340 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/4017892742 >> 172.21.15.44:0/4017892742 conn(0x7f0c240fa9a0 msgr2=0x7f0c240fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/4017892742 shutdown_connections 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/4017892742 wait complete. 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 Processor -- start 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.047+0000 7f0c2a2b1640 1 -- start start 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2a2b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c24100f60 0x7f0c241961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2a2b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c24101880 0x7f0c24196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2a2b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c2410a390 0x7f0c2419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2a2b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2419b130 con 0x7f0c2410a390 2024-09-13T17:25:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2a2b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2419b2a0 con 0x7f0c24101880 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2a2b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2419b410 con 0x7f0c24100f60 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c2410a390 0x7f0c2419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c2410a390 0x7f0c2419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42338/0 (socket says 172.21.15.44:42338) 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 -- 172.21.15.44:0/1759117545 learned_addr learned my addr 172.21.15.44:0/1759117545 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c23fff640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c24100f60 0x7f0c241961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c237fe640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c24101880 0x7f0c24196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 -- 172.21.15.44:0/1759117545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c24100f60 msgr2=0x7f0c241961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c24100f60 0x7f0c241961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 -- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c24101880 msgr2=0x7f0c24196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c24101880 0x7f0c24196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 -- 172.21.15.44:0/1759117545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14009560 con 0x7f0c2410a390 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c2884e640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c2410a390 0x7f0c2419ab30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00eaa0 tx=0x7f0c1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.048+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00cdb0 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.049+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2419b6f0 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.049+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c1c005020 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.049+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0116d0 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.049+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c241a2fb0 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.050+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c2410ff60 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.051+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0c1c004a40 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.051+0000 7f0c217fa640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c0c0a1110 0x7f0c0c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c23fff640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c0c0a1110 0x7f0c0c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0c1c0cd1c0 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c217fa640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] conn(0x7f0c0c0ac1a0 0x7f0c0c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 --> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c0c0aed10 con 0x7f0c0c0ac1a0 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c1c0cd5d0 con 0x7f0c2410a390 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c237fe640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] conn(0x7f0c0c0ac1a0 0x7f0c0c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c23fff640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c0c0a1110 0x7f0c0c0a35d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c1402aa70 tx=0x7f0c14005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:12.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.052+0000 7f0c237fe640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] conn(0x7f0c0c0ac1a0 0x7f0c0c0ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:12.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.053+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== osd.1 v2:172.21.15.70:6800/3261074014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0c0c0aed10 con 0x7f0c0c0ac1a0 2024-09-13T17:25:12.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.067+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 --> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c24069000 con 0x7f0c0c0ac1a0 2024-09-13T17:25:12.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.067+0000 7f0c217fa640 1 -- 172.21.15.44:0/1759117545 <== osd.1 v2:172.21.15.70:6800/3261074014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c24069000 con 0x7f0c0c0ac1a0 2024-09-13T17:25:12.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] conn(0x7f0c0c0ac1a0 msgr2=0x7f0c0c0ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] conn(0x7f0c0c0ac1a0 0x7f0c0c0ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c0c0a1110 msgr2=0x7f0c0c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c0c0a1110 0x7f0c0c0a35d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c1402aa70 tx=0x7f0c14005a20 comp rx=0 tx=0).stop 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c2410a390 msgr2=0x7f0c2419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 --2- 172.21.15.44:0/1759117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c2410a390 0x7f0c2419ab30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00eaa0 tx=0x7f0c1c00ef60 comp rx=0 tx=0).stop 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2884e640 1 -- 172.21.15.44:0/1759117545 reap_dead start 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 shutdown_connections 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 >> 172.21.15.44:0/1759117545 conn(0x7f0c240fa9a0 msgr2=0x7f0c240fb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 shutdown_connections 2024-09-13T17:25:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.068+0000 7f0c2a2b1640 1 -- 172.21.15.44:0/1759117545 wait complete. 2024-09-13T17:25:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 -- 172.21.15.44:0/787102166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec40fffe0 msgr2=0x7fcec41003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/787102166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec40fffe0 0x7fcec41003c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009930 tx=0x7fcec002a420 comp rx=0 tx=0).stop 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 -- 172.21.15.44:0/787102166 shutdown_connections 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/787102166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4105110 0x7fcec410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/787102166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec4100990 0x7fcec41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/787102166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec40fffe0 0x7fcec41003c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 -- 172.21.15.44:0/787102166 >> 172.21.15.44:0/787102166 conn(0x7fcec40fbae0 msgr2=0x7fcec40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 -- 172.21.15.44:0/787102166 shutdown_connections 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.526+0000 7fcecb7cd640 1 -- 172.21.15.44:0/787102166 wait complete. 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 Processor -- start 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 -- start start 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec40fffe0 0x7fcec41962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec4100990 0x7fcec4196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4105110 0x7fcec419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419b210 con 0x7fcec4105110 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419b380 con 0x7fcec4100990 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcecb7cd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419b4f0 con 0x7fcec40fffe0 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcec9d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4105110 0x7fcec419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcec9d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4105110 0x7fcec419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42358/0 (socket says 172.21.15.44:42358) 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcec9d6a640 1 -- 172.21.15.44:0/348674080 learned_addr learned my addr 172.21.15.44:0/348674080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcec9569640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec40fffe0 0x7fcec41962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.527+0000 7fcec8d68640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec4100990 0x7fcec4196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcec9d6a640 1 -- 172.21.15.44:0/348674080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec40fffe0 msgr2=0x7fcec41962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcec9d6a640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec40fffe0 0x7fcec41962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcec9d6a640 1 -- 172.21.15.44:0/348674080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec4100990 msgr2=0x7fcec4196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcec9d6a640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec4100990 0x7fcec4196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcec9d6a640 1 -- 172.21.15.44:0/348674080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0009560 con 0x7fcec4105110 2024-09-13T17:25:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcec9d6a640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4105110 0x7fcec419ac10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00b970 tx=0x7fcebc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00cb40 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcebc00cca0 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419b7d0 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc007610 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.528+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec41a3090 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.529+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcec41014c0 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.530+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcebc014440 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fceba7fc640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceb00a1190 0x7fceb00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fcec9569640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceb00a1190 0x7fceb00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcebc0ccc80 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fceba7fc640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] conn(0x7fceb00ac220 0x7fceb00ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 --> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fceb00aed90 con 0x7fceb00ac220 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcebc0cd090 con 0x7fcec4105110 2024-09-13T17:25:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fcec8d68640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] conn(0x7fceb00ac220 0x7fceb00ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.531+0000 7fcec9569640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceb00a1190 0x7fceb00a3650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcec0006010 tx=0x7fcec0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.532+0000 7fcec8d68640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] conn(0x7fceb00ac220 0x7fceb00ae6c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.532+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== osd.3 v2:172.21.15.44:6810/581674337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fceb00aed90 con 0x7fceb00ac220 2024-09-13T17:25:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 --> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcec410ff20 con 0x7fceb00ac220 2024-09-13T17:25:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fceba7fc640 1 -- 172.21.15.44:0/348674080 <== osd.3 v2:172.21.15.44:6810/581674337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcec410ff20 con 0x7fceb00ac220 2024-09-13T17:25:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] conn(0x7fceb00ac220 msgr2=0x7fceb00ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] conn(0x7fceb00ac220 0x7fceb00ae6c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceb00a1190 msgr2=0x7fceb00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fceb00a1190 0x7fceb00a3650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcec0006010 tx=0x7fcec0000bc0 comp rx=0 tx=0).stop 2024-09-13T17:25:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4105110 msgr2=0x7fcec419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcecb7cd640 1 --2- 172.21.15.44:0/348674080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4105110 0x7fcec419ac10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00b970 tx=0x7fcebc00be30 comp rx=0 tx=0).stop 2024-09-13T17:25:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.546+0000 7fcec9d6a640 1 -- 172.21.15.44:0/348674080 reap_dead start 2024-09-13T17:25:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.547+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 shutdown_connections 2024-09-13T17:25:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.547+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 >> 172.21.15.44:0/348674080 conn(0x7fcec40fbae0 msgr2=0x7fcec40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.547+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 shutdown_connections 2024-09-13T17:25:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:12.547+0000 7fcecb7cd640 1 -- 172.21.15.44:0/348674080 wait complete. 2024-09-13T17:25:13.521 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:13 smithi044 ceph-mon[31175]: pgmap v305: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-13T17:25:13.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:13 smithi100 ceph-mon[35055]: pgmap v305: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-13T17:25:13.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:13 smithi070 ceph-mon[34502]: pgmap v305: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-13T17:25:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.097+0000 7fe4726fa640 1 -- 172.21.15.44:0/2230568198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ff8b0 msgr2=0x7fe46c103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 --2- 172.21.15.44:0/2230568198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ff8b0 0x7fe46c103900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe460009930 tx=0x7fe46002a420 comp rx=0 tx=0).stop 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 -- 172.21.15.44:0/2230568198 shutdown_connections 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 --2- 172.21.15.44:0/2230568198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c104030 0x7fe46c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 --2- 172.21.15.44:0/2230568198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ff8b0 0x7fe46c103900 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 --2- 172.21.15.44:0/2230568198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe46c0fef00 0x7fe46c0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 -- 172.21.15.44:0/2230568198 >> 172.21.15.44:0/2230568198 conn(0x7fe46c0fa9a0 msgr2=0x7fe46c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 -- 172.21.15.44:0/2230568198 shutdown_connections 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 -- 172.21.15.44:0/2230568198 wait complete. 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 Processor -- start 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.098+0000 7fe4726fa640 1 -- start start 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe4726fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c0fef00 0x7fe46c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe4726fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe46c0ff8b0 0x7fe46c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe4726fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c104030 0x7fe46c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe4726fa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1a45c0 con 0x7fe46c104030 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe4726fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1a4730 con 0x7fe46c0ff8b0 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe4726fa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1a48f0 con 0x7fe46c0fef00 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe470c97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c104030 0x7fe46c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe470c97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c104030 0x7fe46c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42382/0 (socket says 172.21.15.44:42382) 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe470c97640 1 -- 172.21.15.44:0/798287219 learned_addr learned my addr 172.21.15.44:0/798287219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe46b7fe640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe46c0ff8b0 0x7fe46c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.099+0000 7fe46bfff640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c0fef00 0x7fe46c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe470c97640 1 -- 172.21.15.44:0/798287219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c0fef00 msgr2=0x7fe46c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe470c97640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c0fef00 0x7fe46c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe470c97640 1 -- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe46c0ff8b0 msgr2=0x7fe46c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe470c97640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe46c0ff8b0 0x7fe46c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe470c97640 1 -- 172.21.15.44:0/798287219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009560 con 0x7fe46c104030 2024-09-13T17:25:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe470c97640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c104030 0x7fe46c101420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe46400b970 tx=0x7fe46400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46400caa0 con 0x7fe46c104030 2024-09-13T17:25:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe46400cc00 con 0x7fe46c104030 2024-09-13T17:25:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c1a4bd0 con 0x7fe46c104030 2024-09-13T17:25:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4640075d0 con 0x7fe46c104030 2024-09-13T17:25:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.100+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c1a5050 con 0x7fe46c104030 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.101+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe46c1003e0 con 0x7fe46c104030 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.103+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe464014440 con 0x7fe46c104030 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.103+0000 7fe4697fa640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe4540a1110 0x7fe4540a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe46bfff640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe4540a1110 0x7fe4540a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4640cd010 con 0x7fe46c104030 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe4697fa640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] conn(0x7fe4540ac1a0 0x7fe4540ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 --> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4540aed10 con 0x7fe4540ac1a0 2024-09-13T17:25:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4640cd420 con 0x7fe46c104030 2024-09-13T17:25:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe46bfff640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe4540a1110 0x7fe4540a35d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00abe0 tx=0x7fe45c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe46b7fe640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] conn(0x7fe4540ac1a0 0x7fe4540ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.104+0000 7fe46b7fe640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] conn(0x7fe4540ac1a0 0x7fe4540ae640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.105+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== osd.7 v2:172.21.15.70:6816/4007129132 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe4540aed10 con 0x7fe4540ac1a0 2024-09-13T17:25:15.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.119+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 --> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe46c10edf0 con 0x7fe4540ac1a0 2024-09-13T17:25:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4697fa640 1 -- 172.21.15.44:0/798287219 <== osd.7 v2:172.21.15.70:6816/4007129132 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe46c10edf0 con 0x7fe4540ac1a0 2024-09-13T17:25:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] conn(0x7fe4540ac1a0 msgr2=0x7fe4540ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] conn(0x7fe4540ac1a0 0x7fe4540ae640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe4540a1110 msgr2=0x7fe4540a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe4540a1110 0x7fe4540a35d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00abe0 tx=0x7fe45c00a2d0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c104030 msgr2=0x7fe46c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 --2- 172.21.15.44:0/798287219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c104030 0x7fe46c101420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe46400b970 tx=0x7fe46400bd70 comp rx=0 tx=0).stop 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe470c97640 1 -- 172.21.15.44:0/798287219 reap_dead start 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 shutdown_connections 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.120+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 >> 172.21.15.44:0/798287219 conn(0x7fe46c0fa9a0 msgr2=0x7fe46c068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.121+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 shutdown_connections 2024-09-13T17:25:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.121+0000 7fe4726fa640 1 -- 172.21.15.44:0/798287219 wait complete. 2024-09-13T17:25:15.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:15 smithi044 ceph-mon[31175]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:25:15.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:15 smithi100 ceph-mon[35055]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:25:15.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:15 smithi070 ceph-mon[34502]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:25:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 -- 172.21.15.44:0/3016281275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78410a3a0 msgr2=0x7fc78410c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 --2- 172.21.15.44:0/3016281275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78410a3a0 0x7fc78410c820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009ca0 tx=0x7fc77c02a950 comp rx=0 tx=0).stop 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 -- 172.21.15.44:0/3016281275 shutdown_connections 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 --2- 172.21.15.44:0/3016281275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7841018e0 0x7fc784109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 --2- 172.21.15.44:0/3016281275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78410a3a0 0x7fc78410c820 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 --2- 172.21.15.44:0/3016281275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc784100fc0 0x7fc7841013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 -- 172.21.15.44:0/3016281275 >> 172.21.15.44:0/3016281275 conn(0x7fc7840fa9a0 msgr2=0x7fc7840fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 -- 172.21.15.44:0/3016281275 shutdown_connections 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.720+0000 7fc78ad42640 1 -- 172.21.15.44:0/3016281275 wait complete. 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 Processor -- start 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 -- start start 2024-09-13T17:25:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc784100fc0 0x7fc7841983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7841018e0 0x7fc784198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78410a3a0 0x7fc78419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78419d300 con 0x7fc784100fc0 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78419d470 con 0x7fc7841018e0 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc78ad42640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78419d5e0 con 0x7fc78410a3a0 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc784100fc0 0x7fc7841983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc784100fc0 0x7fc7841983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42400/0 (socket says 172.21.15.44:42400) 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 -- 172.21.15.44:0/2348463827 learned_addr learned my addr 172.21.15.44:0/2348463827 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc783fff640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7841018e0 0x7fc784198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc7892df640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78410a3a0 0x7fc78419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 -- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78410a3a0 msgr2=0x7fc78419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78410a3a0 0x7fc78419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 -- 172.21.15.44:0/2348463827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7841018e0 msgr2=0x7fc784198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7841018e0 0x7fc784198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc788ade640 1 -- 172.21.15.44:0/2348463827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c009900 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.721+0000 7fc7892df640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78410a3a0 0x7fc78419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.722+0000 7fc788ade640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc784100fc0 0x7fc7841983c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e9f0 tx=0x7fc76c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.722+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c009700 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.722+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc76c00cd80 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.722+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78419d8c0 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.722+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c0187e0 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.722+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841a5180 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.723+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc78410ff10 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.724+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc76c010430 con 0x7fc784100fc0 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc781ffb640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc7700a1190 0x7fc7700a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc783fff640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc7700a1190 0x7fc7700a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc76c014070 con 0x7fc784100fc0 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc781ffb640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] conn(0x7fc7700ac220 0x7fc7700ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 --> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7700aed90 con 0x7fc7700ac220 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc76c0cdbc0 con 0x7fc784100fc0 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.725+0000 7fc783fff640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc7700a1190 0x7fc7700a3650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc774005fd0 tx=0x7fc774005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.726+0000 7fc7892df640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] conn(0x7fc7700ac220 0x7fc7700ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.726+0000 7fc7892df640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] conn(0x7fc7700ac220 0x7fc7700ae6c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.727+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== osd.11 v2:172.21.15.100:6824/1155650564 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc7700aed90 con 0x7fc7700ac220 2024-09-13T17:25:15.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.740+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 --> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc784068fa0 con 0x7fc7700ac220 2024-09-13T17:25:15.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc781ffb640 1 -- 172.21.15.44:0/2348463827 <== osd.11 v2:172.21.15.100:6824/1155650564 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc784068fa0 con 0x7fc7700ac220 2024-09-13T17:25:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] conn(0x7fc7700ac220 msgr2=0x7fc7700ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] conn(0x7fc7700ac220 0x7fc7700ae6c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc7700a1190 msgr2=0x7fc7700a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc7700a1190 0x7fc7700a3650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc774005fd0 tx=0x7fc774005850 comp rx=0 tx=0).stop 2024-09-13T17:25:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc784100fc0 msgr2=0x7fc7841983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 --2- 172.21.15.44:0/2348463827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc784100fc0 0x7fc7841983c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e9f0 tx=0x7fc76c00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:25:15.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc7892df640 1 -- 172.21.15.44:0/2348463827 reap_dead start 2024-09-13T17:25:15.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 shutdown_connections 2024-09-13T17:25:15.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 >> 172.21.15.44:0/2348463827 conn(0x7fc7840fa9a0 msgr2=0x7fc7840ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:15.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 shutdown_connections 2024-09-13T17:25:15.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:15.741+0000 7fc78ad42640 1 -- 172.21.15.44:0/2348463827 wait complete. 2024-09-13T17:25:17.071 INFO:teuthology.orchestra.run.smithi044.stdout:55834574918 2024-09-13T17:25:17.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-13T17:25:17.297 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:17 smithi044 ceph-mon[31175]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-13T17:25:17.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:17 smithi100 ceph-mon[35055]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-13T17:25:17.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:17 smithi070 ceph-mon[34502]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-13T17:25:19.347 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:19 smithi044 ceph-mon[31175]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:19.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 -- 172.21.15.44:0/2450162675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140feaf0 msgr2=0x7f20140feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:19.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 --2- 172.21.15.44:0/2450162675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140feaf0 0x7f20140feed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2004009900 tx=0x7f200402a410 comp rx=0 tx=0).stop 2024-09-13T17:25:19.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 -- 172.21.15.44:0/2450162675 shutdown_connections 2024-09-13T17:25:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 --2- 172.21.15.44:0/2450162675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f201410c6a0 0x7f201410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 --2- 172.21.15.44:0/2450162675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140ff4a0 0x7f201410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 --2- 172.21.15.44:0/2450162675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140feaf0 0x7f20140feed0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 -- 172.21.15.44:0/2450162675 >> 172.21.15.44:0/2450162675 conn(0x7f20140fa9a0 msgr2=0x7f20140fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 -- 172.21.15.44:0/2450162675 shutdown_connections 2024-09-13T17:25:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.344+0000 7f20199dc640 1 -- 172.21.15.44:0/2450162675 wait complete. 2024-09-13T17:25:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.345+0000 7f20199dc640 1 Processor -- start 2024-09-13T17:25:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.345+0000 7f20199dc640 1 -- start start 2024-09-13T17:25:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20199dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140feaf0 0x7f2014198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20199dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140ff4a0 0x7f2014198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20199dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f201410c6a0 0x7f201419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20199dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419d370 con 0x7f20140ff4a0 2024-09-13T17:25:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20199dc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419d4e0 con 0x7f20140feaf0 2024-09-13T17:25:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20199dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419d650 con 0x7f201410c6a0 2024-09-13T17:25:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140ff4a0 0x7f2014198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140ff4a0 0x7f2014198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42420/0 (socket says 172.21.15.44:42420) 2024-09-13T17:25:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 -- 172.21.15.44:0/299659119 learned_addr learned my addr 172.21.15.44:0/299659119 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f2012ffd640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140feaf0 0x7f2014198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20137fe640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f201410c6a0 0x7f201419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 -- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f201410c6a0 msgr2=0x7f201419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f201410c6a0 0x7f201419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 -- 172.21.15.44:0/299659119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140feaf0 msgr2=0x7f2014198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140feaf0 0x7f2014198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f20127fc640 1 -- 172.21.15.44:0/299659119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2004009560 con 0x7f20140ff4a0 2024-09-13T17:25:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.346+0000 7f2012ffd640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140feaf0 0x7f2014198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:25:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.347+0000 7f20127fc640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140ff4a0 0x7f2014198970 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f200800d950 tx=0x7f200800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.347+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008009c30 con 0x7f20140ff4a0 2024-09-13T17:25:19.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.347+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2008009d90 con 0x7f20140ff4a0 2024-09-13T17:25:19.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.347+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201419d930 con 0x7f20140ff4a0 2024-09-13T17:25:19.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.347+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200800f620 con 0x7f20140ff4a0 2024-09-13T17:25:19.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.347+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20141a51f0 con 0x7f20140ff4a0 2024-09-13T17:25:19.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.348+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2014069000 con 0x7f20140ff4a0 2024-09-13T17:25:19.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.349+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f20080109f0 con 0x7f20140ff4a0 2024-09-13T17:25:19.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.350+0000 7f20189da640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:19.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.350+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f20080ccd10 con 0x7f20140ff4a0 2024-09-13T17:25:19.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.350+0000 7f2012ffd640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:19.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.350+0000 7f20189da640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:19.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.350+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 --> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ffc0aed10 con 0x7f1ffc0ac1a0 2024-09-13T17:25:19.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.350+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20080cd120 con 0x7f20140ff4a0 2024-09-13T17:25:19.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.351+0000 7f2012ffd640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f200402aa00 tx=0x7f2004005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:19.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.351+0000 7f20137fe640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:19.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.351+0000 7f20137fe640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:19.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.352+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== osd.2 v2:172.21.15.100:6800/4273545793 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1ffc0aed10 con 0x7f1ffc0ac1a0 2024-09-13T17:25:19.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.365+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 --> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2014060b70 con 0x7f1ffc0ac1a0 2024-09-13T17:25:19.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.366+0000 7f20189da640 1 -- 172.21.15.44:0/299659119 <== osd.2 v2:172.21.15.100:6800/4273545793 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2014060b70 con 0x7f1ffc0ac1a0 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] conn(0x7f1ffc0ac1a0 msgr2=0x7f1ffc0ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1ffc0a1110 msgr2=0x7f1ffc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f200402aa00 tx=0x7f2004005a20 comp rx=0 tx=0).stop 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140ff4a0 msgr2=0x7f2014198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 --2- 172.21.15.44:0/299659119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140ff4a0 0x7f2014198970 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f200800d950 tx=0x7f200800de10 comp rx=0 tx=0).stop 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20137fe640 1 -- 172.21.15.44:0/299659119 reap_dead start 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 shutdown_connections 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 >> 172.21.15.44:0/299659119 conn(0x7f20140fa9a0 msgr2=0x7f20140fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 shutdown_connections 2024-09-13T17:25:19.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:19.367+0000 7f20199dc640 1 -- 172.21.15.44:0/299659119 wait complete. 2024-09-13T17:25:19.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:19 smithi100 ceph-mon[35055]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:19.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:19 smithi070 ceph-mon[34502]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:20.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 -- 172.21.15.44:0/4133200316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c0fffe0 msgr2=0x7fb21c1003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:20.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 --2- 172.21.15.44:0/4133200316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c0fffe0 0x7fb21c1003c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb218009930 tx=0x7fb21802a420 comp rx=0 tx=0).stop 2024-09-13T17:25:20.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 -- 172.21.15.44:0/4133200316 shutdown_connections 2024-09-13T17:25:20.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 --2- 172.21.15.44:0/4133200316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c105110 0x7fb21c10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 --2- 172.21.15.44:0/4133200316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb21c100990 0x7fb21c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 --2- 172.21.15.44:0/4133200316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c0fffe0 0x7fb21c1003c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 -- 172.21.15.44:0/4133200316 >> 172.21.15.44:0/4133200316 conn(0x7fb21c0fbae0 msgr2=0x7fb21c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:20.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 -- 172.21.15.44:0/4133200316 shutdown_connections 2024-09-13T17:25:20.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.009+0000 7fb223a7d640 1 -- 172.21.15.44:0/4133200316 wait complete. 2024-09-13T17:25:20.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 Processor -- start 2024-09-13T17:25:20.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 -- start start 2024-09-13T17:25:20.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb21c0fffe0 0x7fb21c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:20.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c100990 0x7fb21c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:20.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c105110 0x7fb21c073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:20.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1a44f0 con 0x7fb21c100990 2024-09-13T17:25:20.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1a4660 con 0x7fb21c0fffe0 2024-09-13T17:25:20.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb223a7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1a4820 con 0x7fb21c105110 2024-09-13T17:25:20.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb221018640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c100990 0x7fb21c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:20.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb221018640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c100990 0x7fb21c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45332/0 (socket says 172.21.15.44:45332) 2024-09-13T17:25:20.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.010+0000 7fb221018640 1 -- 172.21.15.44:0/2530929523 learned_addr learned my addr 172.21.15.44:0/2530929523 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:20.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb22201a640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c105110 0x7fb21c073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:20.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221819640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb21c0fffe0 0x7fb21c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:20.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221018640 1 -- 172.21.15.44:0/2530929523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c105110 msgr2=0x7fb21c073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:20.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221018640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c105110 0x7fb21c073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221018640 1 -- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb21c0fffe0 msgr2=0x7fb21c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221018640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb21c0fffe0 0x7fb21c0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221018640 1 -- 172.21.15.44:0/2530929523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009560 con 0x7fb21c100990 2024-09-13T17:25:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221819640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb21c0fffe0 0x7fb21c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb22201a640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c105110 0x7fb21c073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb221018640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c100990 0x7fb21c072cf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c920 tx=0x7fb20c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007c70 con 0x7fb21c100990 2024-09-13T17:25:20.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c1a4b00 con 0x7fb21c100990 2024-09-13T17:25:20.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb20c007dd0 con 0x7fb21c100990 2024-09-13T17:25:20.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c018730 con 0x7fb21c100990 2024-09-13T17:25:20.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.011+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c1a5000 con 0x7fb21c100990 2024-09-13T17:25:20.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.012+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb21c10ff10 con 0x7fb21c100990 2024-09-13T17:25:20.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.013+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb20c00f450 con 0x7fb21c100990 2024-09-13T17:25:20.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb212ffd640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb2080a1110 0x7fb2080a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:20.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb221819640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb2080a1110 0x7fb2080a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:20.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb20c0cd3c0 con 0x7fb21c100990 2024-09-13T17:25:20.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb212ffd640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] conn(0x7fb2080ac1a0 0x7fb2080ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 --> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2080aed10 con 0x7fb2080ac1a0 2024-09-13T17:25:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb20c0cd7d0 con 0x7fb21c100990 2024-09-13T17:25:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.014+0000 7fb221819640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb2080a1110 0x7fb2080a35d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb218006010 tx=0x7fb218005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.015+0000 7fb22201a640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] conn(0x7fb2080ac1a0 0x7fb2080ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.015+0000 7fb22201a640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] conn(0x7fb2080ac1a0 0x7fb2080ae640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.016+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== osd.10 v2:172.21.15.70:6824/752114837 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb2080aed10 con 0x7fb2080ac1a0 2024-09-13T17:25:20.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.029+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 --> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb21c060bf0 con 0x7fb2080ac1a0 2024-09-13T17:25:20.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb212ffd640 1 -- 172.21.15.44:0/2530929523 <== osd.10 v2:172.21.15.70:6824/752114837 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb21c060bf0 con 0x7fb2080ac1a0 2024-09-13T17:25:20.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] conn(0x7fb2080ac1a0 msgr2=0x7fb2080ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb223a7d640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] conn(0x7fb2080ac1a0 0x7fb2080ae640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb2080a1110 msgr2=0x7fb2080a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb223a7d640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fb2080a1110 0x7fb2080a35d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb218006010 tx=0x7fb218005a40 comp rx=0 tx=0).stop 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c100990 msgr2=0x7fb21c072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb223a7d640 1 --2- 172.21.15.44:0/2530929523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21c100990 0x7fb21c072cf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c920 tx=0x7fb20c00cde0 comp rx=0 tx=0).stop 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.030+0000 7fb22201a640 1 -- 172.21.15.44:0/2530929523 reap_dead start 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.031+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 shutdown_connections 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.031+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 >> 172.21.15.44:0/2530929523 conn(0x7fb21c0fbae0 msgr2=0x7fb21c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.031+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 shutdown_connections 2024-09-13T17:25:20.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:20.031+0000 7fb223a7d640 1 -- 172.21.15.44:0/2530929523 wait complete. 2024-09-13T17:25:20.810 INFO:teuthology.orchestra.run.smithi044.stdout:304942678046 2024-09-13T17:25:20.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-13T17:25:21.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:21 smithi044 ceph-mon[31175]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:21.378 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:21 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:25:21.398 INFO:teuthology.orchestra.run.smithi044.stdout:103079215164 2024-09-13T17:25:21.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-13T17:25:21.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:21 smithi070 ceph-mon[34502]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:21.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:21 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:25:21.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:21 smithi100 ceph-mon[35055]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:21.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:21 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:25:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 -- 172.21.15.44:0/4182068287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c101880 msgr2=0x7f4b2c109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 --2- 172.21.15.44:0/4182068287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c101880 0x7f4b2c109d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b2002a480 comp rx=0 tx=0).stop 2024-09-13T17:25:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 -- 172.21.15.44:0/4182068287 shutdown_connections 2024-09-13T17:25:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 --2- 172.21.15.44:0/4182068287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2c10a390 0x7f4b2c10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 --2- 172.21.15.44:0/4182068287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c101880 0x7f4b2c109d70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 --2- 172.21.15.44:0/4182068287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b2c100f60 0x7f4b2c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 -- 172.21.15.44:0/4182068287 >> 172.21.15.44:0/4182068287 conn(0x7f4b2c0fa9a0 msgr2=0x7f4b2c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 -- 172.21.15.44:0/4182068287 shutdown_connections 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.839+0000 7f4b316aa640 1 -- 172.21.15.44:0/4182068287 wait complete. 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 Processor -- start 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 -- start start 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b2c100f60 0x7f4b2c0febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:21.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2c101880 0x7f4b2c0ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c10a390 0x7f4b2c0ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1a4590 con 0x7f4b2c10a390 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1a4700 con 0x7f4b2c100f60 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b316aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1a48c0 con 0x7f4b2c101880 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.840+0000 7f4b2b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c10a390 0x7f4b2c0ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2c101880 0x7f4b2c0ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b2c100f60 0x7f4b2c0febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:21.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c10a390 0x7f4b2c0ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45346/0 (socket says 172.21.15.44:45346) 2024-09-13T17:25:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 -- 172.21.15.44:0/2305523924 learned_addr learned my addr 172.21.15.44:0/2305523924 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2c101880 0x7f4b2c0ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:60514/0 (socket says 172.21.15.44:60514) 2024-09-13T17:25:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 -- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2c101880 msgr2=0x7f4b2c0ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2c101880 0x7f4b2c0ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 -- 172.21.15.44:0/2305523924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b2c100f60 msgr2=0x7f4b2c0febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b2c100f60 0x7f4b2c0febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 -- 172.21.15.44:0/2305523924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20009560 con 0x7f4b2c10a390 2024-09-13T17:25:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2affd640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b2c100f60 0x7f4b2c0febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b2b7fe640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c10a390 0x7f4b2c0ff640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400eaa0 tx=0x7f4b2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2400cdb0 con 0x7f4b2c10a390 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c1a4ba0 con 0x7f4b2c10a390 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b24005020 con 0x7f4b2c10a390 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240116d0 con 0x7f4b2c10a390 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.841+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c1a5020 con 0x7f4b2c10a390 2024-09-13T17:25:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.842+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b2c069000 con 0x7f4b2c10a390 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.843+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4b24005190 con 0x7f4b2c10a390 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.844+0000 7f4b13fff640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4b140a1110 0x7f4b140a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.844+0000 7f4b2affd640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4b140a1110 0x7f4b140a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.844+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4b240cd220 con 0x7f4b2c10a390 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.844+0000 7f4b13fff640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] conn(0x7f4b140ac1a0 0x7f4b140ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.844+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 --> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b140aed10 con 0x7f4b140ac1a0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.844+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b240cd630 con 0x7f4b2c10a390 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.845+0000 7f4b2affd640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4b140a1110 0x7f4b140a35d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009fc0 tx=0x7f4b1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.845+0000 7f4b2a7fc640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] conn(0x7f4b140ac1a0 0x7f4b140ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.845+0000 7f4b2a7fc640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] conn(0x7f4b140ac1a0 0x7f4b140ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.846+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== osd.8 v2:172.21.15.100:6816/2612262054 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4b140aed10 con 0x7f4b140ac1a0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.859+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 --> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b2c10fed0 con 0x7f4b140ac1a0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b13fff640 1 -- 172.21.15.44:0/2305523924 <== osd.8 v2:172.21.15.100:6816/2612262054 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b2c10fed0 con 0x7f4b140ac1a0 2024-09-13T17:25:22.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] conn(0x7f4b140ac1a0 msgr2=0x7f4b140ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] conn(0x7f4b140ac1a0 0x7f4b140ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4b140a1110 msgr2=0x7f4b140a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4b140a1110 0x7f4b140a35d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009fc0 tx=0x7f4b1c005f90 comp rx=0 tx=0).stop 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c10a390 msgr2=0x7f4b2c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 --2- 172.21.15.44:0/2305523924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2c10a390 0x7f4b2c0ff640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400eaa0 tx=0x7f4b2400ef60 comp rx=0 tx=0).stop 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b2b7fe640 1 -- 172.21.15.44:0/2305523924 reap_dead start 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 shutdown_connections 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 >> 172.21.15.44:0/2305523924 conn(0x7f4b2c0fa9a0 msgr2=0x7f4b2c0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 shutdown_connections 2024-09-13T17:25:22.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:21.860+0000 7f4b316aa640 1 -- 172.21.15.44:0/2305523924 wait complete. 2024-09-13T17:25:22.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 -- 172.21.15.44:0/2271678949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd56010c670 msgr2=0x7fd56010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 --2- 172.21.15.44:0/2271678949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd56010c670 0x7fd56010ea60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd558009ca0 tx=0x7fd55802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 -- 172.21.15.44:0/2271678949 shutdown_connections 2024-09-13T17:25:22.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 --2- 172.21.15.44:0/2271678949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5600ff470 0x7fd56010c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 --2- 172.21.15.44:0/2271678949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd56010c670 0x7fd56010ea60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 --2- 172.21.15.44:0/2271678949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5600feac0 0x7fd5600feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 -- 172.21.15.44:0/2271678949 >> 172.21.15.44:0/2271678949 conn(0x7fd5600fa9b0 msgr2=0x7fd5600fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 -- 172.21.15.44:0/2271678949 shutdown_connections 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.036+0000 7fd56812a640 1 -- 172.21.15.44:0/2271678949 wait complete. 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 Processor -- start 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 -- start start 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5600feac0 0x7fd560198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5600ff470 0x7fd5601988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56010c670 0x7fd56019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019d2d0 con 0x7fd5600ff470 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019d440 con 0x7fd5600feac0 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd56812a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019d5b0 con 0x7fd56010c670 2024-09-13T17:25:22.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd5656c5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5600ff470 0x7fd5601988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd565ec6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5600feac0 0x7fd560198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd5656c5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5600ff470 0x7fd5601988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45368/0 (socket says 172.21.15.44:45368) 2024-09-13T17:25:22.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.037+0000 7fd5656c5640 1 -- 172.21.15.44:0/344452439 learned_addr learned my addr 172.21.15.44:0/344452439 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:22.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5666c7640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56010c670 0x7fd56019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5656c5640 1 -- 172.21.15.44:0/344452439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56010c670 msgr2=0x7fd56019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5656c5640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56010c670 0x7fd56019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5656c5640 1 -- 172.21.15.44:0/344452439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5600feac0 msgr2=0x7fd560198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5656c5640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5600feac0 0x7fd560198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5656c5640 1 -- 172.21.15.44:0/344452439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558009900 con 0x7fd5600ff470 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd565ec6640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5600feac0 0x7fd560198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd5656c5640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5600ff470 0x7fd5601988d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550009720 con 0x7fd5600ff470 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd55000cd80 con 0x7fd5600ff470 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56019d890 con 0x7fd5600ff470 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550010430 con 0x7fd5600ff470 2024-09-13T17:25:22.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.038+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5601a5150 con 0x7fd5600ff470 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.039+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd560112160 con 0x7fd5600ff470 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.041+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd550004f90 con 0x7fd5600ff470 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd556ffd640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd54c0a1190 0x7fd54c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd550014070 con 0x7fd5600ff470 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd565ec6640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd54c0a1190 0x7fd54c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd556ffd640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] conn(0x7fd54c0ac220 0x7fd54c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 --> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd54c0aed90 con 0x7fd54c0ac220 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5500cd750 con 0x7fd5600ff470 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd5666c7640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] conn(0x7fd54c0ac220 0x7fd54c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:22.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.042+0000 7fd565ec6640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd54c0a1190 0x7fd54c0a3650 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009fc0 tx=0x7fd55c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.043+0000 7fd5666c7640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] conn(0x7fd54c0ac220 0x7fd54c0ae6c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.043+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== osd.0 v2:172.21.15.44:6802/637354587 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd54c0aed90 con 0x7fd54c0ac220 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.059+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 --> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd560060bb0 con 0x7fd54c0ac220 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.059+0000 7fd556ffd640 1 -- 172.21.15.44:0/344452439 <== osd.0 v2:172.21.15.44:6802/637354587 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd560060bb0 con 0x7fd54c0ac220 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] conn(0x7fd54c0ac220 msgr2=0x7fd54c0ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] conn(0x7fd54c0ac220 0x7fd54c0ae6c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd54c0a1190 msgr2=0x7fd54c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd54c0a1190 0x7fd54c0a3650 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009fc0 tx=0x7fd55c005f90 comp rx=0 tx=0).stop 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5600ff470 msgr2=0x7fd5601988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:22.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 --2- 172.21.15.44:0/344452439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5600ff470 0x7fd5601988d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000edf0 comp rx=0 tx=0).stop 2024-09-13T17:25:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd5666c7640 1 -- 172.21.15.44:0/344452439 reap_dead start 2024-09-13T17:25:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 shutdown_connections 2024-09-13T17:25:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 >> 172.21.15.44:0/344452439 conn(0x7fd5600fa9b0 msgr2=0x7fd5600fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 shutdown_connections 2024-09-13T17:25:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:22.060+0000 7fd56812a640 1 -- 172.21.15.44:0/344452439 wait complete. 2024-09-13T17:25:23.124 INFO:teuthology.orchestra.run.smithi044.stdout:339302416407 2024-09-13T17:25:23.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-13T17:25:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:23 smithi100 ceph-mon[35055]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-13T17:25:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:23 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:23 smithi044 ceph-mon[31175]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-13T17:25:23.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:23 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:23 smithi070 ceph-mon[34502]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-13T17:25:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:23.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:23 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:24.082 INFO:teuthology.orchestra.run.smithi044.stdout:34359738445 2024-09-13T17:25:24.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-13T17:25:24.507 INFO:teuthology.orchestra.run.smithi044.stdout:214748364842 2024-09-13T17:25:24.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-13T17:25:25.117 INFO:teuthology.orchestra.run.smithi044.stdout:249108103207 2024-09-13T17:25:25.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-13T17:25:25.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:25 smithi100 ceph-mon[35055]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:25.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:25 smithi044 ceph-mon[31175]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:25.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:25 smithi070 ceph-mon[34502]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:25:25.905 INFO:teuthology.orchestra.run.smithi044.stdout:77309411395 2024-09-13T17:25:25.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-13T17:25:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 -- 172.21.15.44:0/3184404812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 msgr2=0x7f954c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 --2- 172.21.15.44:0/3184404812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c1013b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f953c009900 tx=0x7f953c02a480 comp rx=0 tx=0).stop 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 -- 172.21.15.44:0/3184404812 shutdown_connections 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 --2- 172.21.15.44:0/3184404812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954c1018f0 0x7f954c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 --2- 172.21.15.44:0/3184404812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f954c10a370 0x7f954c10c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 --2- 172.21.15.44:0/3184404812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c1013b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 -- 172.21.15.44:0/3184404812 >> 172.21.15.44:0/3184404812 conn(0x7f954c0fa9b0 msgr2=0x7f954c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 -- 172.21.15.44:0/3184404812 shutdown_connections 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.088+0000 7f9552825640 1 -- 172.21.15.44:0/3184404812 wait complete. 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 Processor -- start 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 -- start start 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954c1018f0 0x7f954c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f954c10a370 0x7f954c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c19b0a0 con 0x7f954c100fd0 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c19b210 con 0x7f954c10a370 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9552825640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c19b380 con 0x7f954c1018f0 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f954bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f954bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45382/0 (socket says 172.21.15.44:45382) 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f954bfff640 1 -- 172.21.15.44:0/3134174153 learned_addr learned my addr 172.21.15.44:0/3134174153 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f954b7fe640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954c1018f0 0x7f954c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.089+0000 7f9550dc2640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f954c10a370 0x7f954c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f954bfff640 1 -- 172.21.15.44:0/3134174153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954c1018f0 msgr2=0x7f954c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f954bfff640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954c1018f0 0x7f954c1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f954bfff640 1 -- 172.21.15.44:0/3134174153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f954c10a370 msgr2=0x7f954c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f954bfff640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f954c10a370 0x7f954c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f954bfff640 1 -- 172.21.15.44:0/3134174153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c009560 con 0x7f954c100fd0 2024-09-13T17:25:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f954bfff640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c196160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f953c0057c0 tx=0x7f953c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c032e10 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f953c0046c0 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c19b600 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c003850 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.090+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c19b920 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.091+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f954c068740 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.092+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f953c004aa0 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.093+0000 7f95497fa640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95340a1190 0x7f95340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.093+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f953c0ede60 con 0x7f954c100fd0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.093+0000 7f954b7fe640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95340a1190 0x7f95340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.093+0000 7f95497fa640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] conn(0x7f95340ac220 0x7f95340ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.093+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 --> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95340aed90 con 0x7f95340ac220 2024-09-13T17:25:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.093+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f953c0ee270 con 0x7f954c100fd0 2024-09-13T17:25:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.094+0000 7f9550dc2640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] conn(0x7f95340ac220 0x7f95340ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.094+0000 7f954b7fe640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95340a1190 0x7f95340a3650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f954c197710 tx=0x7f954000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.094+0000 7f9550dc2640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] conn(0x7f95340ac220 0x7f95340ae6c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.095+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== osd.9 v2:172.21.15.44:6826/280929733 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f95340aed90 con 0x7f95340ac220 2024-09-13T17:25:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.109+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 --> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f954c10ff30 con 0x7f95340ac220 2024-09-13T17:25:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.109+0000 7f95497fa640 1 -- 172.21.15.44:0/3134174153 <== osd.9 v2:172.21.15.44:6826/280929733 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9544009b40 con 0x7f95340ac220 2024-09-13T17:25:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] conn(0x7f95340ac220 msgr2=0x7f95340ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] conn(0x7f95340ac220 0x7f95340ae6c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95340a1190 msgr2=0x7f95340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f95340a1190 0x7f95340a3650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f954c197710 tx=0x7f954000b410 comp rx=0 tx=0).stop 2024-09-13T17:25:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 msgr2=0x7f954c196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 --2- 172.21.15.44:0/3134174153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954c100fd0 0x7f954c196160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f953c0057c0 tx=0x7f953c032c20 comp rx=0 tx=0).stop 2024-09-13T17:25:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9550dc2640 1 -- 172.21.15.44:0/3134174153 reap_dead start 2024-09-13T17:25:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 shutdown_connections 2024-09-13T17:25:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 >> 172.21.15.44:0/3134174153 conn(0x7f954c0fa9b0 msgr2=0x7f954c0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 shutdown_connections 2024-09-13T17:25:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:26.110+0000 7f9552825640 1 -- 172.21.15.44:0/3134174153 wait complete. 2024-09-13T17:25:27.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:27 smithi044 ceph-mon[31175]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-13T17:25:27.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:27 smithi100 ceph-mon[35055]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-13T17:25:27.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:27 smithi070 ceph-mon[34502]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-13T17:25:27.729 INFO:teuthology.orchestra.run.smithi044.stdout:283467841571 2024-09-13T17:25:27.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-13T17:25:28.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 -- 172.21.15.44:0/4294856205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a8104000 msgr2=0x7ff3a810bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 --2- 172.21.15.44:0/4294856205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a8104000 0x7ff3a810bb80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009cd0 tx=0x7ff3a002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 -- 172.21.15.44:0/4294856205 shutdown_connections 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 --2- 172.21.15.44:0/4294856205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a80ff880 0x7ff3a81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 --2- 172.21.15.44:0/4294856205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a8104000 0x7ff3a810bb80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 --2- 172.21.15.44:0/4294856205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a80feed0 0x7ff3a80ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 -- 172.21.15.44:0/4294856205 >> 172.21.15.44:0/4294856205 conn(0x7ff3a80fa9b0 msgr2=0x7ff3a80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 -- 172.21.15.44:0/4294856205 shutdown_connections 2024-09-13T17:25:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.137+0000 7ff3acea5640 1 -- 172.21.15.44:0/4294856205 wait complete. 2024-09-13T17:25:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.138+0000 7ff3acea5640 1 Processor -- start 2024-09-13T17:25:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.138+0000 7ff3acea5640 1 -- start start 2024-09-13T17:25:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.138+0000 7ff3acea5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a80feed0 0x7ff3a8194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.138+0000 7ff3acea5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a80ff880 0x7ff3a81952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3acea5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a8104000 0x7ff3a81957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3acea5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a81a4520 con 0x7ff3a80feed0 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3acea5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a81a4660 con 0x7ff3a8104000 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3acea5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a81a4820 con 0x7ff3a80ff880 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a80feed0 0x7ff3a8194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a80feed0 0x7ff3a8194d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45400/0 (socket says 172.21.15.44:45400) 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 -- 172.21.15.44:0/3302703991 learned_addr learned my addr 172.21.15.44:0/3302703991 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a5d9b640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a80ff880 0x7ff3a81952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a6d9d640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a8104000 0x7ff3a81957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 -- 172.21.15.44:0/3302703991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a80ff880 msgr2=0x7ff3a81952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a80ff880 0x7ff3a81952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 -- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a8104000 msgr2=0x7ff3a81957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a8104000 0x7ff3a81957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 -- 172.21.15.44:0/3302703991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a0009900 con 0x7ff3a80feed0 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a5d9b640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a80ff880 0x7ff3a81952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3a659c640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a80feed0 0x7ff3a8194d70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff39000d950 tx=0x7ff39000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390009c30 con 0x7ff3a80feed0 2024-09-13T17:25:28.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a81a4b00 con 0x7ff3a80feed0 2024-09-13T17:25:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff390009d90 con 0x7ff3a80feed0 2024-09-13T17:25:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.139+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39000f600 con 0x7ff3a80feed0 2024-09-13T17:25:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.140+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a81a5000 con 0x7ff3a80feed0 2024-09-13T17:25:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.140+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3a810ee40 con 0x7ff3a80feed0 2024-09-13T17:25:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.142+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff390010460 con 0x7ff3a80feed0 2024-09-13T17:25:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.143+0000 7ff39f7fe640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff3940a1110 0x7ff3940a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.143+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3900cce00 con 0x7ff3a80feed0 2024-09-13T17:25:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.143+0000 7ff3a5d9b640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff3940a1110 0x7ff3940a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.143+0000 7ff39f7fe640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] conn(0x7ff3940ac1a0 0x7ff3940ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.143+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 --> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3940aed10 con 0x7ff3940ac1a0 2024-09-13T17:25:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.143+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3900cd210 con 0x7ff3a80feed0 2024-09-13T17:25:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.144+0000 7ff3a6d9d640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] conn(0x7ff3940ac1a0 0x7ff3940ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.144+0000 7ff3a5d9b640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff3940a1110 0x7ff3940a35d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff398005e00 tx=0x7ff398005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.144+0000 7ff3a6d9d640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] conn(0x7ff3940ac1a0 0x7ff3940ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.145+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== osd.4 v2:172.21.15.70:6808/2214036944 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff3940aed10 con 0x7ff3940ac1a0 2024-09-13T17:25:28.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.160+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 --> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3a8060bb0 con 0x7ff3940ac1a0 2024-09-13T17:25:28.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.160+0000 7ff39f7fe640 1 -- 172.21.15.44:0/3302703991 <== osd.4 v2:172.21.15.70:6808/2214036944 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff3a8060bb0 con 0x7ff3940ac1a0 2024-09-13T17:25:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] conn(0x7ff3940ac1a0 msgr2=0x7ff3940ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] conn(0x7ff3940ac1a0 0x7ff3940ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff3940a1110 msgr2=0x7ff3940a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff3940a1110 0x7ff3940a35d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff398005e00 tx=0x7ff398005c70 comp rx=0 tx=0).stop 2024-09-13T17:25:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a80feed0 msgr2=0x7ff3a8194d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 --2- 172.21.15.44:0/3302703991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3a80feed0 0x7ff3a8194d70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff39000d950 tx=0x7ff39000de10 comp rx=0 tx=0).stop 2024-09-13T17:25:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3a6d9d640 1 -- 172.21.15.44:0/3302703991 reap_dead start 2024-09-13T17:25:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 shutdown_connections 2024-09-13T17:25:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 >> 172.21.15.44:0/3302703991 conn(0x7ff3a80fa9b0 msgr2=0x7ff3a80fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 shutdown_connections 2024-09-13T17:25:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.161+0000 7ff3acea5640 1 -- 172.21.15.44:0/3302703991 wait complete. 2024-09-13T17:25:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 -- 172.21.15.44:0/1778501048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 msgr2=0x7febf010bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1778501048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf010bb80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febe8009cd0 tx=0x7febe802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 -- 172.21.15.44:0/1778501048 shutdown_connections 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1778501048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febf00ff880 0x7febf01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1778501048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf010bb80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1778501048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf00feed0 0x7febf00ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 -- 172.21.15.44:0/1778501048 >> 172.21.15.44:0/1778501048 conn(0x7febf00fa9b0 msgr2=0x7febf00fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 -- 172.21.15.44:0/1778501048 shutdown_connections 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.692+0000 7febf4c3d640 1 -- 172.21.15.44:0/1778501048 wait complete. 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 Processor -- start 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 -- start start 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febf00feed0 0x7febf0196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf00ff880 0x7febf01974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf01a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf0197d60 con 0x7febf0104000 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01a2540 con 0x7febf00ff880 2024-09-13T17:25:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febf4c3d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01a2680 con 0x7febf00feed0 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf01a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febedd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf00ff880 0x7febf01974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf01a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45428/0 (socket says 172.21.15.44:45428) 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 -- 172.21.15.44:0/1856740832 learned_addr learned my addr 172.21.15.44:0/1856740832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febedd9b640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf00ff880 0x7febf01974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36928/0 (socket says 172.21.15.44:36928) 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febee59c640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febf00feed0 0x7febf0196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 -- 172.21.15.44:0/1856740832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febf00feed0 msgr2=0x7febf0196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febf00feed0 0x7febf0196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 -- 172.21.15.44:0/1856740832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf00ff880 msgr2=0x7febf01974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf00ff880 0x7febf01974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.693+0000 7febeed9d640 1 -- 172.21.15.44:0/1856740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009900 con 0x7febf0104000 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.694+0000 7febeed9d640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf01a2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febe8005cc0 tx=0x7febe8030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.694+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8003970 con 0x7febf0104000 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.694+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf01a28d0 con 0x7febf0104000 2024-09-13T17:25:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.694+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe8004040 con 0x7febf0104000 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.694+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8045670 con 0x7febf0104000 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.694+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf01a2dd0 con 0x7febf0104000 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.695+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7febf010ee40 con 0x7febf0104000 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.696+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7febe800a040 con 0x7febf0104000 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febdf7fe640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7febd80a1110 0x7febd80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7febe80ee4d0 con 0x7febf0104000 2024-09-13T17:25:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febee59c640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7febd80a1110 0x7febd80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febdf7fe640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] conn(0x7febd80ac1a0 0x7febd80ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 --> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd80aed10 con 0x7febd80ac1a0 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe80f0b50 con 0x7febf0104000 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febedd9b640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] conn(0x7febd80ac1a0 0x7febd80ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febee59c640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7febd80a1110 0x7febd80a35d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febe00058c0 tx=0x7febe000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.697+0000 7febedd9b640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] conn(0x7febd80ac1a0 0x7febd80ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.698+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== osd.6 v2:172.21.15.44:6818/3425095841 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7febd80aed10 con 0x7febd80ac1a0 2024-09-13T17:25:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.714+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 --> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7febf01003b0 con 0x7febd80ac1a0 2024-09-13T17:25:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febdf7fe640 1 -- 172.21.15.44:0/1856740832 <== osd.6 v2:172.21.15.44:6818/3425095841 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7febf01003b0 con 0x7febd80ac1a0 2024-09-13T17:25:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] conn(0x7febd80ac1a0 msgr2=0x7febd80ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] conn(0x7febd80ac1a0 0x7febd80ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7febd80a1110 msgr2=0x7febd80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7febd80a1110 0x7febd80a35d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febe00058c0 tx=0x7febe000a2d0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 msgr2=0x7febf01a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febf4c3d640 1 --2- 172.21.15.44:0/1856740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0104000 0x7febf01a2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febe8005cc0 tx=0x7febe8030ec0 comp rx=0 tx=0).stop 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.715+0000 7febeed9d640 1 -- 172.21.15.44:0/1856740832 reap_dead start 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.716+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 shutdown_connections 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.716+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 >> 172.21.15.44:0/1856740832 conn(0x7febf00fa9b0 msgr2=0x7febf00fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.716+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 shutdown_connections 2024-09-13T17:25:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:28.716+0000 7febf4c3d640 1 -- 172.21.15.44:0/1856740832 wait complete. 2024-09-13T17:25:29.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:29 smithi044 ceph-mon[31175]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:25:29.386 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:29.387 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:29.387 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:29.394 INFO:teuthology.orchestra.run.smithi044.stdout:193273528371 2024-09-13T17:25:29.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-13T17:25:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:29.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:29 smithi070 ceph-mon[34502]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:25:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:29 smithi100 ceph-mon[35055]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:25:31.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 -- 172.21.15.44:0/1670776498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e40ff4a0 msgr2=0x7f26e410c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:31.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 --2- 172.21.15.44:0/1670776498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e40ff4a0 0x7f26e410c160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009900 tx=0x7f26d802a410 comp rx=0 tx=0).stop 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 -- 172.21.15.44:0/1670776498 shutdown_connections 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 --2- 172.21.15.44:0/1670776498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e410c6a0 0x7f26e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 --2- 172.21.15.44:0/1670776498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e40ff4a0 0x7f26e410c160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 --2- 172.21.15.44:0/1670776498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26e40feaf0 0x7f26e40feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 -- 172.21.15.44:0/1670776498 >> 172.21.15.44:0/1670776498 conn(0x7f26e40fa9a0 msgr2=0x7f26e40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 -- 172.21.15.44:0/1670776498 shutdown_connections 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.990+0000 7f26ebe66640 1 -- 172.21.15.44:0/1670776498 wait complete. 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 Processor -- start 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 -- start start 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26e40feaf0 0x7f26e41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e40ff4a0 0x7f26e41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e410c6a0 0x7f26e419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e419d3f0 con 0x7f26e410c6a0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e419d560 con 0x7f26e40ff4a0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ebe66640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e419d6d0 con 0x7f26e40feaf0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e410c6a0 0x7f26e419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e410c6a0 0x7f26e419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51186/0 (socket says 172.21.15.44:51186) 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 -- 172.21.15.44:0/3909200481 learned_addr learned my addr 172.21.15.44:0/3909200481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26e9c02640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26e40feaf0 0x7f26e41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26e9401640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e40ff4a0 0x7f26e41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 -- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26e40feaf0 msgr2=0x7f26e41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26e40feaf0 0x7f26e41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 -- 172.21.15.44:0/3909200481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e40ff4a0 msgr2=0x7f26e41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e40ff4a0 0x7f26e41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26ea403640 1 -- 172.21.15.44:0/3909200481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009560 con 0x7f26e410c6a0 2024-09-13T17:25:31.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.991+0000 7f26e9c02640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26e40feaf0 0x7f26e41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.992+0000 7f26ea403640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e410c6a0 0x7f26e419cdf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00eb10 tx=0x7f26dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.992+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00ce80 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.992+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26dc005020 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.992+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e419d9b0 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.992+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc010690 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.992+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e41a5270 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.993+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26e4112180 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.994+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f26dc004a40 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26d6ffd640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26d00a1190 0x7f26d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26e9c02640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26d00a1190 0x7f26d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26dc014070 con 0x7f26e410c6a0 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26d6ffd640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] conn(0x7f26d00ac220 0x7f26d00ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:31.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 --> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26d00aed90 con 0x7f26d00ac220 2024-09-13T17:25:31.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26dc0cd920 con 0x7f26e410c6a0 2024-09-13T17:25:31.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26e9c02640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26d00a1190 0x7f26d00a3650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009b40 tx=0x7f26e0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:31.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.995+0000 7f26e9401640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] conn(0x7f26d00ac220 0x7f26d00ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:31.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.996+0000 7f26e9401640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] conn(0x7f26d00ac220 0x7f26d00ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:31.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:30.997+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== osd.5 v2:172.21.15.100:6808/2555541619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f26d00aed90 con 0x7f26d00ac220 2024-09-13T17:25:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.010+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 --> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26e4069000 con 0x7f26d00ac220 2024-09-13T17:25:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26d6ffd640 1 -- 172.21.15.44:0/3909200481 <== osd.5 v2:172.21.15.100:6808/2555541619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f26e4069000 con 0x7f26d00ac220 2024-09-13T17:25:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] conn(0x7f26d00ac220 msgr2=0x7f26d00ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ebe66640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] conn(0x7f26d00ac220 0x7f26d00ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26d00a1190 msgr2=0x7f26d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ebe66640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f26d00a1190 0x7f26d00a3650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009b40 tx=0x7f26e0005e50 comp rx=0 tx=0).stop 2024-09-13T17:25:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e410c6a0 msgr2=0x7f26e419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:31.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ebe66640 1 --2- 172.21.15.44:0/3909200481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e410c6a0 0x7f26e419cdf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00eb10 tx=0x7f26dc00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:25:31.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.011+0000 7f26ea403640 1 -- 172.21.15.44:0/3909200481 reap_dead start 2024-09-13T17:25:31.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.012+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 shutdown_connections 2024-09-13T17:25:31.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.012+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 >> 172.21.15.44:0/3909200481 conn(0x7f26e40fa9a0 msgr2=0x7f26e40fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:31.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.012+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 shutdown_connections 2024-09-13T17:25:31.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:31.012+0000 7f26ebe66640 1 -- 172.21.15.44:0/3909200481 wait complete. 2024-09-13T17:25:31.190 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:31 smithi044 ceph-mon[31175]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-13T17:25:31.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:31 smithi070 ceph-mon[34502]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-13T17:25:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:31 smithi100 ceph-mon[35055]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-13T17:25:31.835 INFO:teuthology.orchestra.run.smithi044.stdout:137438953531 2024-09-13T17:25:31.835 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-13T17:25:33.072 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:33 smithi044 ceph-mon[31175]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-13T17:25:33.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:33 smithi070 ceph-mon[34502]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-13T17:25:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:33 smithi100 ceph-mon[35055]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-13T17:25:35.082 INFO:teuthology.orchestra.run.smithi044.stdout:158913790008 2024-09-13T17:25:35.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-13T17:25:35.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:35 smithi100 ceph-mon[35055]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:25:35.293 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:35 smithi044 ceph-mon[31175]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:25:35.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:35 smithi070 ceph-mon[34502]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:25:35.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 -- 172.21.15.44:0/2995467787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281810c670 msgr2=0x7f281810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2995467787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281810c670 0x7f281810ea60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2810009ca0 tx=0x7f281002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 -- 172.21.15.44:0/2995467787 shutdown_connections 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2995467787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281810c670 0x7f281810ea60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2995467787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28180ff4b0 0x7f281810c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2995467787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28180feb90 0x7f28180fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 -- 172.21.15.44:0/2995467787 >> 172.21.15.44:0/2995467787 conn(0x7f2818074130 msgr2=0x7f2818074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 -- 172.21.15.44:0/2995467787 shutdown_connections 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 -- 172.21.15.44:0/2995467787 wait complete. 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 Processor -- start 2024-09-13T17:25:35.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.893+0000 7f281f4bd640 1 -- start start 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 0x7f2818198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28180ff4b0 0x7f2818198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f281810c670 0x7f281819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281819d440 con 0x7f28180feb90 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281819d5b0 con 0x7f281810c670 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281819d720 con 0x7f28180ff4b0 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 0x7f2818198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 0x7f2818198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51206/0 (socket says 172.21.15.44:51206) 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 -- 172.21.15.44:0/2119348029 learned_addr learned my addr 172.21.15.44:0/2119348029 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281ca58640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28180ff4b0 0x7f2818198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281da5a640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f281810c670 0x7f281819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 -- 172.21.15.44:0/2119348029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28180ff4b0 msgr2=0x7f2818198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28180ff4b0 0x7f2818198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 -- 172.21.15.44:0/2119348029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f281810c670 msgr2=0x7f281819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f281810c670 0x7f281819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 -- 172.21.15.44:0/2119348029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810009900 con 0x7f28180feb90 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281da5a640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f281810c670 0x7f281819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281d259640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 0x7f2818198500 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f281400b4a0 tx=0x7f281400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814011830 con 0x7f28180feb90 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281819da00 con 0x7f28180feb90 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.894+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2814011e20 con 0x7f28180feb90 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.895+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814010e30 con 0x7f28180feb90 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.895+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28181a52c0 con 0x7f28180feb90 2024-09-13T17:25:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.895+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28180680e0 con 0x7f28180feb90 2024-09-13T17:25:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.897+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2814011990 con 0x7f28180feb90 2024-09-13T17:25:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.897+0000 7f280e7fc640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28040a1110 0x7f28040a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.897+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28140cc870 con 0x7f28180feb90 2024-09-13T17:25:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.897+0000 7f281ca58640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28040a1110 0x7f28040a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.898+0000 7f281ca58640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28040a1110 0x7f28040a35d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2818199ab0 tx=0x7f2808009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:35.901+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2814095ca0 con 0x7f28180feb90 2024-09-13T17:25:36.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.075+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f28180feff0 con 0x7f28180feb90 2024-09-13T17:25:36.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.076+0000 7f280e7fc640 1 -- 172.21.15.44:0/2119348029 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28140992c0 con 0x7f28180feb90 2024-09-13T17:25:36.077 INFO:teuthology.orchestra.run.smithi044.stdout:249108103210 2024-09-13T17:25:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28040a1110 msgr2=0x7f28040a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28040a1110 0x7f28040a35d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2818199ab0 tx=0x7f2808009110 comp rx=0 tx=0).stop 2024-09-13T17:25:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 msgr2=0x7f2818198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 0x7f2818198500 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f281400b4a0 tx=0x7f281400b8a0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 shutdown_connections 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f28040a1110 0x7f28040a35d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28180ff4b0 0x7f2818198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f281810c670 0x7f281819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 --2- 172.21.15.44:0/2119348029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28180feb90 0x7f2818198500 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 >> 172.21.15.44:0/2119348029 conn(0x7f2818074130 msgr2=0x7f281810cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 shutdown_connections 2024-09-13T17:25:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.079+0000 7f281f4bd640 1 -- 172.21.15.44:0/2119348029 wait complete. 2024-09-13T17:25:36.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 -- 172.21.15.44:0/1992612004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200ff880 msgr2=0x7fbc201038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 --2- 172.21.15.44:0/1992612004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200ff880 0x7fbc201038d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009900 tx=0x7fbc1402a410 comp rx=0 tx=0).stop 2024-09-13T17:25:36.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 -- 172.21.15.44:0/1992612004 shutdown_connections 2024-09-13T17:25:36.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 --2- 172.21.15.44:0/1992612004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc20104000 0x7fbc20067f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 --2- 172.21.15.44:0/1992612004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200ff880 0x7fbc201038d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 --2- 172.21.15.44:0/1992612004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc200feed0 0x7fbc200ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 -- 172.21.15.44:0/1992612004 >> 172.21.15.44:0/1992612004 conn(0x7fbc200fa9b0 msgr2=0x7fbc200fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 -- 172.21.15.44:0/1992612004 shutdown_connections 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.099+0000 7fbc2571d640 1 -- 172.21.15.44:0/1992612004 wait complete. 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 Processor -- start 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 -- start start 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 0x7fbc20198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc200ff880 0x7fbc201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc20104000 0x7fbc2019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2019d2d0 con 0x7fbc200feed0 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2019d440 con 0x7fbc20104000 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc2571d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2019d5b0 con 0x7fbc200ff880 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc1effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 0x7fbc20198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:36.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc1effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 0x7fbc20198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51222/0 (socket says 172.21.15.44:51222) 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc1effd640 1 -- 172.21.15.44:0/2105505277 learned_addr learned my addr 172.21.15.44:0/2105505277 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc1f7fe640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc20104000 0x7fbc2019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.100+0000 7fbc1e7fc640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc200ff880 0x7fbc201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1effd640 1 -- 172.21.15.44:0/2105505277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc200ff880 msgr2=0x7fbc201988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1effd640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc200ff880 0x7fbc201988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1effd640 1 -- 172.21.15.44:0/2105505277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc20104000 msgr2=0x7fbc2019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1effd640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc20104000 0x7fbc2019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1effd640 1 -- 172.21.15.44:0/2105505277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009560 con 0x7fbc200feed0 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1e7fc640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc200ff880 0x7fbc201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc1effd640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 0x7fbc20198390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000d950 tx=0x7fbc1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10009c30 con 0x7fbc200feed0 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc20107bd0 con 0x7fbc200feed0 2024-09-13T17:25:36.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc10009d90 con 0x7fbc200feed0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000f620 con 0x7fbc200feed0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.101+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc201a4f80 con 0x7fbc200feed0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.102+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc201003b0 con 0x7fbc200feed0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.103+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbc100109f0 con 0x7fbc200feed0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.104+0000 7fbc07fff640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbc080a1110 0x7fbc080a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.104+0000 7fbc1e7fc640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbc080a1110 0x7fbc080a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.104+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbc100cc950 con 0x7fbc200feed0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.105+0000 7fbc1e7fc640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbc080a1110 0x7fbc080a35d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc14005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.111+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbc10095c70 con 0x7fbc200feed0 2024-09-13T17:25:36.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.285+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fbc201996f0 con 0x7fbc200feed0 2024-09-13T17:25:36.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.285+0000 7fbc07fff640 1 -- 172.21.15.44:0/2105505277 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fbc10020090 con 0x7fbc200feed0 2024-09-13T17:25:36.287 INFO:teuthology.orchestra.run.smithi044.stdout:339302416411 2024-09-13T17:25:36.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbc080a1110 msgr2=0x7fbc080a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbc080a1110 0x7fbc080a35d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc14005e00 comp rx=0 tx=0).stop 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 msgr2=0x7fbc20198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 0x7fbc20198390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000d950 tx=0x7fbc1000de10 comp rx=0 tx=0).stop 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 shutdown_connections 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbc080a1110 0x7fbc080a35d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc200ff880 0x7fbc201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc20104000 0x7fbc2019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.287+0000 7fbc2571d640 1 --2- 172.21.15.44:0/2105505277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc200feed0 0x7fbc20198390 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.288+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 >> 172.21.15.44:0/2105505277 conn(0x7fbc200fa9b0 msgr2=0x7fbc200fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.288+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 shutdown_connections 2024-09-13T17:25:36.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:36.288+0000 7fbc2571d640 1 -- 172.21.15.44:0/2105505277 wait complete. 2024-09-13T17:25:36.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416407 got 339302416411 for osd.11 2024-09-13T17:25:36.892 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:37.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:37 smithi044 ceph-mon[31175]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:25:37.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2119348029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T17:25:37.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2105505277' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T17:25:37.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103207 got 249108103210 for osd.8 2024-09-13T17:25:37.282 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:37.425 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:37.427 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:37.427 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:37.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:37 smithi070 ceph-mon[34502]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:25:37.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2119348029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T17:25:37.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2105505277' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T17:25:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:37 smithi100 ceph-mon[35055]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:25:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2119348029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T17:25:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2105505277' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T17:25:38.175 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:39.191 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:39 smithi044 ceph-mon[31175]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:25:39.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:39 smithi070 ceph-mon[34502]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:25:39.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:39 smithi100 ceph-mon[35055]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:25:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 -- 172.21.15.44:0/2131886104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae410a370 msgr2=0x7faae410c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 --2- 172.21.15.44:0/2131886104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae410a370 0x7faae410c7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faadc009ca0 tx=0x7faadc02a950 comp rx=0 tx=0).stop 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 -- 172.21.15.44:0/2131886104 shutdown_connections 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 --2- 172.21.15.44:0/2131886104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae410a370 0x7faae410c7f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 --2- 172.21.15.44:0/2131886104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae41018b0 0x7faae4109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 --2- 172.21.15.44:0/2131886104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae4101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 -- 172.21.15.44:0/2131886104 >> 172.21.15.44:0/2131886104 conn(0x7faae40fa9b0 msgr2=0x7faae40fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 -- 172.21.15.44:0/2131886104 shutdown_connections 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.081+0000 7faaebc69640 1 -- 172.21.15.44:0/2131886104 wait complete. 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 Processor -- start 2024-09-13T17:25:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 -- start start 2024-09-13T17:25:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae40febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae41018b0 0x7faae40ff130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae410a370 0x7faae40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae41a44f0 con 0x7faae4100f90 2024-09-13T17:25:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae41a4660 con 0x7faae410a370 2024-09-13T17:25:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.082+0000 7faaebc69640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae41a4820 con 0x7faae41018b0 2024-09-13T17:25:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae40febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faaea206640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae410a370 0x7faae40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae40febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32938/0 (socket says 172.21.15.44:32938) 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 -- 172.21.15.44:0/4117927895 learned_addr learned my addr 172.21.15.44:0/4117927895 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9204640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae41018b0 0x7faae40ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 -- 172.21.15.44:0/4117927895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae41018b0 msgr2=0x7faae40ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae41018b0 0x7faae40ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 -- 172.21.15.44:0/4117927895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae410a370 msgr2=0x7faae40ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae410a370 0x7faae40ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 -- 172.21.15.44:0/4117927895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009490 con 0x7faae4100f90 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faaea206640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae410a370 0x7faae40ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.083+0000 7faae9a05640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae40febf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faae000ea60 tx=0x7faae000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.084+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000cde0 con 0x7faae4100f90 2024-09-13T17:25:41.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.084+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faae00044b0 con 0x7faae4100f90 2024-09-13T17:25:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.084+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc009900 con 0x7faae4100f90 2024-09-13T17:25:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.084+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0010430 con 0x7faae4100f90 2024-09-13T17:25:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.084+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae41a4e70 con 0x7faae4100f90 2024-09-13T17:25:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.085+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae4068780 con 0x7faae4100f90 2024-09-13T17:25:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.086+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faae0004000 con 0x7faae4100f90 2024-09-13T17:25:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.087+0000 7faadaffd640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faad00a1110 0x7faad00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:41.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.087+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faae0014070 con 0x7faae4100f90 2024-09-13T17:25:41.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.087+0000 7faae9204640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faad00a1110 0x7faad00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:41.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.087+0000 7faae9204640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faad00a1110 0x7faad00a35d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faae4100370 tx=0x7faad400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.090+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faae0096660 con 0x7faae4100f90 2024-09-13T17:25:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.270+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7faae4101370 con 0x7faae4100f90 2024-09-13T17:25:41.271 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:41 smithi044 ceph-mon[31175]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:25:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.270+0000 7faadaffd640 1 -- 172.21.15.44:0/4117927895 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faae0099c80 con 0x7faae4100f90 2024-09-13T17:25:41.272 INFO:teuthology.orchestra.run.smithi044.stdout:283467841573 2024-09-13T17:25:41.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faad00a1110 msgr2=0x7faad00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faad00a1110 0x7faad00a35d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faae4100370 tx=0x7faad400a250 comp rx=0 tx=0).stop 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 msgr2=0x7faae40febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae40febf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faae000ea60 tx=0x7faae000ef20 comp rx=0 tx=0).stop 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 shutdown_connections 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7faad00a1110 0x7faad00a35d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae41018b0 0x7faae40ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faae410a370 0x7faae40ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 --2- 172.21.15.44:0/4117927895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae4100f90 0x7faae40febf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 >> 172.21.15.44:0/4117927895 conn(0x7faae40fa9b0 msgr2=0x7faae40fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 shutdown_connections 2024-09-13T17:25:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:41.272+0000 7faaebc69640 1 -- 172.21.15.44:0/4117927895 wait complete. 2024-09-13T17:25:41.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:41 smithi070 ceph-mon[34502]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:25:41.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:41 smithi100 ceph-mon[35055]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:25:42.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:42 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4117927895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T17:25:42.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-13T17:25:42.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:42 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-09-13T17:25:42.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:42 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4117927895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T17:25:42.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-13T17:25:42.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:42 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-09-13T17:25:42.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:42 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4117927895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T17:25:42.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-13T17:25:42.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:42 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-09-13T17:25:42.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841571 got 283467841573 for osd.9 2024-09-13T17:25:42.726 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:43.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:43 smithi044 ceph-mon[31175]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-13T17:25:43.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-13T17:25:43.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:43 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-09-13T17:25:43.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:43 smithi044 ceph-mon[31175]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T17:25:43.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 -- 172.21.15.44:0/322369225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c100f60 msgr2=0x7fc12c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 --2- 172.21.15.44:0/322369225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c100f60 0x7fc12c101340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009900 tx=0x7fc11c02a480 comp rx=0 tx=0).stop 2024-09-13T17:25:43.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 -- 172.21.15.44:0/322369225 shutdown_connections 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 --2- 172.21.15.44:0/322369225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c10a390 0x7fc12c10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 --2- 172.21.15.44:0/322369225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c101880 0x7fc12c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 --2- 172.21.15.44:0/322369225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c100f60 0x7fc12c101340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 -- 172.21.15.44:0/322369225 >> 172.21.15.44:0/322369225 conn(0x7fc12c0fa9a0 msgr2=0x7fc12c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 -- 172.21.15.44:0/322369225 shutdown_connections 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.246+0000 7fc132382640 1 -- 172.21.15.44:0/322369225 wait complete. 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 Processor -- start 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 -- start start 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c100f60 0x7fc12c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c101880 0x7fc12c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 0x7fc12c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19d400 con 0x7fc12c101880 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19d570 con 0x7fc12c10a390 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc132382640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19d6e0 con 0x7fc12c100f60 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.247+0000 7fc12b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c101880 0x7fc12c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 0x7fc12c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc12bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c100f60 0x7fc12c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 0x7fc12c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54920/0 (socket says 172.21.15.44:54920) 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 -- 172.21.15.44:0/2930064331 learned_addr learned my addr 172.21.15.44:0/2930064331 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 -- 172.21.15.44:0/2930064331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c100f60 msgr2=0x7fc12c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c100f60 0x7fc12c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 -- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c101880 msgr2=0x7fc12c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c101880 0x7fc12c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 -- 172.21.15.44:0/2930064331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c009560 con 0x7fc12c10a390 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc12b7fe640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c101880 0x7fc12c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:25:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc13091f640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 0x7fc12c19ce00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc12400eaa0 tx=0x7fc12400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400cdb0 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c19d9c0 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc124005020 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c1a5280 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.248+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124011670 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.249+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c069000 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.251+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc124004a40 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.251+0000 7fc1297fa640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc1140a1110 0x7fc1140a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.251+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fc1240cce40 con 0x7fc12c10a390 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.251+0000 7fc12bfff640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc1140a1110 0x7fc1140a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.252+0000 7fc12bfff640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc1140a1110 0x7fc1140a35d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc11c02aa70 tx=0x7fc11c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.255+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc124096180 con 0x7fc12c10a390 2024-09-13T17:25:43.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.437+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc12c101340 con 0x7fc12c10a390 2024-09-13T17:25:43.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.438+0000 7fc1297fa640 1 -- 172.21.15.44:0/2930064331 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc1240997a0 con 0x7fc12c10a390 2024-09-13T17:25:43.439 INFO:teuthology.orchestra.run.smithi044.stdout:304942678050 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.439+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc1140a1110 msgr2=0x7fc1140a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc1140a1110 0x7fc1140a35d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc11c02aa70 tx=0x7fc11c005a20 comp rx=0 tx=0).stop 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 msgr2=0x7fc12c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 0x7fc12c19ce00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc12400eaa0 tx=0x7fc12400ef60 comp rx=0 tx=0).stop 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 shutdown_connections 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc1140a1110 0x7fc1140a35d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc12c10a390 0x7fc12c19ce00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c101880 0x7fc12c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 --2- 172.21.15.44:0/2930064331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c100f60 0x7fc12c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 >> 172.21.15.44:0/2930064331 conn(0x7fc12c0fa9a0 msgr2=0x7fc12c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 shutdown_connections 2024-09-13T17:25:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.440+0000 7fc132382640 1 -- 172.21.15.44:0/2930064331 wait complete. 2024-09-13T17:25:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:43 smithi070 ceph-mon[34502]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-13T17:25:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-13T17:25:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:43 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-09-13T17:25:43.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:43 smithi070 ceph-mon[34502]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T17:25:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:43 smithi100 ceph-mon[35055]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-13T17:25:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-13T17:25:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:43 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-09-13T17:25:43.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:43 smithi100 ceph-mon[35055]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T17:25:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 -- 172.21.15.44:0/2212928357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff144105110 msgr2=0x7ff14410cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 --2- 172.21.15.44:0/2212928357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff144105110 0x7ff14410cc90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009cd0 tx=0x7ff13c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 -- 172.21.15.44:0/2212928357 shutdown_connections 2024-09-13T17:25:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 --2- 172.21.15.44:0/2212928357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144100990 0x7ff1441049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 --2- 172.21.15.44:0/2212928357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff144105110 0x7ff14410cc90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 --2- 172.21.15.44:0/2212928357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1440fffe0 0x7ff1441003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 -- 172.21.15.44:0/2212928357 >> 172.21.15.44:0/2212928357 conn(0x7ff1440fbae0 msgr2=0x7ff1440fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 -- 172.21.15.44:0/2212928357 shutdown_connections 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.551+0000 7ff149ee6640 1 -- 172.21.15.44:0/2212928357 wait complete. 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 Processor -- start 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 -- start start 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 0x7ff144196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff144100990 0x7ff144196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144105110 0x7ff14419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14419b140 con 0x7ff1440fffe0 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14419b2b0 con 0x7ff144100990 2024-09-13T17:25:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff149ee6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14419b420 con 0x7ff144105110 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff1437fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 0x7ff144196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff1437fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 0x7ff144196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32964/0 (socket says 172.21.15.44:32964) 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff1437fe640 1 -- 172.21.15.44:0/1828433472 learned_addr learned my addr 172.21.15.44:0/1828433472 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff142ffd640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff144100990 0x7ff144196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff1437fe640 1 -- 172.21.15.44:0/1828433472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144105110 msgr2=0x7ff14419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.552+0000 7ff1437fe640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144105110 0x7ff14419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff1437fe640 1 -- 172.21.15.44:0/1828433472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff144100990 msgr2=0x7ff144196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff1437fe640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff144100990 0x7ff144196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff1437fe640 1 -- 172.21.15.44:0/1828433472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c009900 con 0x7ff1440fffe0 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff142ffd640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff144100990 0x7ff144196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff1437fe640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 0x7ff144196200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134009700 con 0x7ff1440fffe0 2024-09-13T17:25:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff13400cd80 con 0x7ff1440fffe0 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14419b700 con 0x7ff1440fffe0 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1340187e0 con 0x7ff1440fffe0 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.553+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1441a2fc0 con 0x7ff1440fffe0 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.554+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1441014c0 con 0x7ff1440fffe0 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.555+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff134010430 con 0x7ff1440fffe0 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.556+0000 7ff140ff9640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff12c0a1190 0x7ff12c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stdout:55834574924 2024-09-13T17:25:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.556+0000 7ff142ffd640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff12c0a1190 0x7ff12c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.556+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7ff134014070 con 0x7ff1440fffe0 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.557+0000 7ff142ffd640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff12c0a1190 0x7ff12c0a3650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1441977b0 tx=0x7ff138005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.560+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff134052e20 con 0x7ff1440fffe0 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.732+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff144197560 con 0x7ff1440fffe0 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.733+0000 7ff140ff9640 1 -- 172.21.15.44:0/1828433472 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1340964b0 con 0x7ff1440fffe0 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff12c0a1190 msgr2=0x7ff12c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff12c0a1190 0x7ff12c0a3650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1441977b0 tx=0x7ff138005cb0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 msgr2=0x7ff144196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 0x7ff144196200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 shutdown_connections 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff12c0a1190 0x7ff12c0a3650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144105110 0x7ff14419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff144100990 0x7ff144196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 --2- 172.21.15.44:0/1828433472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1440fffe0 0x7ff144196200 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 >> 172.21.15.44:0/1828433472 conn(0x7ff1440fbae0 msgr2=0x7ff1440fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 shutdown_connections 2024-09-13T17:25:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:43.735+0000 7ff149ee6640 1 -- 172.21.15.44:0/1828433472 wait complete. 2024-09-13T17:25:44.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/268767619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 msgr2=0x7fcf680fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/268767619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf680fef70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009900 tx=0x7fcf5802a410 comp rx=0 tx=0).stop 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/268767619 shutdown_connections 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/268767619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf6810a3a0 0x7fcf6810c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/268767619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf680ff540 0x7fcf68109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/268767619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf680fef70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/268767619 >> 172.21.15.44:0/268767619 conn(0x7fcf68074130 msgr2=0x7fcf68074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/268767619 shutdown_connections 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.283+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/268767619 wait complete. 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 Processor -- start 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 -- start start 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf68196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:44.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680ff540 0x7fcf68196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6810a3a0 0x7fcf6819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819b140 con 0x7fcf680feb90 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819b2b0 con 0x7fcf6810a3a0 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6d0b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819b420 con 0x7fcf680ff540 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf66d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf68196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf66d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf68196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32982/0 (socket says 172.21.15.44:32982) 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf66d9d640 1 -- 172.21.15.44:0/1180340852 learned_addr learned my addr 172.21.15.44:0/1180340852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6659c640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680ff540 0x7fcf68196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.284+0000 7fcf6759e640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6810a3a0 0x7fcf6819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf66d9d640 1 -- 172.21.15.44:0/1180340852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680ff540 msgr2=0x7fcf68196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf66d9d640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680ff540 0x7fcf68196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf66d9d640 1 -- 172.21.15.44:0/1180340852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6810a3a0 msgr2=0x7fcf6819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf66d9d640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6810a3a0 0x7fcf6819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf66d9d640 1 -- 172.21.15.44:0/1180340852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009560 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf6759e640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6810a3a0 0x7fcf6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf66d9d640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf68196200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcf580057c0 tx=0x7fcf58032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58032e10 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf580046c0 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6819b6a0 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58003850 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.285+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf681a2df0 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.286+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680680a0 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.287+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcf58005210 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.288+0000 7fcf4ffff640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcf500a1110 0x7fcf500a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.288+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fcf580edd20 con 0x7fcf680feb90 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.288+0000 7fcf6659c640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcf500a1110 0x7fcf500a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.288+0000 7fcf6659c640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcf500a1110 0x7fcf500a35d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf681977b0 tx=0x7fcf5c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:44.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.292+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcf580f2070 con 0x7fcf680feb90 2024-09-13T17:25:44.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:44 smithi044 ceph-mon[31175]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T17:25:44.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2930064331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T17:25:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1828433472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T17:25:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:44 smithi070 ceph-mon[34502]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T17:25:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2930064331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T17:25:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1828433472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T17:25:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.467+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcf68197560 con 0x7fcf680feb90 2024-09-13T17:25:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.467+0000 7fcf4ffff640 1 -- 172.21.15.44:0/1180340852 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf5804a090 con 0x7fcf680feb90 2024-09-13T17:25:44.468 INFO:teuthology.orchestra.run.smithi044.stdout:103079215171 2024-09-13T17:25:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcf500a1110 msgr2=0x7fcf500a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcf500a1110 0x7fcf500a35d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf681977b0 tx=0x7fcf5c00b390 comp rx=0 tx=0).stop 2024-09-13T17:25:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 msgr2=0x7fcf68196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf68196200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcf580057c0 tx=0x7fcf58032c20 comp rx=0 tx=0).stop 2024-09-13T17:25:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 shutdown_connections 2024-09-13T17:25:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcf500a1110 0x7fcf500a35d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680ff540 0x7fcf68196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6810a3a0 0x7fcf6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 --2- 172.21.15.44:0/1180340852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf680feb90 0x7fcf68196200 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:44.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 >> 172.21.15.44:0/1180340852 conn(0x7fcf68074130 msgr2=0x7fcf680fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:44.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 shutdown_connections 2024-09-13T17:25:44.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:44.469+0000 7fcf6d0b0640 1 -- 172.21.15.44:0/1180340852 wait complete. 2024-09-13T17:25:44.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:44 smithi100 ceph-mon[35055]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T17:25:44.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2930064331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T17:25:44.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1828433472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T17:25:44.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574924 for osd.1 2024-09-13T17:25:44.743 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:45.166 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:45.188 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678050 for osd.10 2024-09-13T17:25:45.189 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:45.212 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:45.212 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:45.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:45 smithi044 ceph-mon[31175]: pgmap v323: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1/194 objects degraded (0.515%) 2024-09-13T17:25:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:45 smithi044 ceph-mon[31175]: Health check failed: Degraded data redundancy: 1/194 objects degraded (0.515%), 1 pg degraded (PG_DEGRADED) 2024-09-13T17:25:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:45 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1180340852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T17:25:45.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:45 smithi070 ceph-mon[34502]: pgmap v323: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1/194 objects degraded (0.515%) 2024-09-13T17:25:45.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:45 smithi070 ceph-mon[34502]: Health check failed: Degraded data redundancy: 1/194 objects degraded (0.515%), 1 pg degraded (PG_DEGRADED) 2024-09-13T17:25:45.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:45 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1180340852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T17:25:45.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:45 smithi100 ceph-mon[35055]: pgmap v323: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1/194 objects degraded (0.515%) 2024-09-13T17:25:45.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:45 smithi100 ceph-mon[35055]: Health check failed: Degraded data redundancy: 1/194 objects degraded (0.515%), 1 pg degraded (PG_DEGRADED) 2024-09-13T17:25:45.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:45 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1180340852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T17:25:45.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215164 got 103079215171 for osd.3 2024-09-13T17:25:45.844 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:46.463 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:47.185 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:47 smithi044 ceph-mon[31175]: pgmap v324: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 1/194 objects degraded (0.515%); 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:47.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:47 smithi070 ceph-mon[34502]: pgmap v324: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 1/194 objects degraded (0.515%); 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:47.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:47 smithi100 ceph-mon[35055]: pgmap v324: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 1/194 objects degraded (0.515%); 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 -- 172.21.15.44:0/421250099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44100990 msgr2=0x7fbb441049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 --2- 172.21.15.44:0/421250099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44100990 0x7fbb441049e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009930 tx=0x7fbb3402a420 comp rx=0 tx=0).stop 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 -- 172.21.15.44:0/421250099 shutdown_connections 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 --2- 172.21.15.44:0/421250099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb44105110 0x7fbb4410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 --2- 172.21.15.44:0/421250099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44100990 0x7fbb441049e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 --2- 172.21.15.44:0/421250099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb440fffe0 0x7fbb441003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 -- 172.21.15.44:0/421250099 >> 172.21.15.44:0/421250099 conn(0x7fbb440fbae0 msgr2=0x7fbb440fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 -- 172.21.15.44:0/421250099 shutdown_connections 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 -- 172.21.15.44:0/421250099 wait complete. 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 Processor -- start 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.985+0000 7fbb4b595640 1 -- start start 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb440fffe0 0x7fbb44196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb44100990 0x7fbb44196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 0x7fbb4419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4419b180 con 0x7fbb44105110 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4419b2f0 con 0x7fbb440fffe0 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4419b460 con 0x7fbb44100990 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 0x7fbb4419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 0x7fbb4419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33008/0 (socket says 172.21.15.44:33008) 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 -- 172.21.15.44:0/2051924046 learned_addr learned my addr 172.21.15.44:0/2051924046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb48b30640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb44100990 0x7fbb44196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49331640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb440fffe0 0x7fbb44196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 -- 172.21.15.44:0/2051924046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb44100990 msgr2=0x7fbb44196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb44100990 0x7fbb44196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 -- 172.21.15.44:0/2051924046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb440fffe0 msgr2=0x7fbb44196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb440fffe0 0x7fbb44196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 -- 172.21.15.44:0/2051924046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009560 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb49b32640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 0x7fbb4419ab80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b970 tx=0x7fbb3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00cb40 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb3c00cca0 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.986+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4419b740 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.987+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c007610 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.987+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb441a3000 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.987+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb441014c0 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.989+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbb3c014440 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.989+0000 7fbb3a7fc640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbb300a1190 0x7fbb300a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.990+0000 7fbb49331640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbb300a1190 0x7fbb300a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.990+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fbb3c0cca40 con 0x7fbb44105110 2024-09-13T17:25:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.990+0000 7fbb49331640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbb300a1190 0x7fbb300a3650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb400058c0 tx=0x7fbb4000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:47.992+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbb3c095e40 con 0x7fbb44105110 2024-09-13T17:25:48.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.166+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbb441003c0 con 0x7fbb44105110 2024-09-13T17:25:48.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.167+0000 7fbb3a7fc640 1 -- 172.21.15.44:0/2051924046 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb3c019090 con 0x7fbb44105110 2024-09-13T17:25:48.168 INFO:teuthology.orchestra.run.smithi044.stdout:214748364849 2024-09-13T17:25:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbb300a1190 msgr2=0x7fbb300a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbb300a1190 0x7fbb300a3650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb400058c0 tx=0x7fbb4000a4c0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 msgr2=0x7fbb4419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 0x7fbb4419ab80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b970 tx=0x7fbb3c00be30 comp rx=0 tx=0).stop 2024-09-13T17:25:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 shutdown_connections 2024-09-13T17:25:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fbb300a1190 0x7fbb300a3650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb44100990 0x7fbb44196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44105110 0x7fbb4419ab80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 --2- 172.21.15.44:0/2051924046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb440fffe0 0x7fbb44196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 >> 172.21.15.44:0/2051924046 conn(0x7fbb440fbae0 msgr2=0x7fbb440fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 shutdown_connections 2024-09-13T17:25:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:48.169+0000 7fbb4b595640 1 -- 172.21.15.44:0/2051924046 wait complete. 2024-09-13T17:25:49.352 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:49 smithi044 ceph-mon[31175]: pgmap v325: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 1/194 objects degraded (0.515%); 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:49.352 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:49 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2051924046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T17:25:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:49 smithi070 ceph-mon[34502]: pgmap v325: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 1/194 objects degraded (0.515%); 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:49 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2051924046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T17:25:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:49 smithi100 ceph-mon[35055]: pgmap v325: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 1/194 objects degraded (0.515%); 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:49 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2051924046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T17:25:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:50 smithi070 ceph-mon[34502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/194 objects degraded (0.515%), 1 pg degraded) 2024-09-13T17:25:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:50 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:25:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:50 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:50.458 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:50 smithi044 ceph-mon[31175]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/194 objects degraded (0.515%), 1 pg degraded) 2024-09-13T17:25:50.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:50 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:25:50.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:50.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:50 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:50 smithi100 ceph-mon[35055]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/194 objects degraded (0.515%), 1 pg degraded) 2024-09-13T17:25:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:50 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:25:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:50 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:51.405 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:51 smithi044 ceph-mon[31175]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:51 smithi070 ceph-mon[34502]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:51 smithi100 ceph-mon[35055]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:52.349 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:25:52.349 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:25:52.350 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:52 smithi044 ceph-mon[31175]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:25:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:25:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:52 smithi070 ceph-mon[34502]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:25:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:25:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:52 smithi100 ceph-mon[35055]: from='mgr.14150 172.21.15.44:0/4022229024' entity='mgr.x' 2024-09-13T17:25:52.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 -- 172.21.15.44:0/2681257027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340ff8b0 msgr2=0x7fda34103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 --2- 172.21.15.44:0/2681257027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340ff8b0 0x7fda34103900 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009930 tx=0x7fda1c02a420 comp rx=0 tx=0).stop 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 -- 172.21.15.44:0/2681257027 shutdown_connections 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 --2- 172.21.15.44:0/2681257027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda34104030 0x7fda3410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 --2- 172.21.15.44:0/2681257027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340ff8b0 0x7fda34103900 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 --2- 172.21.15.44:0/2681257027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda340fef00 0x7fda340ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 -- 172.21.15.44:0/2681257027 >> 172.21.15.44:0/2681257027 conn(0x7fda340fa9a0 msgr2=0x7fda340fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 -- 172.21.15.44:0/2681257027 shutdown_connections 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.810+0000 7fda3a605640 1 -- 172.21.15.44:0/2681257027 wait complete. 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 Processor -- start 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 -- start start 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 0x7fda341961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:52.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda340ff8b0 0x7fda34196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda34104030 0x7fda3419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419b0d0 con 0x7fda340fef00 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419b240 con 0x7fda340ff8b0 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda3a605640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419b3b0 con 0x7fda34104030 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 0x7fda341961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 0x7fda341961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60116/0 (socket says 172.21.15.44:60116) 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 -- 172.21.15.44:0/468105662 learned_addr learned my addr 172.21.15.44:0/468105662 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda2bfff640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda340ff8b0 0x7fda34196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda38ba2640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda34104030 0x7fda3419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 -- 172.21.15.44:0/468105662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda34104030 msgr2=0x7fda3419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda34104030 0x7fda3419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 -- 172.21.15.44:0/468105662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda340ff8b0 msgr2=0x7fda34196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda340ff8b0 0x7fda34196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.811+0000 7fda33fff640 1 -- 172.21.15.44:0/468105662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009560 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.812+0000 7fda33fff640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 0x7fda341961e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.812+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24009700 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.812+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda2400cd80 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.812+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3419b690 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.812+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240187e0 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.812+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda341a2f50 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.813+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda341003e0 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.814+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fda24010430 con 0x7fda340fef00 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.815+0000 7fda31ffb640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda200a1190 0x7fda200a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:52.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.815+0000 7fda2bfff640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda200a1190 0x7fda200a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:52.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.815+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fda24014070 con 0x7fda340fef00 2024-09-13T17:25:52.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.815+0000 7fda2bfff640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda200a1190 0x7fda200a3650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0057c0 tx=0x7fda1c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:52.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.819+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fda24052e20 con 0x7fda340fef00 2024-09-13T17:25:52.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.993+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fda341975b0 con 0x7fda340fef00 2024-09-13T17:25:52.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.994+0000 7fda31ffb640 1 -- 172.21.15.44:0/468105662 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda240963e0 con 0x7fda340fef00 2024-09-13T17:25:52.995 INFO:teuthology.orchestra.run.smithi044.stdout:77309411402 2024-09-13T17:25:52.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda200a1190 msgr2=0x7fda200a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda200a1190 0x7fda200a3650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0057c0 tx=0x7fda1c005a20 comp rx=0 tx=0).stop 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 msgr2=0x7fda341961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 0x7fda341961e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 shutdown_connections 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda200a1190 0x7fda200a3650 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda34104030 0x7fda3419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda340ff8b0 0x7fda34196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 --2- 172.21.15.44:0/468105662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda340fef00 0x7fda341961e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 >> 172.21.15.44:0/468105662 conn(0x7fda340fa9a0 msgr2=0x7fda340fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:52.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 shutdown_connections 2024-09-13T17:25:52.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:52.996+0000 7fda3a605640 1 -- 172.21.15.44:0/468105662 wait complete. 2024-09-13T17:25:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:53 smithi100 ceph-mon[35055]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:53 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/468105662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T17:25:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:53 smithi044 ceph-mon[31175]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:53 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/468105662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T17:25:53.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411402 for osd.2 2024-09-13T17:25:53.895 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:53 smithi070 ceph-mon[34502]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T17:25:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:53 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/468105662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 -- 172.21.15.44:0/1250872518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 msgr2=0x7ff058101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 --2- 172.21.15.44:0/1250872518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff058101370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff048009900 tx=0x7ff04802a480 comp rx=0 tx=0).stop 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 -- 172.21.15.44:0/1250872518 shutdown_connections 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 --2- 172.21.15.44:0/1250872518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0581018b0 0x7ff058109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 --2- 172.21.15.44:0/1250872518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05810a370 0x7ff05810c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 --2- 172.21.15.44:0/1250872518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff058101370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 -- 172.21.15.44:0/1250872518 >> 172.21.15.44:0/1250872518 conn(0x7ff0580fa9b0 msgr2=0x7ff0580fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 -- 172.21.15.44:0/1250872518 shutdown_connections 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.148+0000 7ff05c827640 1 -- 172.21.15.44:0/1250872518 wait complete. 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 Processor -- start 2024-09-13T17:25:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 -- start start 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff0580febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0581018b0 0x7ff0580ff130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05810a370 0x7ff0580ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0581a44c0 con 0x7ff058100f90 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0581a4630 con 0x7ff05810a370 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05c827640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0581a47f0 con 0x7ff0581018b0 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff0580febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff0580febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60134/0 (socket says 172.21.15.44:60134) 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05659c640 1 -- 172.21.15.44:0/2647197041 learned_addr learned my addr 172.21.15.44:0/2647197041 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff055d9b640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0581018b0 0x7ff0580ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff056d9d640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05810a370 0x7ff0580ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05659c640 1 -- 172.21.15.44:0/2647197041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0581018b0 msgr2=0x7ff0580ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05659c640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0581018b0 0x7ff0580ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.149+0000 7ff05659c640 1 -- 172.21.15.44:0/2647197041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05810a370 msgr2=0x7ff0580ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff05659c640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05810a370 0x7ff0580ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff05659c640 1 -- 172.21.15.44:0/2647197041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009560 con 0x7ff058100f90 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff05659c640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff0580febf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff0480057c0 tx=0x7ff048032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048032e10 con 0x7ff058100f90 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0581a4a70 con 0x7ff058100f90 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0480046c0 con 0x7ff058100f90 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048003850 con 0x7ff058100f90 2024-09-13T17:25:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.150+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0581a4f60 con 0x7ff058100f90 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.151+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff058068780 con 0x7ff058100f90 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.152+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff048004aa0 con 0x7ff058100f90 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.153+0000 7ff04f7fe640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff0440a1110 0x7ff0440a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.153+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7ff0480ede50 con 0x7ff058100f90 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.153+0000 7ff055d9b640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff0440a1110 0x7ff0440a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.154+0000 7ff055d9b640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff0440a1110 0x7ff0440a35d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff058100370 tx=0x7ff040005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.156+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff0480b71f0 con 0x7ff058100f90 2024-09-13T17:25:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.336+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff058105410 con 0x7ff058100f90 2024-09-13T17:25:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.336+0000 7ff04f7fe640 1 -- 172.21.15.44:0/2647197041 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0480ba810 con 0x7ff058100f90 2024-09-13T17:25:54.337 INFO:teuthology.orchestra.run.smithi044.stdout:193273528375 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff0440a1110 msgr2=0x7ff0440a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff0440a1110 0x7ff0440a35d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff058100370 tx=0x7ff040005f90 comp rx=0 tx=0).stop 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 msgr2=0x7ff0580febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff0580febf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff0480057c0 tx=0x7ff048032c20 comp rx=0 tx=0).stop 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 shutdown_connections 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff0440a1110 0x7ff0440a35d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0581018b0 0x7ff0580ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05810a370 0x7ff0580ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 --2- 172.21.15.44:0/2647197041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff058100f90 0x7ff0580febf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.338+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 >> 172.21.15.44:0/2647197041 conn(0x7ff0580fa9b0 msgr2=0x7ff0580fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.339+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 shutdown_connections 2024-09-13T17:25:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.339+0000 7ff05c827640 1 -- 172.21.15.44:0/2647197041 wait complete. 2024-09-13T17:25:54.585 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364842 got 214748364849 for osd.7 2024-09-13T17:25:54.585 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:54.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:54 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2647197041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T17:25:54.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:54 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2647197041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T17:25:54.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:54 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2647197041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T17:25:54.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 -- 172.21.15.44:0/2245859022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028101880 msgr2=0x7fa028109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 --2- 172.21.15.44:0/2245859022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028101880 0x7fa028109d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa018009900 tx=0x7fa01802a480 comp rx=0 tx=0).stop 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 -- 172.21.15.44:0/2245859022 shutdown_connections 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 --2- 172.21.15.44:0/2245859022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa02810a390 0x7fa02810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 --2- 172.21.15.44:0/2245859022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028101880 0x7fa028109d70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 --2- 172.21.15.44:0/2245859022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa028100f60 0x7fa028101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 -- 172.21.15.44:0/2245859022 >> 172.21.15.44:0/2245859022 conn(0x7fa0280fa9a0 msgr2=0x7fa0280fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 -- 172.21.15.44:0/2245859022 shutdown_connections 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.959+0000 7fa030218640 1 -- 172.21.15.44:0/2245859022 wait complete. 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 Processor -- start 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 -- start start 2024-09-13T17:25:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 0x7fa0281961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa028101880 0x7fa028196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa02810a390 0x7fa02819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02819b130 con 0x7fa028100f60 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02819b2a0 con 0x7fa028101880 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa030218640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02819b410 con 0x7fa02810a390 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa02dfb4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 0x7fa0281961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa02dfb4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 0x7fa0281961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60156/0 (socket says 172.21.15.44:60156) 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa02dfb4640 1 -- 172.21.15.44:0/836016003 learned_addr learned my addr 172.21.15.44:0/836016003 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa02d7b3640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa028101880 0x7fa028196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.960+0000 7fa02e7b5640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa02810a390 0x7fa02819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa02dfb4640 1 -- 172.21.15.44:0/836016003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa02810a390 msgr2=0x7fa02819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa02dfb4640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa02810a390 0x7fa02819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa02dfb4640 1 -- 172.21.15.44:0/836016003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa028101880 msgr2=0x7fa028196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa02dfb4640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa028101880 0x7fa028196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa02dfb4640 1 -- 172.21.15.44:0/836016003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018009560 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa02dfb4640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 0x7fa0281961f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa02400d770 tx=0x7fa02400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024009c30 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa02400de80 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024004d40 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02819b6f0 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.961+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02819b9a0 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.962+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa028069000 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.963+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa024004ea0 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.964+0000 7fa01effd640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa0140a1190 0x7fa0140a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.964+0000 7fa02d7b3640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa0140a1190 0x7fa0140a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.964+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fa0240cc5c0 con 0x7fa028100f60 2024-09-13T17:25:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.965+0000 7fa02d7b3640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa0140a1190 0x7fa0140a3650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa01802aa70 tx=0x7fa018005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:54.969+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa024095900 con 0x7fa028100f60 2024-09-13T17:25:55.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.141+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa028101340 con 0x7fa028100f60 2024-09-13T17:25:55.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.141+0000 7fa01effd640 1 -- 172.21.15.44:0/836016003 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa024098f20 con 0x7fa028100f60 2024-09-13T17:25:55.142 INFO:teuthology.orchestra.run.smithi044.stdout:158913790012 2024-09-13T17:25:55.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa0140a1190 msgr2=0x7fa0140a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:55.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa0140a1190 0x7fa0140a3650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa01802aa70 tx=0x7fa018005a20 comp rx=0 tx=0).stop 2024-09-13T17:25:55.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 msgr2=0x7fa0281961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:55.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 0x7fa0281961f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa02400d770 tx=0x7fa02400dc30 comp rx=0 tx=0).stop 2024-09-13T17:25:55.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 shutdown_connections 2024-09-13T17:25:55.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa0140a1190 0x7fa0140a3650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:55.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa02810a390 0x7fa02819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:55.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa028101880 0x7fa028196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:55.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 --2- 172.21.15.44:0/836016003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa028100f60 0x7fa0281961f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:55.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 >> 172.21.15.44:0/836016003 conn(0x7fa0280fa9a0 msgr2=0x7fa0280fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:55.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 shutdown_connections 2024-09-13T17:25:55.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:55.143+0000 7fa030218640 1 -- 172.21.15.44:0/836016003 wait complete. 2024-09-13T17:25:55.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:55 smithi100 ceph-mon[35055]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-13T17:25:55.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:55 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/836016003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T17:25:55.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:55 smithi044 ceph-mon[31175]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-13T17:25:55.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:55 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/836016003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T17:25:55.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790008 got 158913790012 for osd.5 2024-09-13T17:25:55.832 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:55 smithi070 ceph-mon[34502]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-13T17:25:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:55 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/836016003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T17:25:56.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528371 got 193273528375 for osd.6 2024-09-13T17:25:56.196 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:56.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 -- 172.21.15.44:0/2695849246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 msgr2=0x7f5fbc10cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/2695849246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc10cc90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009930 tx=0x7f5fb402a420 comp rx=0 tx=0).stop 2024-09-13T17:25:56.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 -- 172.21.15.44:0/2695849246 shutdown_connections 2024-09-13T17:25:56.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/2695849246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc10cc90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/2695849246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fbc100990 0x7f5fbc1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/2695849246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fbc0fffe0 0x7f5fbc1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 -- 172.21.15.44:0/2695849246 >> 172.21.15.44:0/2695849246 conn(0x7f5fbc0fbae0 msgr2=0x7f5fbc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:56.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 -- 172.21.15.44:0/2695849246 shutdown_connections 2024-09-13T17:25:56.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.387+0000 7f5fc17ba640 1 -- 172.21.15.44:0/2695849246 wait complete. 2024-09-13T17:25:56.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 Processor -- start 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 -- start start 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fbc0fffe0 0x7f5fbc0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fbc100990 0x7f5fbc072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1a4570 con 0x7f5fbc105110 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1a46e0 con 0x7f5fbc100990 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.388+0000 7f5fc17ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1a48a0 con 0x7f5fbc0fffe0 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60180/0 (socket says 172.21.15.44:60180) 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 -- 172.21.15.44:0/1817311100 learned_addr learned my addr 172.21.15.44:0/1817311100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fba7fc640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fbc100990 0x7f5fbc072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbaffd640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fbc0fffe0 0x7f5fbc0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 -- 172.21.15.44:0/1817311100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fbc0fffe0 msgr2=0x7f5fbc0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fbc0fffe0 0x7f5fbc0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 -- 172.21.15.44:0/1817311100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fbc100990 msgr2=0x7f5fbc072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fbc100990 0x7f5fbc072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 -- 172.21.15.44:0/1817311100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4009560 con 0x7f5fbc105110 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fbb7fe640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc073230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009900 tx=0x7f5fb4032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40044f0 con 0x7f5fbc105110 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fb4032e10 con 0x7f5fbc105110 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4003850 con 0x7f5fbc105110 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.389+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc1a4b20 con 0x7f5fbc105110 2024-09-13T17:25:56.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.390+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc1a4fc0 con 0x7f5fbc105110 2024-09-13T17:25:56.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.391+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc1014c0 con 0x7f5fbc105110 2024-09-13T17:25:56.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.392+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5fb40048c0 con 0x7f5fbc105110 2024-09-13T17:25:56.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.392+0000 7f5fb3fff640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5fa80a1190 0x7f5fa80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.392+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f5fb40ed8d0 con 0x7f5fbc105110 2024-09-13T17:25:56.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.393+0000 7f5fbaffd640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5fa80a1190 0x7f5fa80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.393+0000 7f5fbaffd640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5fa80a1190 0x7f5fa80a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005fd0 tx=0x7f5fac009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:56.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.397+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5fb40b6c70 con 0x7f5fbc105110 2024-09-13T17:25:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.573+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5fbc1003c0 con 0x7f5fbc105110 2024-09-13T17:25:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.573+0000 7f5fb3fff640 1 -- 172.21.15.44:0/1817311100 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fb40ba290 con 0x7f5fbc105110 2024-09-13T17:25:56.574 INFO:teuthology.orchestra.run.smithi044.stdout:137438953536 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.575+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5fa80a1190 msgr2=0x7f5fa80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.575+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5fa80a1190 0x7f5fa80a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005fd0 tx=0x7f5fac009680 comp rx=0 tx=0).stop 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.575+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 msgr2=0x7f5fbc073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.575+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc073230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009900 tx=0x7f5fb4032c20 comp rx=0 tx=0).stop 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 shutdown_connections 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5fa80a1190 0x7f5fa80a3650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc105110 0x7f5fbc073230 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fbc100990 0x7f5fbc072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 --2- 172.21.15.44:0/1817311100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fbc0fffe0 0x7f5fbc0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 >> 172.21.15.44:0/1817311100 conn(0x7f5fbc0fbae0 msgr2=0x7f5fbc0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 shutdown_connections 2024-09-13T17:25:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.576+0000 7f5fc17ba640 1 -- 172.21.15.44:0/1817311100 wait complete. 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 -- 172.21.15.44:0/2638573194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 msgr2=0x7f3e900ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2638573194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e900ff2b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009930 tx=0x7f3e8002a420 comp rx=0 tx=0).stop 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 -- 172.21.15.44:0/2638573194 shutdown_connections 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2638573194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e90104000 0x7f3e9010bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2638573194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 0x7f3e901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2638573194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e900ff2b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 -- 172.21.15.44:0/2638573194 >> 172.21.15.44:0/2638573194 conn(0x7f3e900fa9b0 msgr2=0x7f3e900fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 -- 172.21.15.44:0/2638573194 shutdown_connections 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 -- 172.21.15.44:0/2638573194 wait complete. 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 Processor -- start 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.651+0000 7f3e95f22640 1 -- start start 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e95f22640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e90198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e95f22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 0x7f3e901989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e95f22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e90104000 0x7f3e9019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e95f22640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019d380 con 0x7f3e900feed0 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e95f22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019d4f0 con 0x7f3e900ff880 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e95f22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019d660 con 0x7f3e90104000 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e90198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e90198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60198/0 (socket says 172.21.15.44:60198) 2024-09-13T17:25:56.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 -- 172.21.15.44:0/2262305486 learned_addr learned my addr 172.21.15.44:0/2262305486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8ffff640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e90104000 0x7f3e9019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8effd640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 0x7f3e901989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 -- 172.21.15.44:0/2262305486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e90104000 msgr2=0x7f3e9019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e90104000 0x7f3e9019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 -- 172.21.15.44:0/2262305486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 msgr2=0x7f3e901989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 0x7f3e901989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 -- 172.21.15.44:0/2262305486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009560 con 0x7f3e900feed0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8effd640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 0x7f3e901989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8f7fe640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e90198490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005ab0 tx=0x7f3e80032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.652+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80032e10 con 0x7f3e900feed0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.653+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9019d8e0 con 0x7f3e900feed0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.653+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e800046c0 con 0x7f3e900feed0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.653+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80003850 con 0x7f3e900feed0 2024-09-13T17:25:56.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.653+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9019dc00 con 0x7f3e900feed0 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.654+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e901003b0 con 0x7f3e900feed0 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.655+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3e80004830 con 0x7f3e900feed0 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.655+0000 7f3e8cff9640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e780a1110 0x7f3e780a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.656+0000 7f3e8effd640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e780a1110 0x7f3e780a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.656+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f3e800edea0 con 0x7f3e900feed0 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.656+0000 7f3e8effd640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e780a1110 0x7f3e780a35d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e901999f0 tx=0x7f3e84005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:56.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.659+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3e800b7130 con 0x7f3e900feed0 2024-09-13T17:25:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.834+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3e9010fc80 con 0x7f3e900feed0 2024-09-13T17:25:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.835+0000 7f3e8cff9640 1 -- 172.21.15.44:0/2262305486 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e80035090 con 0x7f3e900feed0 2024-09-13T17:25:56.836 INFO:teuthology.orchestra.run.smithi044.stdout:34359738451 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e780a1110 msgr2=0x7f3e780a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e780a1110 0x7f3e780a35d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e901999f0 tx=0x7f3e84005d30 comp rx=0 tx=0).stop 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 msgr2=0x7f3e90198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e90198490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005ab0 tx=0x7f3e80032c20 comp rx=0 tx=0).stop 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 shutdown_connections 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3e780a1110 0x7f3e780a35d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e90104000 0x7f3e9019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e900ff880 0x7f3e901989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 --2- 172.21.15.44:0/2262305486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e900feed0 0x7f3e90198490 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:56.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 >> 172.21.15.44:0/2262305486 conn(0x7f3e900fa9b0 msgr2=0x7f3e900fbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:56.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 shutdown_connections 2024-09-13T17:25:56.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:56.837+0000 7f3e95f22640 1 -- 172.21.15.44:0/2262305486 wait complete. 2024-09-13T17:25:57.302 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953531 got 137438953536 for osd.4 2024-09-13T17:25:57.302 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:57.574 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:57 smithi044 ceph-mon[31175]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-13T17:25:57.574 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:57 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1817311100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T17:25:57.574 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:57 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2262305486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T17:25:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:57 smithi100 ceph-mon[35055]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-13T17:25:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:57 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1817311100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T17:25:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:57 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2262305486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T17:25:57.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:57 smithi070 ceph-mon[34502]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-13T17:25:57.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:57 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1817311100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T17:25:57.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:57 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2262305486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T17:25:57.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738451 for osd.0 2024-09-13T17:25:57.995 DEBUG:teuthology.parallel:result is None 2024-09-13T17:25:57.996 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-13T17:25:57.996 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T17:25:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:25:59.704 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:25:59 smithi044 ceph-mon[31175]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:25:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:25:59 smithi100 ceph-mon[35055]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:25:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 -- 172.21.15.44:0/2755549117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880ff880 msgr2=0x7f81881038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 --2- 172.21.15.44:0/2755549117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880ff880 0x7f81881038d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8170009930 tx=0x7f817002a420 comp rx=0 tx=0).stop 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 -- 172.21.15.44:0/2755549117 shutdown_connections 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 --2- 172.21.15.44:0/2755549117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8188104000 0x7f818810bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 --2- 172.21.15.44:0/2755549117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880ff880 0x7f81881038d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 --2- 172.21.15.44:0/2755549117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81880feed0 0x7f81880ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 -- 172.21.15.44:0/2755549117 >> 172.21.15.44:0/2755549117 conn(0x7f81880fa9b0 msgr2=0x7f81880fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 -- 172.21.15.44:0/2755549117 shutdown_connections 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.934+0000 7f818cd22640 1 -- 172.21.15.44:0/2755549117 wait complete. 2024-09-13T17:25:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 Processor -- start 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 -- start start 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 0x7f81881950d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81880ff880 0x7f8188195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188104000 0x7f8188199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818819a010 con 0x7f81880feed0 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818819a180 con 0x7f8188104000 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818cd22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818819a2f0 con 0x7f81880ff880 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 0x7f81881950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 0x7f81881950d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34298/0 (socket says 172.21.15.44:34298) 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f818659c640 1 -- 172.21.15.44:0/136076950 learned_addr learned my addr 172.21.15.44:0/136076950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:25:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f817dd9b640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81880ff880 0x7f8188195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.935+0000 7f8186d9d640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188104000 0x7f8188199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818659c640 1 -- 172.21.15.44:0/136076950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81880ff880 msgr2=0x7f8188195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818659c640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81880ff880 0x7f8188195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818659c640 1 -- 172.21.15.44:0/136076950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188104000 msgr2=0x7f8188199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818659c640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188104000 0x7f8188199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818659c640 1 -- 172.21.15.44:0/136076950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170009560 con 0x7f81880feed0 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f817dd9b640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81880ff880 0x7f8188195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818659c640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 0x7f81881950d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8178009970 tx=0x7f8178009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178012070 con 0x7f81880feed0 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818819a5d0 con 0x7f81880feed0 2024-09-13T17:25:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8178007860 con 0x7f81880feed0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817800c8f0 con 0x7f81880feed0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.936+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81880680d0 con 0x7f81880feed0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.937+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81881003b0 con 0x7f81880feed0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.938+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8178016070 con 0x7f81880feed0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.939+0000 7f817ffff640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f81740a1110 0x7f81740a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.939+0000 7f817dd9b640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f81740a1110 0x7f81740a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.939+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f81780cde60 con 0x7f81880feed0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.939+0000 7f817dd9b640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f81740a1110 0x7f81740a35d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8170006010 tx=0x7f8170005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:25:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:25:59.943+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f817809ab80 con 0x7f81880feed0 2024-09-13T17:25:59.950 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:25:59 smithi070 ceph-mon[34502]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:26:00.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.112+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81881964d0 con 0x7f81740a1110 2024-09-13T17:26:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.123+0000 7f817ffff640 1 -- 172.21.15.44:0/136076950 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427538 (secure 0 0 0) 0x7f81881964d0 con 0x7f81740a1110 2024-09-13T17:26:00.129 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f81740a1110 msgr2=0x7f81740a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f81740a1110 0x7f81740a35d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8170006010 tx=0x7f8170005a40 comp rx=0 tx=0).stop 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 msgr2=0x7f81881950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 0x7f81881950d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8178009970 tx=0x7f8178009e30 comp rx=0 tx=0).stop 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 shutdown_connections 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f81740a1110 0x7f81740a35d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81880ff880 0x7f8188195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188104000 0x7f8188199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 --2- 172.21.15.44:0/136076950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81880feed0 0x7f81881950d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 >> 172.21.15.44:0/136076950 conn(0x7f81880fa9b0 msgr2=0x7f81880fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 shutdown_connections 2024-09-13T17:26:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:00.127+0000 7f818cd22640 1 -- 172.21.15.44:0/136076950 wait complete. 2024-09-13T17:26:00.831 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-13T17:25:59.579065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656795,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":580,"num_read_kb":413,"num_write":341,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385768,"kb_used_data":12344,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124556056,"statfs":{"total":1151940427776,"available":1151545401344,"internally_reserved":0,"allocated":12640256,"data_stored":8127554,"data_compressed":391110,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003296"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048059+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:25:42.048059+0000","last_peered":"2024-09-13T17:25:42.048059+0000","last_clean":"2024-09-13T17:25:42.048059+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:25:42.048059+0000","last_undegraded":"2024-09-13T17:25:42.048059+0000","last_fullsized":"2024-09-13T17:25:42.048059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053730+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:25:42.053730+0000","last_peered":"2024-09-13T17:25:42.053730+0000","last_clean":"2024-09-13T17:25:42.053730+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:25:42.053730+0000","last_undegraded":"2024-09-13T17:25:42.053730+0000","last_fullsized":"2024-09-13T17:25:42.053730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048684+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:25:42.048684+0000","last_peered":"2024-09-13T17:25:42.048684+0000","last_clean":"2024-09-13T17:25:42.048684+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:25:42.048684+0000","last_undegraded":"2024-09-13T17:25:42.048684+0000","last_fullsized":"2024-09-13T17:25:42.048684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053847+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:25:42.053847+0000","last_peered":"2024-09-13T17:25:42.053847+0000","last_clean":"2024-09-13T17:25:42.053847+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:25:42.053847+0000","last_undegraded":"2024-09-13T17:25:42.053847+0000","last_fullsized":"2024-09-13T17:25:42.053847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443551+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:25:42.443551+0000","last_peered":"2024-09-13T17:25:42.443551+0000","last_clean":"2024-09-13T17:25:42.443551+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:25:42.443551+0000","last_undegraded":"2024-09-13T17:25:42.443551+0000","last_fullsized":"2024-09-13T17:25:42.443551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048486+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:25:42.048486+0000","last_peered":"2024-09-13T17:25:42.048486+0000","last_clean":"2024-09-13T17:25:42.048486+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:25:42.048486+0000","last_undegraded":"2024-09-13T17:25:42.048486+0000","last_fullsized":"2024-09-13T17:25:42.048486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.062868+0000","last_change":"2024-09-13T17:25:43.062483+0000","last_active":"2024-09-13T17:25:43.062868+0000","last_peered":"2024-09-13T17:25:43.062868+0000","last_clean":"2024-09-13T17:25:43.062868+0000","last_became_active":"2024-09-13T17:25:43.058145+0000","last_became_peered":"2024-09-13T17:25:43.058145+0000","last_unstale":"2024-09-13T17:25:43.062868+0000","last_undegraded":"2024-09-13T17:25:43.062868+0000","last_fullsized":"2024-09-13T17:25:43.062868+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:10.848768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054176+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:25:42.054176+0000","last_peered":"2024-09-13T17:25:42.054176+0000","last_clean":"2024-09-13T17:25:42.054176+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:25:42.054176+0000","last_undegraded":"2024-09-13T17:25:42.054176+0000","last_fullsized":"2024-09-13T17:25:42.054176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048684+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:42.048684+0000","last_peered":"2024-09-13T17:25:42.048684+0000","last_clean":"2024-09-13T17:25:42.048684+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:25:42.048684+0000","last_undegraded":"2024-09-13T17:25:42.048684+0000","last_fullsized":"2024-09-13T17:25:42.048684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053857+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:25:42.053857+0000","last_peered":"2024-09-13T17:25:42.053857+0000","last_clean":"2024-09-13T17:25:42.053857+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:25:42.053857+0000","last_undegraded":"2024-09-13T17:25:42.053857+0000","last_fullsized":"2024-09-13T17:25:42.053857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233291+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:25:42.233291+0000","last_peered":"2024-09-13T17:25:42.233291+0000","last_clean":"2024-09-13T17:25:42.233291+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:25:42.233291+0000","last_undegraded":"2024-09-13T17:25:42.233291+0000","last_fullsized":"2024-09-13T17:25:42.233291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048504+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:25:42.048504+0000","last_peered":"2024-09-13T17:25:42.048504+0000","last_clean":"2024-09-13T17:25:42.048504+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:25:42.048504+0000","last_undegraded":"2024-09-13T17:25:42.048504+0000","last_fullsized":"2024-09-13T17:25:42.048504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053492+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:25:42.053492+0000","last_peered":"2024-09-13T17:25:42.053492+0000","last_clean":"2024-09-13T17:25:42.053492+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:25:42.053492+0000","last_undegraded":"2024-09-13T17:25:42.053492+0000","last_fullsized":"2024-09-13T17:25:42.053492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053607+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:25:42.053607+0000","last_peered":"2024-09-13T17:25:42.053607+0000","last_clean":"2024-09-13T17:25:42.053607+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:25:42.053607+0000","last_undegraded":"2024-09-13T17:25:42.053607+0000","last_fullsized":"2024-09-13T17:25:42.053607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048930+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:25:42.048930+0000","last_peered":"2024-09-13T17:25:42.048930+0000","last_clean":"2024-09-13T17:25:42.048930+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:25:42.048930+0000","last_undegraded":"2024-09-13T17:25:42.048930+0000","last_fullsized":"2024-09-13T17:25:42.048930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053323+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:25:42.053323+0000","last_peered":"2024-09-13T17:25:42.053323+0000","last_clean":"2024-09-13T17:25:42.053323+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:25:42.053323+0000","last_undegraded":"2024-09-13T17:25:42.053323+0000","last_fullsized":"2024-09-13T17:25:42.053323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048858+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:25:42.048858+0000","last_peered":"2024-09-13T17:25:42.048858+0000","last_clean":"2024-09-13T17:25:42.048858+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:25:42.048858+0000","last_undegraded":"2024-09-13T17:25:42.048858+0000","last_fullsized":"2024-09-13T17:25:42.048858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048745+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:25:42.048745+0000","last_peered":"2024-09-13T17:25:42.048745+0000","last_clean":"2024-09-13T17:25:42.048745+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:25:42.048745+0000","last_undegraded":"2024-09-13T17:25:42.048745+0000","last_fullsized":"2024-09-13T17:25:42.048745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051895+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:25:42.051895+0000","last_peered":"2024-09-13T17:25:42.051895+0000","last_clean":"2024-09-13T17:25:42.051895+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:25:42.051895+0000","last_undegraded":"2024-09-13T17:25:42.051895+0000","last_fullsized":"2024-09-13T17:25:42.051895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053394+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:25:42.053394+0000","last_peered":"2024-09-13T17:25:42.053394+0000","last_clean":"2024-09-13T17:25:42.053394+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:25:42.053394+0000","last_undegraded":"2024-09-13T17:25:42.053394+0000","last_fullsized":"2024-09-13T17:25:42.053394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047948+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:25:42.047948+0000","last_peered":"2024-09-13T17:25:42.047948+0000","last_clean":"2024-09-13T17:25:42.047948+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:25:42.047948+0000","last_undegraded":"2024-09-13T17:25:42.047948+0000","last_fullsized":"2024-09-13T17:25:42.047948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053348+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:25:42.053348+0000","last_peered":"2024-09-13T17:25:42.053348+0000","last_clean":"2024-09-13T17:25:42.053348+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:25:42.053348+0000","last_undegraded":"2024-09-13T17:25:42.053348+0000","last_fullsized":"2024-09-13T17:25:42.053348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053613+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:25:42.053613+0000","last_peered":"2024-09-13T17:25:42.053613+0000","last_clean":"2024-09-13T17:25:42.053613+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:25:42.053613+0000","last_undegraded":"2024-09-13T17:25:42.053613+0000","last_fullsized":"2024-09-13T17:25:42.053613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048853+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:25:42.048853+0000","last_peered":"2024-09-13T17:25:42.048853+0000","last_clean":"2024-09-13T17:25:42.048853+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:25:42.048853+0000","last_undegraded":"2024-09-13T17:25:42.048853+0000","last_fullsized":"2024-09-13T17:25:42.048853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443378+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:25:42.443378+0000","last_peered":"2024-09-13T17:25:42.443378+0000","last_clean":"2024-09-13T17:25:42.443378+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:25:42.443378+0000","last_undegraded":"2024-09-13T17:25:42.443378+0000","last_fullsized":"2024-09-13T17:25:42.443378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049117+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:25:42.049117+0000","last_peered":"2024-09-13T17:25:42.049117+0000","last_clean":"2024-09-13T17:25:42.049117+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:25:42.049117+0000","last_undegraded":"2024-09-13T17:25:42.049117+0000","last_fullsized":"2024-09-13T17:25:42.049117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048693+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:25:42.048693+0000","last_peered":"2024-09-13T17:25:42.048693+0000","last_clean":"2024-09-13T17:25:42.048693+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:25:42.048693+0000","last_undegraded":"2024-09-13T17:25:42.048693+0000","last_fullsized":"2024-09-13T17:25:42.048693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053779+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:25:42.053779+0000","last_peered":"2024-09-13T17:25:42.053779+0000","last_clean":"2024-09-13T17:25:42.053779+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:25:42.053779+0000","last_undegraded":"2024-09-13T17:25:42.053779+0000","last_fullsized":"2024-09-13T17:25:42.053779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048127+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:25:42.048127+0000","last_peered":"2024-09-13T17:25:42.048127+0000","last_clean":"2024-09-13T17:25:42.048127+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:25:42.048127+0000","last_undegraded":"2024-09-13T17:25:42.048127+0000","last_fullsized":"2024-09-13T17:25:42.048127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053099+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:25:42.053099+0000","last_peered":"2024-09-13T17:25:42.053099+0000","last_clean":"2024-09-13T17:25:42.053099+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:25:42.053099+0000","last_undegraded":"2024-09-13T17:25:42.053099+0000","last_fullsized":"2024-09-13T17:25:42.053099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051478+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:25:42.051478+0000","last_peered":"2024-09-13T17:25:42.051478+0000","last_clean":"2024-09-13T17:25:42.051478+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:25:42.051478+0000","last_undegraded":"2024-09-13T17:25:42.051478+0000","last_fullsized":"2024-09-13T17:25:42.051478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052259+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:25:42.052259+0000","last_peered":"2024-09-13T17:25:42.052259+0000","last_clean":"2024-09-13T17:25:42.052259+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:25:42.052259+0000","last_undegraded":"2024-09-13T17:25:42.052259+0000","last_fullsized":"2024-09-13T17:25:42.052259+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052005+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:25:42.052005+0000","last_peered":"2024-09-13T17:25:42.052005+0000","last_clean":"2024-09-13T17:25:42.052005+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:25:42.052005+0000","last_undegraded":"2024-09-13T17:25:42.052005+0000","last_fullsized":"2024-09-13T17:25:42.052005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229783+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:25:42.229783+0000","last_peered":"2024-09-13T17:25:42.229783+0000","last_clean":"2024-09-13T17:25:42.229783+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:25:42.229783+0000","last_undegraded":"2024-09-13T17:25:42.229783+0000","last_fullsized":"2024-09-13T17:25:42.229783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047876+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:25:42.047876+0000","last_peered":"2024-09-13T17:25:42.047876+0000","last_clean":"2024-09-13T17:25:42.047876+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:25:42.047876+0000","last_undegraded":"2024-09-13T17:25:42.047876+0000","last_fullsized":"2024-09-13T17:25:42.047876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053350+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:25:42.053350+0000","last_peered":"2024-09-13T17:25:42.053350+0000","last_clean":"2024-09-13T17:25:42.053350+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:25:42.053350+0000","last_undegraded":"2024-09-13T17:25:42.053350+0000","last_fullsized":"2024-09-13T17:25:42.053350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229716+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:25:42.229716+0000","last_peered":"2024-09-13T17:25:42.229716+0000","last_clean":"2024-09-13T17:25:42.229716+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:25:42.229716+0000","last_undegraded":"2024-09-13T17:25:42.229716+0000","last_fullsized":"2024-09-13T17:25:42.229716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053579+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:25:42.053579+0000","last_peered":"2024-09-13T17:25:42.053579+0000","last_clean":"2024-09-13T17:25:42.053579+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:25:42.053579+0000","last_undegraded":"2024-09-13T17:25:42.053579+0000","last_fullsized":"2024-09-13T17:25:42.053579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051383+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:25:42.051383+0000","last_peered":"2024-09-13T17:25:42.051383+0000","last_clean":"2024-09-13T17:25:42.051383+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:25:42.051383+0000","last_undegraded":"2024-09-13T17:25:42.051383+0000","last_fullsized":"2024-09-13T17:25:42.051383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232793+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:25:42.232793+0000","last_peered":"2024-09-13T17:25:42.232793+0000","last_clean":"2024-09-13T17:25:42.232793+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:25:42.232793+0000","last_undegraded":"2024-09-13T17:25:42.232793+0000","last_fullsized":"2024-09-13T17:25:42.232793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:56.270839+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:25:56.270839+0000","last_peered":"2024-09-13T17:25:56.270839+0000","last_clean":"2024-09-13T17:25:56.270839+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:25:56.270839+0000","last_undegraded":"2024-09-13T17:25:56.270839+0000","last_fullsized":"2024-09-13T17:25:56.270839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051450+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:25:42.051450+0000","last_peered":"2024-09-13T17:25:42.051450+0000","last_clean":"2024-09-13T17:25:42.051450+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:25:42.051450+0000","last_undegraded":"2024-09-13T17:25:42.051450+0000","last_fullsized":"2024-09-13T17:25:42.051450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051026+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:25:42.051026+0000","last_peered":"2024-09-13T17:25:42.051026+0000","last_clean":"2024-09-13T17:25:42.051026+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:25:42.051026+0000","last_undegraded":"2024-09-13T17:25:42.051026+0000","last_fullsized":"2024-09-13T17:25:42.051026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053611+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:25:42.053611+0000","last_peered":"2024-09-13T17:25:42.053611+0000","last_clean":"2024-09-13T17:25:42.053611+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:25:42.053611+0000","last_undegraded":"2024-09-13T17:25:42.053611+0000","last_fullsized":"2024-09-13T17:25:42.053611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232974+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:25:42.232974+0000","last_peered":"2024-09-13T17:25:42.232974+0000","last_clean":"2024-09-13T17:25:42.232974+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:25:42.232974+0000","last_undegraded":"2024-09-13T17:25:42.232974+0000","last_fullsized":"2024-09-13T17:25:42.232974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053676+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:25:42.053676+0000","last_peered":"2024-09-13T17:25:42.053676+0000","last_clean":"2024-09-13T17:25:42.053676+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:25:42.053676+0000","last_undegraded":"2024-09-13T17:25:42.053676+0000","last_fullsized":"2024-09-13T17:25:42.053676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048854+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:25:42.048854+0000","last_peered":"2024-09-13T17:25:42.048854+0000","last_clean":"2024-09-13T17:25:42.048854+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:25:42.048854+0000","last_undegraded":"2024-09-13T17:25:42.048854+0000","last_fullsized":"2024-09-13T17:25:42.048854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":342,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053750+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:25:42.053750+0000","last_peered":"2024-09-13T17:25:42.053750+0000","last_clean":"2024-09-13T17:25:42.053750+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:25:42.053750+0000","last_undegraded":"2024-09-13T17:25:42.053750+0000","last_fullsized":"2024-09-13T17:25:42.053750+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054029+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:25:42.054029+0000","last_peered":"2024-09-13T17:25:42.054029+0000","last_clean":"2024-09-13T17:25:42.054029+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:25:42.054029+0000","last_undegraded":"2024-09-13T17:25:42.054029+0000","last_fullsized":"2024-09-13T17:25:42.054029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054201+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:25:42.054201+0000","last_peered":"2024-09-13T17:25:42.054201+0000","last_clean":"2024-09-13T17:25:42.054201+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:25:42.054201+0000","last_undegraded":"2024-09-13T17:25:42.054201+0000","last_fullsized":"2024-09-13T17:25:42.054201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049166+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:25:42.049166+0000","last_peered":"2024-09-13T17:25:42.049166+0000","last_clean":"2024-09-13T17:25:42.049166+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:25:42.049166+0000","last_undegraded":"2024-09-13T17:25:42.049166+0000","last_fullsized":"2024-09-13T17:25:42.049166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053830+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:25:42.053830+0000","last_peered":"2024-09-13T17:25:42.053830+0000","last_clean":"2024-09-13T17:25:42.053830+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:25:42.053830+0000","last_undegraded":"2024-09-13T17:25:42.053830+0000","last_fullsized":"2024-09-13T17:25:42.053830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054247+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:25:42.054247+0000","last_peered":"2024-09-13T17:25:42.054247+0000","last_clean":"2024-09-13T17:25:42.054247+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:25:42.054247+0000","last_undegraded":"2024-09-13T17:25:42.054247+0000","last_fullsized":"2024-09-13T17:25:42.054247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233410+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:25:42.233410+0000","last_peered":"2024-09-13T17:25:42.233410+0000","last_clean":"2024-09-13T17:25:42.233410+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:25:42.233410+0000","last_undegraded":"2024-09-13T17:25:42.233410+0000","last_fullsized":"2024-09-13T17:25:42.233410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443172+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:25:42.443172+0000","last_peered":"2024-09-13T17:25:42.443172+0000","last_clean":"2024-09-13T17:25:42.443172+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:25:42.443172+0000","last_undegraded":"2024-09-13T17:25:42.443172+0000","last_fullsized":"2024-09-13T17:25:42.443172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054003+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:25:42.054003+0000","last_peered":"2024-09-13T17:25:42.054003+0000","last_clean":"2024-09-13T17:25:42.054003+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:25:42.054003+0000","last_undegraded":"2024-09-13T17:25:42.054003+0000","last_fullsized":"2024-09-13T17:25:42.054003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.395166+0000","last_change":"2024-09-13T17:25:43.395166+0000","last_active":"2024-09-13T17:25:43.395166+0000","last_peered":"2024-09-13T17:25:43.395166+0000","last_clean":"2024-09-13T17:25:43.395166+0000","last_became_active":"2024-09-13T17:25:43.392868+0000","last_became_peered":"2024-09-13T17:25:43.392868+0000","last_unstale":"2024-09-13T17:25:43.395166+0000","last_undegraded":"2024-09-13T17:25:43.395166+0000","last_fullsized":"2024-09-13T17:25:43.395166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:24.860809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232528+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:25:42.232528+0000","last_peered":"2024-09-13T17:25:42.232528+0000","last_clean":"2024-09-13T17:25:42.232528+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:25:42.232528+0000","last_undegraded":"2024-09-13T17:25:42.232528+0000","last_fullsized":"2024-09-13T17:25:42.232528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052449+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:25:42.052449+0000","last_peered":"2024-09-13T17:25:42.052449+0000","last_clean":"2024-09-13T17:25:42.052449+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:25:42.052449+0000","last_undegraded":"2024-09-13T17:25:42.052449+0000","last_fullsized":"2024-09-13T17:25:42.052449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054210+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:25:42.054210+0000","last_peered":"2024-09-13T17:25:42.054210+0000","last_clean":"2024-09-13T17:25:42.054210+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:25:42.054210+0000","last_undegraded":"2024-09-13T17:25:42.054210+0000","last_fullsized":"2024-09-13T17:25:42.054210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232619+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:25:42.232619+0000","last_peered":"2024-09-13T17:25:42.232619+0000","last_clean":"2024-09-13T17:25:42.232619+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:25:42.232619+0000","last_undegraded":"2024-09-13T17:25:42.232619+0000","last_fullsized":"2024-09-13T17:25:42.232619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232940+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:25:42.232940+0000","last_peered":"2024-09-13T17:25:42.232940+0000","last_clean":"2024-09-13T17:25:42.232940+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:25:42.232940+0000","last_undegraded":"2024-09-13T17:25:42.232940+0000","last_fullsized":"2024-09-13T17:25:42.232940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053355+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:25:42.053355+0000","last_peered":"2024-09-13T17:25:42.053355+0000","last_clean":"2024-09-13T17:25:42.053355+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:25:42.053355+0000","last_undegraded":"2024-09-13T17:25:42.053355+0000","last_fullsized":"2024-09-13T17:25:42.053355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051553+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:25:42.051553+0000","last_peered":"2024-09-13T17:25:42.051553+0000","last_clean":"2024-09-13T17:25:42.051553+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:25:42.051553+0000","last_undegraded":"2024-09-13T17:25:42.051553+0000","last_fullsized":"2024-09-13T17:25:42.051553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053263+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:25:42.053263+0000","last_peered":"2024-09-13T17:25:42.053263+0000","last_clean":"2024-09-13T17:25:42.053263+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:25:42.053263+0000","last_undegraded":"2024-09-13T17:25:42.053263+0000","last_fullsized":"2024-09-13T17:25:42.053263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:55.269612+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:25:55.269612+0000","last_peered":"2024-09-13T17:25:55.269612+0000","last_clean":"2024-09-13T17:25:55.269612+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:25:55.269612+0000","last_undegraded":"2024-09-13T17:25:55.269612+0000","last_fullsized":"2024-09-13T17:25:55.269612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054061+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:25:42.054061+0000","last_peered":"2024-09-13T17:25:42.054061+0000","last_clean":"2024-09-13T17:25:42.054061+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:25:42.054061+0000","last_undegraded":"2024-09-13T17:25:42.054061+0000","last_fullsized":"2024-09-13T17:25:42.054061+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053677+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:25:42.053677+0000","last_peered":"2024-09-13T17:25:42.053677+0000","last_clean":"2024-09-13T17:25:42.053677+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:25:42.053677+0000","last_undegraded":"2024-09-13T17:25:42.053677+0000","last_fullsized":"2024-09-13T17:25:42.053677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233293+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:25:42.233293+0000","last_peered":"2024-09-13T17:25:42.233293+0000","last_clean":"2024-09-13T17:25:42.233293+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:25:42.233293+0000","last_undegraded":"2024-09-13T17:25:42.233293+0000","last_fullsized":"2024-09-13T17:25:42.233293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051954+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:42.051954+0000","last_peered":"2024-09-13T17:25:42.051954+0000","last_clean":"2024-09-13T17:25:42.051954+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:25:42.051954+0000","last_undegraded":"2024-09-13T17:25:42.051954+0000","last_fullsized":"2024-09-13T17:25:42.051954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051222+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:25:42.051222+0000","last_peered":"2024-09-13T17:25:42.051222+0000","last_clean":"2024-09-13T17:25:42.051222+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:25:42.051222+0000","last_undegraded":"2024-09-13T17:25:42.051222+0000","last_fullsized":"2024-09-13T17:25:42.051222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048933+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:25:42.048933+0000","last_peered":"2024-09-13T17:25:42.048933+0000","last_clean":"2024-09-13T17:25:42.048933+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:25:42.048933+0000","last_undegraded":"2024-09-13T17:25:42.048933+0000","last_fullsized":"2024-09-13T17:25:42.048933+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233055+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:25:42.233055+0000","last_peered":"2024-09-13T17:25:42.233055+0000","last_clean":"2024-09-13T17:25:42.233055+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:25:42.233055+0000","last_undegraded":"2024-09-13T17:25:42.233055+0000","last_fullsized":"2024-09-13T17:25:42.233055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053907+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:25:42.053907+0000","last_peered":"2024-09-13T17:25:42.053907+0000","last_clean":"2024-09-13T17:25:42.053907+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:25:42.053907+0000","last_undegraded":"2024-09-13T17:25:42.053907+0000","last_fullsized":"2024-09-13T17:25:42.053907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":199,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:54.252993+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:25:54.252993+0000","last_peered":"2024-09-13T17:25:54.252993+0000","last_clean":"2024-09-13T17:25:54.252993+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:25:54.252993+0000","last_undegraded":"2024-09-13T17:25:54.252993+0000","last_fullsized":"2024-09-13T17:25:54.252993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62287,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047639+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:25:42.047639+0000","last_peered":"2024-09-13T17:25:42.047639+0000","last_clean":"2024-09-13T17:25:42.047639+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:25:42.047639+0000","last_undegraded":"2024-09-13T17:25:42.047639+0000","last_fullsized":"2024-09-13T17:25:42.047639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054093+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:25:42.054093+0000","last_peered":"2024-09-13T17:25:42.054093+0000","last_clean":"2024-09-13T17:25:42.054093+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:25:42.054093+0000","last_undegraded":"2024-09-13T17:25:42.054093+0000","last_fullsized":"2024-09-13T17:25:42.054093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233421+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:25:42.233421+0000","last_peered":"2024-09-13T17:25:42.233421+0000","last_clean":"2024-09-13T17:25:42.233421+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:25:42.233421+0000","last_undegraded":"2024-09-13T17:25:42.233421+0000","last_fullsized":"2024-09-13T17:25:42.233421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048574+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:25:42.048574+0000","last_peered":"2024-09-13T17:25:42.048574+0000","last_clean":"2024-09-13T17:25:42.048574+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:25:42.048574+0000","last_undegraded":"2024-09-13T17:25:42.048574+0000","last_fullsized":"2024-09-13T17:25:42.048574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443590+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:25:42.443590+0000","last_peered":"2024-09-13T17:25:42.443590+0000","last_clean":"2024-09-13T17:25:42.443590+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:25:42.443590+0000","last_undegraded":"2024-09-13T17:25:42.443590+0000","last_fullsized":"2024-09-13T17:25:42.443590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054180+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:25:42.054180+0000","last_peered":"2024-09-13T17:25:42.054180+0000","last_clean":"2024-09-13T17:25:42.054180+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:25:42.054180+0000","last_undegraded":"2024-09-13T17:25:42.054180+0000","last_fullsized":"2024-09-13T17:25:42.054180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047490+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:25:42.047490+0000","last_peered":"2024-09-13T17:25:42.047490+0000","last_clean":"2024-09-13T17:25:42.047490+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:25:42.047490+0000","last_undegraded":"2024-09-13T17:25:42.047490+0000","last_fullsized":"2024-09-13T17:25:42.047490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054352+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:25:42.054352+0000","last_peered":"2024-09-13T17:25:42.054352+0000","last_clean":"2024-09-13T17:25:42.054352+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:25:42.054352+0000","last_undegraded":"2024-09-13T17:25:42.054352+0000","last_fullsized":"2024-09-13T17:25:42.054352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":92,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:53.251319+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:25:53.251319+0000","last_peered":"2024-09-13T17:25:53.251319+0000","last_clean":"2024-09-13T17:25:53.251319+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:25:53.251319+0000","last_undegraded":"2024-09-13T17:25:53.251319+0000","last_fullsized":"2024-09-13T17:25:53.251319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053237+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:25:42.053237+0000","last_peered":"2024-09-13T17:25:42.053237+0000","last_clean":"2024-09-13T17:25:42.053237+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:25:42.053237+0000","last_undegraded":"2024-09-13T17:25:42.053237+0000","last_fullsized":"2024-09-13T17:25:42.053237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443326+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:25:42.443326+0000","last_peered":"2024-09-13T17:25:42.443326+0000","last_clean":"2024-09-13T17:25:42.443326+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:25:42.443326+0000","last_undegraded":"2024-09-13T17:25:42.443326+0000","last_fullsized":"2024-09-13T17:25:42.443326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047633+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:25:42.047633+0000","last_peered":"2024-09-13T17:25:42.047633+0000","last_clean":"2024-09-13T17:25:42.047633+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:25:42.047633+0000","last_undegraded":"2024-09-13T17:25:42.047633+0000","last_fullsized":"2024-09-13T17:25:42.047633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233208+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:25:42.233208+0000","last_peered":"2024-09-13T17:25:42.233208+0000","last_clean":"2024-09-13T17:25:42.233208+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:25:42.233208+0000","last_undegraded":"2024-09-13T17:25:42.233208+0000","last_fullsized":"2024-09-13T17:25:42.233208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052502+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:25:42.052502+0000","last_peered":"2024-09-13T17:25:42.052502+0000","last_clean":"2024-09-13T17:25:42.052502+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:25:42.052502+0000","last_undegraded":"2024-09-13T17:25:42.052502+0000","last_fullsized":"2024-09-13T17:25:42.052502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053216+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:25:42.053216+0000","last_peered":"2024-09-13T17:25:42.053216+0000","last_clean":"2024-09-13T17:25:42.053216+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:25:42.053216+0000","last_undegraded":"2024-09-13T17:25:42.053216+0000","last_fullsized":"2024-09-13T17:25:42.053216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051043+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:25:42.051043+0000","last_peered":"2024-09-13T17:25:42.051043+0000","last_clean":"2024-09-13T17:25:42.051043+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:25:42.051043+0000","last_undegraded":"2024-09-13T17:25:42.051043+0000","last_fullsized":"2024-09-13T17:25:42.051043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229612+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:25:42.229612+0000","last_peered":"2024-09-13T17:25:42.229612+0000","last_clean":"2024-09-13T17:25:42.229612+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:25:42.229612+0000","last_undegraded":"2024-09-13T17:25:42.229612+0000","last_fullsized":"2024-09-13T17:25:42.229612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233114+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:25:42.233114+0000","last_peered":"2024-09-13T17:25:42.233114+0000","last_clean":"2024-09-13T17:25:42.233114+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:25:42.233114+0000","last_undegraded":"2024-09-13T17:25:42.233114+0000","last_fullsized":"2024-09-13T17:25:42.233114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048848+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:25:42.048848+0000","last_peered":"2024-09-13T17:25:42.048848+0000","last_clean":"2024-09-13T17:25:42.048848+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:25:42.048848+0000","last_undegraded":"2024-09-13T17:25:42.048848+0000","last_fullsized":"2024-09-13T17:25:42.048848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":96,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:53.252087+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:25:53.252087+0000","last_peered":"2024-09-13T17:25:53.252087+0000","last_clean":"2024-09-13T17:25:53.252087+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:25:53.252087+0000","last_undegraded":"2024-09-13T17:25:53.252087+0000","last_fullsized":"2024-09-13T17:25:53.252087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053747+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:25:42.053747+0000","last_peered":"2024-09-13T17:25:42.053747+0000","last_clean":"2024-09-13T17:25:42.053747+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:25:42.053747+0000","last_undegraded":"2024-09-13T17:25:42.053747+0000","last_fullsized":"2024-09-13T17:25:42.053747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049147+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:25:42.049147+0000","last_peered":"2024-09-13T17:25:42.049147+0000","last_clean":"2024-09-13T17:25:42.049147+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:25:42.049147+0000","last_undegraded":"2024-09-13T17:25:42.049147+0000","last_fullsized":"2024-09-13T17:25:42.049147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051901+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:25:42.051901+0000","last_peered":"2024-09-13T17:25:42.051901+0000","last_clean":"2024-09-13T17:25:42.051901+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:25:42.051901+0000","last_undegraded":"2024-09-13T17:25:42.051901+0000","last_fullsized":"2024-09-13T17:25:42.051901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049119+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:25:42.049119+0000","last_peered":"2024-09-13T17:25:42.049119+0000","last_clean":"2024-09-13T17:25:42.049119+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:25:42.049119+0000","last_undegraded":"2024-09-13T17:25:42.049119+0000","last_fullsized":"2024-09-13T17:25:42.049119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048934+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:25:42.048934+0000","last_peered":"2024-09-13T17:25:42.048934+0000","last_clean":"2024-09-13T17:25:42.048934+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:25:42.048934+0000","last_undegraded":"2024-09-13T17:25:42.048934+0000","last_fullsized":"2024-09-13T17:25:42.048934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443260+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:25:42.443260+0000","last_peered":"2024-09-13T17:25:42.443260+0000","last_clean":"2024-09-13T17:25:42.443260+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:25:42.443260+0000","last_undegraded":"2024-09-13T17:25:42.443260+0000","last_fullsized":"2024-09-13T17:25:42.443260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051383+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:25:42.051383+0000","last_peered":"2024-09-13T17:25:42.051383+0000","last_clean":"2024-09-13T17:25:42.051383+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:25:42.051383+0000","last_undegraded":"2024-09-13T17:25:42.051383+0000","last_fullsized":"2024-09-13T17:25:42.051383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052549+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:25:42.052549+0000","last_peered":"2024-09-13T17:25:42.052549+0000","last_clean":"2024-09-13T17:25:42.052549+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:25:42.052549+0000","last_undegraded":"2024-09-13T17:25:42.052549+0000","last_fullsized":"2024-09-13T17:25:42.052549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049352+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:25:42.049352+0000","last_peered":"2024-09-13T17:25:42.049352+0000","last_clean":"2024-09-13T17:25:42.049352+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:25:42.049352+0000","last_undegraded":"2024-09-13T17:25:42.049352+0000","last_fullsized":"2024-09-13T17:25:42.049352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053653+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:25:42.053653+0000","last_peered":"2024-09-13T17:25:42.053653+0000","last_clean":"2024-09-13T17:25:42.053653+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:25:42.053653+0000","last_undegraded":"2024-09-13T17:25:42.053653+0000","last_fullsized":"2024-09-13T17:25:42.053653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232744+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:25:42.232744+0000","last_peered":"2024-09-13T17:25:42.232744+0000","last_clean":"2024-09-13T17:25:42.232744+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:25:42.232744+0000","last_undegraded":"2024-09-13T17:25:42.232744+0000","last_fullsized":"2024-09-13T17:25:42.232744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053281+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:25:42.053281+0000","last_peered":"2024-09-13T17:25:42.053281+0000","last_clean":"2024-09-13T17:25:42.053281+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:25:42.053281+0000","last_undegraded":"2024-09-13T17:25:42.053281+0000","last_fullsized":"2024-09-13T17:25:42.053281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053997+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:25:42.053997+0000","last_peered":"2024-09-13T17:25:42.053997+0000","last_clean":"2024-09-13T17:25:42.053997+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:25:42.053997+0000","last_undegraded":"2024-09-13T17:25:42.053997+0000","last_fullsized":"2024-09-13T17:25:42.053997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051462+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:25:42.051462+0000","last_peered":"2024-09-13T17:25:42.051462+0000","last_clean":"2024-09-13T17:25:42.051462+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:25:42.051462+0000","last_undegraded":"2024-09-13T17:25:42.051462+0000","last_fullsized":"2024-09-13T17:25:42.051462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052052+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:25:42.052052+0000","last_peered":"2024-09-13T17:25:42.052052+0000","last_clean":"2024-09-13T17:25:42.052052+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:25:42.052052+0000","last_undegraded":"2024-09-13T17:25:42.052052+0000","last_fullsized":"2024-09-13T17:25:42.052052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049260+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:25:42.049260+0000","last_peered":"2024-09-13T17:25:42.049260+0000","last_clean":"2024-09-13T17:25:42.049260+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:25:42.049260+0000","last_undegraded":"2024-09-13T17:25:42.049260+0000","last_fullsized":"2024-09-13T17:25:42.049260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048078+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:25:42.048078+0000","last_peered":"2024-09-13T17:25:42.048078+0000","last_clean":"2024-09-13T17:25:42.048078+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:25:42.048078+0000","last_undegraded":"2024-09-13T17:25:42.048078+0000","last_fullsized":"2024-09-13T17:25:42.048078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052670+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:25:42.052670+0000","last_peered":"2024-09-13T17:25:42.052670+0000","last_clean":"2024-09-13T17:25:42.052670+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:25:42.052670+0000","last_undegraded":"2024-09-13T17:25:42.052670+0000","last_fullsized":"2024-09-13T17:25:42.052670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053531+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:25:42.053531+0000","last_peered":"2024-09-13T17:25:42.053531+0000","last_clean":"2024-09-13T17:25:42.053531+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:25:42.053531+0000","last_undegraded":"2024-09-13T17:25:42.053531+0000","last_fullsized":"2024-09-13T17:25:42.053531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443108+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:25:42.443108+0000","last_peered":"2024-09-13T17:25:42.443108+0000","last_clean":"2024-09-13T17:25:42.443108+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:25:42.443108+0000","last_undegraded":"2024-09-13T17:25:42.443108+0000","last_fullsized":"2024-09-13T17:25:42.443108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443031+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:25:42.443031+0000","last_peered":"2024-09-13T17:25:42.443031+0000","last_clean":"2024-09-13T17:25:42.443031+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:25:42.443031+0000","last_undegraded":"2024-09-13T17:25:42.443031+0000","last_fullsized":"2024-09-13T17:25:42.443031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051385+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:25:42.051385+0000","last_peered":"2024-09-13T17:25:42.051385+0000","last_clean":"2024-09-13T17:25:42.051385+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:25:42.051385+0000","last_undegraded":"2024-09-13T17:25:42.051385+0000","last_fullsized":"2024-09-13T17:25:42.051385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048893+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:25:42.048893+0000","last_peered":"2024-09-13T17:25:42.048893+0000","last_clean":"2024-09-13T17:25:42.048893+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:25:42.048893+0000","last_undegraded":"2024-09-13T17:25:42.048893+0000","last_fullsized":"2024-09-13T17:25:42.048893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054254+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:25:42.054254+0000","last_peered":"2024-09-13T17:25:42.054254+0000","last_clean":"2024-09-13T17:25:42.054254+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:25:42.054254+0000","last_undegraded":"2024-09-13T17:25:42.054254+0000","last_fullsized":"2024-09-13T17:25:42.054254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229947+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:25:42.229947+0000","last_peered":"2024-09-13T17:25:42.229947+0000","last_clean":"2024-09-13T17:25:42.229947+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:25:42.229947+0000","last_undegraded":"2024-09-13T17:25:42.229947+0000","last_fullsized":"2024-09-13T17:25:42.229947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054458+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:25:42.054458+0000","last_peered":"2024-09-13T17:25:42.054458+0000","last_clean":"2024-09-13T17:25:42.054458+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:25:42.054458+0000","last_undegraded":"2024-09-13T17:25:42.054458+0000","last_fullsized":"2024-09-13T17:25:42.054458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053189+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:25:42.053189+0000","last_peered":"2024-09-13T17:25:42.053189+0000","last_clean":"2024-09-13T17:25:42.053189+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:25:42.053189+0000","last_undegraded":"2024-09-13T17:25:42.053189+0000","last_fullsized":"2024-09-13T17:25:42.053189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053477+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:25:42.053477+0000","last_peered":"2024-09-13T17:25:42.053477+0000","last_clean":"2024-09-13T17:25:42.053477+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:25:42.053477+0000","last_undegraded":"2024-09-13T17:25:42.053477+0000","last_fullsized":"2024-09-13T17:25:42.053477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229458+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:25:42.229458+0000","last_peered":"2024-09-13T17:25:42.229458+0000","last_clean":"2024-09-13T17:25:42.229458+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:25:42.229458+0000","last_undegraded":"2024-09-13T17:25:42.229458+0000","last_fullsized":"2024-09-13T17:25:42.229458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053452+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:25:42.053452+0000","last_peered":"2024-09-13T17:25:42.053452+0000","last_clean":"2024-09-13T17:25:42.053452+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:25:42.053452+0000","last_undegraded":"2024-09-13T17:25:42.053452+0000","last_fullsized":"2024-09-13T17:25:42.053452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047704+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:25:42.047704+0000","last_peered":"2024-09-13T17:25:42.047704+0000","last_clean":"2024-09-13T17:25:42.047704+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:25:42.047704+0000","last_undegraded":"2024-09-13T17:25:42.047704+0000","last_fullsized":"2024-09-13T17:25:42.047704+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443225+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:25:42.443225+0000","last_peered":"2024-09-13T17:25:42.443225+0000","last_clean":"2024-09-13T17:25:42.443225+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:25:42.443225+0000","last_undegraded":"2024-09-13T17:25:42.443225+0000","last_fullsized":"2024-09-13T17:25:42.443225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443250+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:25:42.443250+0000","last_peered":"2024-09-13T17:25:42.443250+0000","last_clean":"2024-09-13T17:25:42.443250+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:25:42.443250+0000","last_undegraded":"2024-09-13T17:25:42.443250+0000","last_fullsized":"2024-09-13T17:25:42.443250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052621+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:25:42.052621+0000","last_peered":"2024-09-13T17:25:42.052621+0000","last_clean":"2024-09-13T17:25:42.052621+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:25:42.052621+0000","last_undegraded":"2024-09-13T17:25:42.052621+0000","last_fullsized":"2024-09-13T17:25:42.052621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66193,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":444,"num_read_kb":294,"num_write":158,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285499,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":515208,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.72299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84999999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68600000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.313,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60699999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81499999999999995,"5min":1.05,"15min":1.05},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88400000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.89800000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92000000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.752}]},{"osd":10,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.73099999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1159168,"data_stored":1168850,"data_compressed":59819,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.438,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":1.101,"15min":1.101},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74399999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51300000000000001,"15min":0.52200000000000002},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.44900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.83099999999999996,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.752,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.628}]},{"osd":6,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":1.073,"15min":1.073},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82299999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.84399999999999997,"5min":1.147,"15min":1.147},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.53300000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.77000000000000002}]},{"osd":11,"last update":"Fri Sep 13 17:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.71699999999999997},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.79400000000000004,"5min":1.175,"15min":1.175},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.84399999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.77800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":515422,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45500000000000002,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64200000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.49299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68200000000000005,"5min":0.999,"15min":0.999},"last":0.317}]},{"osd":1,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.852,"5min":13.852,"15min":13.852},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":0.88300000000000001,"15min":0.77400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":17.055,"5min":17.055,"15min":17.055},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61399999999999999,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.495}]},{"osd":3,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57699999999999996,"15min":0.623},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.21,"5min":0.77700000000000002,"15min":0.71699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.074999999999999,"5min":17.074999999999999,"15min":17.074999999999999},"last":11.68}]},{"osd":5,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67200000000000004,"15min":0.70899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88700000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.753,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":1.054,"15min":1.054},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86499999999999999,"5min":1.083,"15min":1.083},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":11.696999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.745,"15min":0.76500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92100000000000004,"5min":1.04,"15min":1.04},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.77900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.64100000000000001}]},{"osd":11,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.67100000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":514989,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.44900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73999999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.41599999999999998,"15min":0.41799999999999998},"min":{"1min":0.26700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.33800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.40799999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.41399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40200000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46700000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.62},{"interface":"front","average":{"1min":0.75,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.56399999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.64200000000000002,"15min":0.63400000000000001},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.504,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61099999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.73099999999999998,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.44,"15min":0.44},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72799999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63},{"interface":"front","average":{"1min":0.755,"5min":0.66100000000000003,"15min":0.64500000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71599999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68100000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364851,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":1077248,"data_stored":1111156,"data_compressed":59819,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.29899999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.54600000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.56000000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.53800000000000003},"min":{"1min":0.48999999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.47599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.44600000000000001,"15min":0.42899999999999999},"min":{"1min":0.33100000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.67800000000000005,"5min":0.75,"15min":0.75},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.37}]},{"osd":5,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.82199999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.51500000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.46700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.621},"min":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73599999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58999999999999997}]},{"osd":10,"last update":"Fri Sep 13 17:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84999999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86699999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72099999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.77900000000000003}]}]},{"osd":6,"up_from":45,"seq":193273528377,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":510598,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.65300000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40300000000000002,"15min":0.41299999999999998},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54100000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.751},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91500000000000004,"5min":1.008,"15min":1.008},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.41599999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.65900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.874,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":1.018,"15min":1.018},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85499999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.70499999999999996},"min":{"1min":0.48599999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.52200000000000002}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.046,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.60199999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.629}]},{"osd":11,"last update":"Fri Sep 13 17:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0780000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1106400,"data_compressed":59819,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.495,"15min":0.47499999999999998},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51900000000000002,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.39100000000000001,"15min":0.35699999999999998},"min":{"1min":0.29199999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.36799999999999999},"min":{"1min":0.307,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.53200000000000003,"15min":0.48399999999999999},"min":{"1min":0.46400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.443,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.53100000000000003},"min":{"1min":0.47299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84699999999999998,"5min":1.3,"15min":1.3},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.57799999999999996},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":1.244,"15min":1.244},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.57599999999999996,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83699999999999997,"5min":4.327,"15min":4.327},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54600000000000004,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96799999999999997,"5min":1.212,"15min":1.212},"last":0.42499999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Sep 13 17:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68999999999999995,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.66000000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":1019904,"data_stored":515592,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.34300000000000003},"min":{"1min":0.32300000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.497,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.29699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55700000000000005,"15min":0.5},"min":{"1min":0.48999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83599999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.53200000000000003,"15min":0.502},"min":{"1min":0.45800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.436}]},{"osd":5,"last update":"Fri Sep 13 17:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.498},"min":{"1min":0.38300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77600000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.505},"min":{"1min":0.42199999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.437,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.49,"5min":0.76300000000000001,"15min":0.63},"min":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":14.458,"5min":14.458,"15min":14.458},"last":0.46800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.317,"5min":0.309,"15min":0.309},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51300000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84099999999999997,"5min":0.872,"15min":0.872},"last":0.63},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73799999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.58699999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53500000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63,"15min":0.626},"min":{"1min":0.443,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":1.02,"15min":1.02},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56699999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.753},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79200000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":1019904,"data_stored":510792,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.35699999999999998},"min":{"1min":0.27000000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.36799999999999999},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56499999999999995,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":0.504}]},{"osd":2,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78500000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.753}]},{"osd":4,"last update":"Fri Sep 13 17:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86799999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.44},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.115,"15min":1.115},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.52600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89400000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65500000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92300000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98899999999999999,"5min":1.069,"15min":1.069},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.95099999999999996,"5min":1.05,"15min":1.05},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95099999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73699999999999999}]},{"osd":11,"last update":"Fri Sep 13 17:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.69299999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":1097728,"data_stored":573766,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.496,"15min":0.45200000000000001},"min":{"1min":0.32600000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.47799999999999998},"min":{"1min":0.371,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.35999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.747,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52700000000000002,"15min":0.48799999999999999},"min":{"1min":0.39800000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.53000000000000003,"15min":0.47599999999999998},"min":{"1min":0.42199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63}]},{"osd":4,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46999999999999997,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44900000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.55100000000000005,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76000000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41399999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.625,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52200000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.57799999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67200000000000004}]},{"osd":11,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":1077248,"data_stored":572795,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84499999999999997,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.75}]},{"osd":2,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56899999999999995,"15min":0.51100000000000001},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.39300000000000002},"min":{"1min":0.34000000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.40899999999999997},"min":{"1min":0.33000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73999999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.374,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61499999999999999,"15min":0.56499999999999995},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.623,"15min":0.59699999999999998},"min":{"1min":0.504,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Sep 13 17:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.63300000000000001,"15min":0.60799999999999998},"min":{"1min":0.50800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.58899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98099999999999998,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.57099999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.878},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":511986,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.45800000000000002},"min":{"1min":0.40400000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55700000000000005,"15min":0.504},"min":{"1min":0.45200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.71899999999999997,"15min":0.56100000000000005},"min":{"1min":0.39900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.82199999999999995,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.70699999999999996,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":17.844999999999999,"5min":17.844999999999999,"15min":17.844999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60199999999999998,"15min":0.56599999999999995},"min":{"1min":0.439,"5min":0.379,"15min":0.379},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.628},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60299999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89700000000000002,"5min":1.099,"15min":1.099},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.51100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72199999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82099999999999995,"15min":0.81799999999999995},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.627},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90700000000000003,"5min":1.048,"15min":1.048},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:26:00.840 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T17:26:01.045 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:01.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:01 smithi100 ceph-mon[35055]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:26:01.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:01 smithi100 ceph-mon[35055]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:01.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:01 smithi044 ceph-mon[31175]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:26:01.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:01 smithi044 ceph-mon[31175]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:01 smithi070 ceph-mon[34502]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T17:26:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:01 smithi070 ceph-mon[34502]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.684+0000 7f3a893f5640 1 -- 172.21.15.44:0/242619377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a840fffe0 msgr2=0x7f3a841003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.684+0000 7f3a893f5640 1 --2- 172.21.15.44:0/242619377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a840fffe0 0x7f3a841003c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009930 tx=0x7f3a7402a420 comp rx=0 tx=0).stop 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 -- 172.21.15.44:0/242619377 shutdown_connections 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 --2- 172.21.15.44:0/242619377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a84100990 0x7f3a841049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 --2- 172.21.15.44:0/242619377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a84105110 0x7f3a8410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 --2- 172.21.15.44:0/242619377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a840fffe0 0x7f3a841003c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 -- 172.21.15.44:0/242619377 >> 172.21.15.44:0/242619377 conn(0x7f3a840fbae0 msgr2=0x7f3a840fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 -- 172.21.15.44:0/242619377 shutdown_connections 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 -- 172.21.15.44:0/242619377 wait complete. 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 Processor -- start 2024-09-13T17:26:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.685+0000 7f3a893f5640 1 -- start start 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a893f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a840fffe0 0x7f3a84196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a893f5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 0x7f3a841967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a893f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a84105110 0x7f3a8419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a893f5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8419b1c0 con 0x7f3a84100990 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a893f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8419b330 con 0x7f3a840fffe0 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a893f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8419b4a0 con 0x7f3a84105110 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 0x7f3a841967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 0x7f3a841967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34324/0 (socket says 172.21.15.44:34324) 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 -- 172.21.15.44:0/3807999430 learned_addr learned my addr 172.21.15.44:0/3807999430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a837fe640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a84105110 0x7f3a8419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a82ffd640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a840fffe0 0x7f3a84196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 -- 172.21.15.44:0/3807999430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a84105110 msgr2=0x7f3a8419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a84105110 0x7f3a8419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 -- 172.21.15.44:0/3807999430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a840fffe0 msgr2=0x7f3a84196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a840fffe0 0x7f3a84196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 -- 172.21.15.44:0/3807999430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009560 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.686+0000 7f3a827fc640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 0x7f3a841967c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800d950 tx=0x7f3a7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.687+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78009c30 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.687+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a78009d90 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.687+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8419b780 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.687+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800f620 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.687+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a841a3040 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.688+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a841014c0 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.689+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3a780109f0 con 0x7f3a84100990 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.689+0000 7f3a6bfff640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3a6c0a1210 0x7f3a6c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.690+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f3a780cc9c0 con 0x7f3a84100990 2024-09-13T17:26:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.690+0000 7f3a82ffd640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3a6c0a1210 0x7f3a6c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.690+0000 7f3a82ffd640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3a6c0a1210 0x7f3a6c0a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006010 tx=0x7f3a74005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.694+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3a78095c50 con 0x7f3a84100990 2024-09-13T17:26:02.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.863+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a84197680 con 0x7f3a6c0a1210 2024-09-13T17:26:02.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.874+0000 7f3a6bfff640 1 -- 172.21.15.44:0/3807999430 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427442 (secure 0 0 0) 0x7f3a84197680 con 0x7f3a6c0a1210 2024-09-13T17:26:02.881 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:26:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3a6c0a1210 msgr2=0x7f3a6c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3a6c0a1210 0x7f3a6c0a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006010 tx=0x7f3a74005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 msgr2=0x7f3a841967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 0x7f3a841967c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800d950 tx=0x7f3a7800de10 comp rx=0 tx=0).stop 2024-09-13T17:26:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 shutdown_connections 2024-09-13T17:26:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f3a6c0a1210 0x7f3a6c0a36d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a84105110 0x7f3a8419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a84100990 0x7f3a841967c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 --2- 172.21.15.44:0/3807999430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a840fffe0 0x7f3a84196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 >> 172.21.15.44:0/3807999430 conn(0x7f3a840fbae0 msgr2=0x7f3a840fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.878+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 shutdown_connections 2024-09-13T17:26:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:02.879+0000 7f3a893f5640 1 -- 172.21.15.44:0/3807999430 wait complete. 2024-09-13T17:26:03.549 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-13T17:26:01.579649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656795,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":600,"num_read_kb":423,"num_write":341,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385768,"kb_used_data":12344,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124556056,"statfs":{"total":1151940427776,"available":1151545401344,"internally_reserved":0,"allocated":12640256,"data_stored":8127554,"data_compressed":391110,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003259"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048059+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:25:42.048059+0000","last_peered":"2024-09-13T17:25:42.048059+0000","last_clean":"2024-09-13T17:25:42.048059+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:25:42.048059+0000","last_undegraded":"2024-09-13T17:25:42.048059+0000","last_fullsized":"2024-09-13T17:25:42.048059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053730+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:25:42.053730+0000","last_peered":"2024-09-13T17:25:42.053730+0000","last_clean":"2024-09-13T17:25:42.053730+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:25:42.053730+0000","last_undegraded":"2024-09-13T17:25:42.053730+0000","last_fullsized":"2024-09-13T17:25:42.053730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048684+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:25:42.048684+0000","last_peered":"2024-09-13T17:25:42.048684+0000","last_clean":"2024-09-13T17:25:42.048684+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:25:42.048684+0000","last_undegraded":"2024-09-13T17:25:42.048684+0000","last_fullsized":"2024-09-13T17:25:42.048684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053847+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:25:42.053847+0000","last_peered":"2024-09-13T17:25:42.053847+0000","last_clean":"2024-09-13T17:25:42.053847+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:25:42.053847+0000","last_undegraded":"2024-09-13T17:25:42.053847+0000","last_fullsized":"2024-09-13T17:25:42.053847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443551+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:25:42.443551+0000","last_peered":"2024-09-13T17:25:42.443551+0000","last_clean":"2024-09-13T17:25:42.443551+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:25:42.443551+0000","last_undegraded":"2024-09-13T17:25:42.443551+0000","last_fullsized":"2024-09-13T17:25:42.443551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048486+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:25:42.048486+0000","last_peered":"2024-09-13T17:25:42.048486+0000","last_clean":"2024-09-13T17:25:42.048486+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:25:42.048486+0000","last_undegraded":"2024-09-13T17:25:42.048486+0000","last_fullsized":"2024-09-13T17:25:42.048486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.062868+0000","last_change":"2024-09-13T17:25:43.062483+0000","last_active":"2024-09-13T17:25:43.062868+0000","last_peered":"2024-09-13T17:25:43.062868+0000","last_clean":"2024-09-13T17:25:43.062868+0000","last_became_active":"2024-09-13T17:25:43.058145+0000","last_became_peered":"2024-09-13T17:25:43.058145+0000","last_unstale":"2024-09-13T17:25:43.062868+0000","last_undegraded":"2024-09-13T17:25:43.062868+0000","last_fullsized":"2024-09-13T17:25:43.062868+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:10.848768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054176+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:25:42.054176+0000","last_peered":"2024-09-13T17:25:42.054176+0000","last_clean":"2024-09-13T17:25:42.054176+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:25:42.054176+0000","last_undegraded":"2024-09-13T17:25:42.054176+0000","last_fullsized":"2024-09-13T17:25:42.054176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048684+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:42.048684+0000","last_peered":"2024-09-13T17:25:42.048684+0000","last_clean":"2024-09-13T17:25:42.048684+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:25:42.048684+0000","last_undegraded":"2024-09-13T17:25:42.048684+0000","last_fullsized":"2024-09-13T17:25:42.048684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053857+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:25:42.053857+0000","last_peered":"2024-09-13T17:25:42.053857+0000","last_clean":"2024-09-13T17:25:42.053857+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:25:42.053857+0000","last_undegraded":"2024-09-13T17:25:42.053857+0000","last_fullsized":"2024-09-13T17:25:42.053857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233291+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:25:42.233291+0000","last_peered":"2024-09-13T17:25:42.233291+0000","last_clean":"2024-09-13T17:25:42.233291+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:25:42.233291+0000","last_undegraded":"2024-09-13T17:25:42.233291+0000","last_fullsized":"2024-09-13T17:25:42.233291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048504+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:25:42.048504+0000","last_peered":"2024-09-13T17:25:42.048504+0000","last_clean":"2024-09-13T17:25:42.048504+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:25:42.048504+0000","last_undegraded":"2024-09-13T17:25:42.048504+0000","last_fullsized":"2024-09-13T17:25:42.048504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053492+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:25:42.053492+0000","last_peered":"2024-09-13T17:25:42.053492+0000","last_clean":"2024-09-13T17:25:42.053492+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:25:42.053492+0000","last_undegraded":"2024-09-13T17:25:42.053492+0000","last_fullsized":"2024-09-13T17:25:42.053492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053607+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:25:42.053607+0000","last_peered":"2024-09-13T17:25:42.053607+0000","last_clean":"2024-09-13T17:25:42.053607+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:25:42.053607+0000","last_undegraded":"2024-09-13T17:25:42.053607+0000","last_fullsized":"2024-09-13T17:25:42.053607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048930+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:25:42.048930+0000","last_peered":"2024-09-13T17:25:42.048930+0000","last_clean":"2024-09-13T17:25:42.048930+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:25:42.048930+0000","last_undegraded":"2024-09-13T17:25:42.048930+0000","last_fullsized":"2024-09-13T17:25:42.048930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053323+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:25:42.053323+0000","last_peered":"2024-09-13T17:25:42.053323+0000","last_clean":"2024-09-13T17:25:42.053323+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:25:42.053323+0000","last_undegraded":"2024-09-13T17:25:42.053323+0000","last_fullsized":"2024-09-13T17:25:42.053323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048858+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:25:42.048858+0000","last_peered":"2024-09-13T17:25:42.048858+0000","last_clean":"2024-09-13T17:25:42.048858+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:25:42.048858+0000","last_undegraded":"2024-09-13T17:25:42.048858+0000","last_fullsized":"2024-09-13T17:25:42.048858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048745+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:25:42.048745+0000","last_peered":"2024-09-13T17:25:42.048745+0000","last_clean":"2024-09-13T17:25:42.048745+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:25:42.048745+0000","last_undegraded":"2024-09-13T17:25:42.048745+0000","last_fullsized":"2024-09-13T17:25:42.048745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051895+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:25:42.051895+0000","last_peered":"2024-09-13T17:25:42.051895+0000","last_clean":"2024-09-13T17:25:42.051895+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:25:42.051895+0000","last_undegraded":"2024-09-13T17:25:42.051895+0000","last_fullsized":"2024-09-13T17:25:42.051895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053394+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:25:42.053394+0000","last_peered":"2024-09-13T17:25:42.053394+0000","last_clean":"2024-09-13T17:25:42.053394+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:25:42.053394+0000","last_undegraded":"2024-09-13T17:25:42.053394+0000","last_fullsized":"2024-09-13T17:25:42.053394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047948+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:25:42.047948+0000","last_peered":"2024-09-13T17:25:42.047948+0000","last_clean":"2024-09-13T17:25:42.047948+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:25:42.047948+0000","last_undegraded":"2024-09-13T17:25:42.047948+0000","last_fullsized":"2024-09-13T17:25:42.047948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053348+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:25:42.053348+0000","last_peered":"2024-09-13T17:25:42.053348+0000","last_clean":"2024-09-13T17:25:42.053348+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:25:42.053348+0000","last_undegraded":"2024-09-13T17:25:42.053348+0000","last_fullsized":"2024-09-13T17:25:42.053348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053613+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:25:42.053613+0000","last_peered":"2024-09-13T17:25:42.053613+0000","last_clean":"2024-09-13T17:25:42.053613+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:25:42.053613+0000","last_undegraded":"2024-09-13T17:25:42.053613+0000","last_fullsized":"2024-09-13T17:25:42.053613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048853+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:25:42.048853+0000","last_peered":"2024-09-13T17:25:42.048853+0000","last_clean":"2024-09-13T17:25:42.048853+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:25:42.048853+0000","last_undegraded":"2024-09-13T17:25:42.048853+0000","last_fullsized":"2024-09-13T17:25:42.048853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443378+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:25:42.443378+0000","last_peered":"2024-09-13T17:25:42.443378+0000","last_clean":"2024-09-13T17:25:42.443378+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:25:42.443378+0000","last_undegraded":"2024-09-13T17:25:42.443378+0000","last_fullsized":"2024-09-13T17:25:42.443378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049117+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:25:42.049117+0000","last_peered":"2024-09-13T17:25:42.049117+0000","last_clean":"2024-09-13T17:25:42.049117+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:25:42.049117+0000","last_undegraded":"2024-09-13T17:25:42.049117+0000","last_fullsized":"2024-09-13T17:25:42.049117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048693+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:25:42.048693+0000","last_peered":"2024-09-13T17:25:42.048693+0000","last_clean":"2024-09-13T17:25:42.048693+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:25:42.048693+0000","last_undegraded":"2024-09-13T17:25:42.048693+0000","last_fullsized":"2024-09-13T17:25:42.048693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053779+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:25:42.053779+0000","last_peered":"2024-09-13T17:25:42.053779+0000","last_clean":"2024-09-13T17:25:42.053779+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:25:42.053779+0000","last_undegraded":"2024-09-13T17:25:42.053779+0000","last_fullsized":"2024-09-13T17:25:42.053779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048127+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:25:42.048127+0000","last_peered":"2024-09-13T17:25:42.048127+0000","last_clean":"2024-09-13T17:25:42.048127+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:25:42.048127+0000","last_undegraded":"2024-09-13T17:25:42.048127+0000","last_fullsized":"2024-09-13T17:25:42.048127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053099+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:25:42.053099+0000","last_peered":"2024-09-13T17:25:42.053099+0000","last_clean":"2024-09-13T17:25:42.053099+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:25:42.053099+0000","last_undegraded":"2024-09-13T17:25:42.053099+0000","last_fullsized":"2024-09-13T17:25:42.053099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051478+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:25:42.051478+0000","last_peered":"2024-09-13T17:25:42.051478+0000","last_clean":"2024-09-13T17:25:42.051478+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:25:42.051478+0000","last_undegraded":"2024-09-13T17:25:42.051478+0000","last_fullsized":"2024-09-13T17:25:42.051478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052259+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:25:42.052259+0000","last_peered":"2024-09-13T17:25:42.052259+0000","last_clean":"2024-09-13T17:25:42.052259+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:25:42.052259+0000","last_undegraded":"2024-09-13T17:25:42.052259+0000","last_fullsized":"2024-09-13T17:25:42.052259+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052005+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:25:42.052005+0000","last_peered":"2024-09-13T17:25:42.052005+0000","last_clean":"2024-09-13T17:25:42.052005+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:25:42.052005+0000","last_undegraded":"2024-09-13T17:25:42.052005+0000","last_fullsized":"2024-09-13T17:25:42.052005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229783+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:25:42.229783+0000","last_peered":"2024-09-13T17:25:42.229783+0000","last_clean":"2024-09-13T17:25:42.229783+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:25:42.229783+0000","last_undegraded":"2024-09-13T17:25:42.229783+0000","last_fullsized":"2024-09-13T17:25:42.229783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047876+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:25:42.047876+0000","last_peered":"2024-09-13T17:25:42.047876+0000","last_clean":"2024-09-13T17:25:42.047876+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:25:42.047876+0000","last_undegraded":"2024-09-13T17:25:42.047876+0000","last_fullsized":"2024-09-13T17:25:42.047876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053350+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:25:42.053350+0000","last_peered":"2024-09-13T17:25:42.053350+0000","last_clean":"2024-09-13T17:25:42.053350+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:25:42.053350+0000","last_undegraded":"2024-09-13T17:25:42.053350+0000","last_fullsized":"2024-09-13T17:25:42.053350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229716+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:25:42.229716+0000","last_peered":"2024-09-13T17:25:42.229716+0000","last_clean":"2024-09-13T17:25:42.229716+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:25:42.229716+0000","last_undegraded":"2024-09-13T17:25:42.229716+0000","last_fullsized":"2024-09-13T17:25:42.229716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053579+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:25:42.053579+0000","last_peered":"2024-09-13T17:25:42.053579+0000","last_clean":"2024-09-13T17:25:42.053579+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:25:42.053579+0000","last_undegraded":"2024-09-13T17:25:42.053579+0000","last_fullsized":"2024-09-13T17:25:42.053579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051383+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:25:42.051383+0000","last_peered":"2024-09-13T17:25:42.051383+0000","last_clean":"2024-09-13T17:25:42.051383+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:25:42.051383+0000","last_undegraded":"2024-09-13T17:25:42.051383+0000","last_fullsized":"2024-09-13T17:25:42.051383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232793+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:25:42.232793+0000","last_peered":"2024-09-13T17:25:42.232793+0000","last_clean":"2024-09-13T17:25:42.232793+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:25:42.232793+0000","last_undegraded":"2024-09-13T17:25:42.232793+0000","last_fullsized":"2024-09-13T17:25:42.232793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:56.270839+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:25:56.270839+0000","last_peered":"2024-09-13T17:25:56.270839+0000","last_clean":"2024-09-13T17:25:56.270839+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:25:56.270839+0000","last_undegraded":"2024-09-13T17:25:56.270839+0000","last_fullsized":"2024-09-13T17:25:56.270839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051450+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:25:42.051450+0000","last_peered":"2024-09-13T17:25:42.051450+0000","last_clean":"2024-09-13T17:25:42.051450+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:25:42.051450+0000","last_undegraded":"2024-09-13T17:25:42.051450+0000","last_fullsized":"2024-09-13T17:25:42.051450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051026+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:25:42.051026+0000","last_peered":"2024-09-13T17:25:42.051026+0000","last_clean":"2024-09-13T17:25:42.051026+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:25:42.051026+0000","last_undegraded":"2024-09-13T17:25:42.051026+0000","last_fullsized":"2024-09-13T17:25:42.051026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053611+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:25:42.053611+0000","last_peered":"2024-09-13T17:25:42.053611+0000","last_clean":"2024-09-13T17:25:42.053611+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:25:42.053611+0000","last_undegraded":"2024-09-13T17:25:42.053611+0000","last_fullsized":"2024-09-13T17:25:42.053611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232974+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:25:42.232974+0000","last_peered":"2024-09-13T17:25:42.232974+0000","last_clean":"2024-09-13T17:25:42.232974+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:25:42.232974+0000","last_undegraded":"2024-09-13T17:25:42.232974+0000","last_fullsized":"2024-09-13T17:25:42.232974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053676+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:25:42.053676+0000","last_peered":"2024-09-13T17:25:42.053676+0000","last_clean":"2024-09-13T17:25:42.053676+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:25:42.053676+0000","last_undegraded":"2024-09-13T17:25:42.053676+0000","last_fullsized":"2024-09-13T17:25:42.053676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048854+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:25:42.048854+0000","last_peered":"2024-09-13T17:25:42.048854+0000","last_clean":"2024-09-13T17:25:42.048854+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:25:42.048854+0000","last_undegraded":"2024-09-13T17:25:42.048854+0000","last_fullsized":"2024-09-13T17:25:42.048854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056602+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:25:43.056602+0000","last_peered":"2024-09-13T17:25:43.056602+0000","last_clean":"2024-09-13T17:25:43.056602+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:25:43.056602+0000","last_undegraded":"2024-09-13T17:25:43.056602+0000","last_fullsized":"2024-09-13T17:25:43.056602+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054029+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:25:42.054029+0000","last_peered":"2024-09-13T17:25:42.054029+0000","last_clean":"2024-09-13T17:25:42.054029+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:25:42.054029+0000","last_undegraded":"2024-09-13T17:25:42.054029+0000","last_fullsized":"2024-09-13T17:25:42.054029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054201+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:25:42.054201+0000","last_peered":"2024-09-13T17:25:42.054201+0000","last_clean":"2024-09-13T17:25:42.054201+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:25:42.054201+0000","last_undegraded":"2024-09-13T17:25:42.054201+0000","last_fullsized":"2024-09-13T17:25:42.054201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049166+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:25:42.049166+0000","last_peered":"2024-09-13T17:25:42.049166+0000","last_clean":"2024-09-13T17:25:42.049166+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:25:42.049166+0000","last_undegraded":"2024-09-13T17:25:42.049166+0000","last_fullsized":"2024-09-13T17:25:42.049166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053830+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:25:42.053830+0000","last_peered":"2024-09-13T17:25:42.053830+0000","last_clean":"2024-09-13T17:25:42.053830+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:25:42.053830+0000","last_undegraded":"2024-09-13T17:25:42.053830+0000","last_fullsized":"2024-09-13T17:25:42.053830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054247+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:25:42.054247+0000","last_peered":"2024-09-13T17:25:42.054247+0000","last_clean":"2024-09-13T17:25:42.054247+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:25:42.054247+0000","last_undegraded":"2024-09-13T17:25:42.054247+0000","last_fullsized":"2024-09-13T17:25:42.054247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233410+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:25:42.233410+0000","last_peered":"2024-09-13T17:25:42.233410+0000","last_clean":"2024-09-13T17:25:42.233410+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:25:42.233410+0000","last_undegraded":"2024-09-13T17:25:42.233410+0000","last_fullsized":"2024-09-13T17:25:42.233410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443172+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:25:42.443172+0000","last_peered":"2024-09-13T17:25:42.443172+0000","last_clean":"2024-09-13T17:25:42.443172+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:25:42.443172+0000","last_undegraded":"2024-09-13T17:25:42.443172+0000","last_fullsized":"2024-09-13T17:25:42.443172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054003+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:25:42.054003+0000","last_peered":"2024-09-13T17:25:42.054003+0000","last_clean":"2024-09-13T17:25:42.054003+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:25:42.054003+0000","last_undegraded":"2024-09-13T17:25:42.054003+0000","last_fullsized":"2024-09-13T17:25:42.054003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.395166+0000","last_change":"2024-09-13T17:25:43.395166+0000","last_active":"2024-09-13T17:25:43.395166+0000","last_peered":"2024-09-13T17:25:43.395166+0000","last_clean":"2024-09-13T17:25:43.395166+0000","last_became_active":"2024-09-13T17:25:43.392868+0000","last_became_peered":"2024-09-13T17:25:43.392868+0000","last_unstale":"2024-09-13T17:25:43.395166+0000","last_undegraded":"2024-09-13T17:25:43.395166+0000","last_fullsized":"2024-09-13T17:25:43.395166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:24.860809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232528+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:25:42.232528+0000","last_peered":"2024-09-13T17:25:42.232528+0000","last_clean":"2024-09-13T17:25:42.232528+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:25:42.232528+0000","last_undegraded":"2024-09-13T17:25:42.232528+0000","last_fullsized":"2024-09-13T17:25:42.232528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052449+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:25:42.052449+0000","last_peered":"2024-09-13T17:25:42.052449+0000","last_clean":"2024-09-13T17:25:42.052449+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:25:42.052449+0000","last_undegraded":"2024-09-13T17:25:42.052449+0000","last_fullsized":"2024-09-13T17:25:42.052449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054210+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:25:42.054210+0000","last_peered":"2024-09-13T17:25:42.054210+0000","last_clean":"2024-09-13T17:25:42.054210+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:25:42.054210+0000","last_undegraded":"2024-09-13T17:25:42.054210+0000","last_fullsized":"2024-09-13T17:25:42.054210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232619+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:25:42.232619+0000","last_peered":"2024-09-13T17:25:42.232619+0000","last_clean":"2024-09-13T17:25:42.232619+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:25:42.232619+0000","last_undegraded":"2024-09-13T17:25:42.232619+0000","last_fullsized":"2024-09-13T17:25:42.232619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232940+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:25:42.232940+0000","last_peered":"2024-09-13T17:25:42.232940+0000","last_clean":"2024-09-13T17:25:42.232940+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:25:42.232940+0000","last_undegraded":"2024-09-13T17:25:42.232940+0000","last_fullsized":"2024-09-13T17:25:42.232940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053355+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:25:42.053355+0000","last_peered":"2024-09-13T17:25:42.053355+0000","last_clean":"2024-09-13T17:25:42.053355+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:25:42.053355+0000","last_undegraded":"2024-09-13T17:25:42.053355+0000","last_fullsized":"2024-09-13T17:25:42.053355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051553+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:25:42.051553+0000","last_peered":"2024-09-13T17:25:42.051553+0000","last_clean":"2024-09-13T17:25:42.051553+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:25:42.051553+0000","last_undegraded":"2024-09-13T17:25:42.051553+0000","last_fullsized":"2024-09-13T17:25:42.051553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053263+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:25:42.053263+0000","last_peered":"2024-09-13T17:25:42.053263+0000","last_clean":"2024-09-13T17:25:42.053263+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:25:42.053263+0000","last_undegraded":"2024-09-13T17:25:42.053263+0000","last_fullsized":"2024-09-13T17:25:42.053263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:55.269612+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:25:55.269612+0000","last_peered":"2024-09-13T17:25:55.269612+0000","last_clean":"2024-09-13T17:25:55.269612+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:25:55.269612+0000","last_undegraded":"2024-09-13T17:25:55.269612+0000","last_fullsized":"2024-09-13T17:25:55.269612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054061+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:25:42.054061+0000","last_peered":"2024-09-13T17:25:42.054061+0000","last_clean":"2024-09-13T17:25:42.054061+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:25:42.054061+0000","last_undegraded":"2024-09-13T17:25:42.054061+0000","last_fullsized":"2024-09-13T17:25:42.054061+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053677+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:25:42.053677+0000","last_peered":"2024-09-13T17:25:42.053677+0000","last_clean":"2024-09-13T17:25:42.053677+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:25:42.053677+0000","last_undegraded":"2024-09-13T17:25:42.053677+0000","last_fullsized":"2024-09-13T17:25:42.053677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233293+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:25:42.233293+0000","last_peered":"2024-09-13T17:25:42.233293+0000","last_clean":"2024-09-13T17:25:42.233293+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:25:42.233293+0000","last_undegraded":"2024-09-13T17:25:42.233293+0000","last_fullsized":"2024-09-13T17:25:42.233293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051954+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:42.051954+0000","last_peered":"2024-09-13T17:25:42.051954+0000","last_clean":"2024-09-13T17:25:42.051954+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:25:42.051954+0000","last_undegraded":"2024-09-13T17:25:42.051954+0000","last_fullsized":"2024-09-13T17:25:42.051954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051222+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:25:42.051222+0000","last_peered":"2024-09-13T17:25:42.051222+0000","last_clean":"2024-09-13T17:25:42.051222+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:25:42.051222+0000","last_undegraded":"2024-09-13T17:25:42.051222+0000","last_fullsized":"2024-09-13T17:25:42.051222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048933+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:25:42.048933+0000","last_peered":"2024-09-13T17:25:42.048933+0000","last_clean":"2024-09-13T17:25:42.048933+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:25:42.048933+0000","last_undegraded":"2024-09-13T17:25:42.048933+0000","last_fullsized":"2024-09-13T17:25:42.048933+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233055+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:25:42.233055+0000","last_peered":"2024-09-13T17:25:42.233055+0000","last_clean":"2024-09-13T17:25:42.233055+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:25:42.233055+0000","last_undegraded":"2024-09-13T17:25:42.233055+0000","last_fullsized":"2024-09-13T17:25:42.233055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053907+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:25:42.053907+0000","last_peered":"2024-09-13T17:25:42.053907+0000","last_clean":"2024-09-13T17:25:42.053907+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:25:42.053907+0000","last_undegraded":"2024-09-13T17:25:42.053907+0000","last_fullsized":"2024-09-13T17:25:42.053907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":209,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:59.259897+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:25:59.259897+0000","last_peered":"2024-09-13T17:25:59.259897+0000","last_clean":"2024-09-13T17:25:59.259897+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:25:59.259897+0000","last_undegraded":"2024-09-13T17:25:59.259897+0000","last_fullsized":"2024-09-13T17:25:59.259897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62287,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":156,"num_read_kb":140,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047639+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:25:42.047639+0000","last_peered":"2024-09-13T17:25:42.047639+0000","last_clean":"2024-09-13T17:25:42.047639+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:25:42.047639+0000","last_undegraded":"2024-09-13T17:25:42.047639+0000","last_fullsized":"2024-09-13T17:25:42.047639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054093+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:25:42.054093+0000","last_peered":"2024-09-13T17:25:42.054093+0000","last_clean":"2024-09-13T17:25:42.054093+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:25:42.054093+0000","last_undegraded":"2024-09-13T17:25:42.054093+0000","last_fullsized":"2024-09-13T17:25:42.054093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233421+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:25:42.233421+0000","last_peered":"2024-09-13T17:25:42.233421+0000","last_clean":"2024-09-13T17:25:42.233421+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:25:42.233421+0000","last_undegraded":"2024-09-13T17:25:42.233421+0000","last_fullsized":"2024-09-13T17:25:42.233421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048574+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:25:42.048574+0000","last_peered":"2024-09-13T17:25:42.048574+0000","last_clean":"2024-09-13T17:25:42.048574+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:25:42.048574+0000","last_undegraded":"2024-09-13T17:25:42.048574+0000","last_fullsized":"2024-09-13T17:25:42.048574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443590+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:25:42.443590+0000","last_peered":"2024-09-13T17:25:42.443590+0000","last_clean":"2024-09-13T17:25:42.443590+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:25:42.443590+0000","last_undegraded":"2024-09-13T17:25:42.443590+0000","last_fullsized":"2024-09-13T17:25:42.443590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054180+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:25:42.054180+0000","last_peered":"2024-09-13T17:25:42.054180+0000","last_clean":"2024-09-13T17:25:42.054180+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:25:42.054180+0000","last_undegraded":"2024-09-13T17:25:42.054180+0000","last_fullsized":"2024-09-13T17:25:42.054180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047490+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:25:42.047490+0000","last_peered":"2024-09-13T17:25:42.047490+0000","last_clean":"2024-09-13T17:25:42.047490+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:25:42.047490+0000","last_undegraded":"2024-09-13T17:25:42.047490+0000","last_fullsized":"2024-09-13T17:25:42.047490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054352+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:25:42.054352+0000","last_peered":"2024-09-13T17:25:42.054352+0000","last_clean":"2024-09-13T17:25:42.054352+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:25:42.054352+0000","last_undegraded":"2024-09-13T17:25:42.054352+0000","last_fullsized":"2024-09-13T17:25:42.054352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:58.258522+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:25:58.258522+0000","last_peered":"2024-09-13T17:25:58.258522+0000","last_clean":"2024-09-13T17:25:58.258522+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:25:58.258522+0000","last_undegraded":"2024-09-13T17:25:58.258522+0000","last_fullsized":"2024-09-13T17:25:58.258522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053237+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:25:42.053237+0000","last_peered":"2024-09-13T17:25:42.053237+0000","last_clean":"2024-09-13T17:25:42.053237+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:25:42.053237+0000","last_undegraded":"2024-09-13T17:25:42.053237+0000","last_fullsized":"2024-09-13T17:25:42.053237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443326+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:25:42.443326+0000","last_peered":"2024-09-13T17:25:42.443326+0000","last_clean":"2024-09-13T17:25:42.443326+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:25:42.443326+0000","last_undegraded":"2024-09-13T17:25:42.443326+0000","last_fullsized":"2024-09-13T17:25:42.443326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047633+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:25:42.047633+0000","last_peered":"2024-09-13T17:25:42.047633+0000","last_clean":"2024-09-13T17:25:42.047633+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:25:42.047633+0000","last_undegraded":"2024-09-13T17:25:42.047633+0000","last_fullsized":"2024-09-13T17:25:42.047633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233208+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:25:42.233208+0000","last_peered":"2024-09-13T17:25:42.233208+0000","last_clean":"2024-09-13T17:25:42.233208+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:25:42.233208+0000","last_undegraded":"2024-09-13T17:25:42.233208+0000","last_fullsized":"2024-09-13T17:25:42.233208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052502+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:25:42.052502+0000","last_peered":"2024-09-13T17:25:42.052502+0000","last_clean":"2024-09-13T17:25:42.052502+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:25:42.052502+0000","last_undegraded":"2024-09-13T17:25:42.052502+0000","last_fullsized":"2024-09-13T17:25:42.052502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053216+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:25:42.053216+0000","last_peered":"2024-09-13T17:25:42.053216+0000","last_clean":"2024-09-13T17:25:42.053216+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:25:42.053216+0000","last_undegraded":"2024-09-13T17:25:42.053216+0000","last_fullsized":"2024-09-13T17:25:42.053216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051043+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:25:42.051043+0000","last_peered":"2024-09-13T17:25:42.051043+0000","last_clean":"2024-09-13T17:25:42.051043+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:25:42.051043+0000","last_undegraded":"2024-09-13T17:25:42.051043+0000","last_fullsized":"2024-09-13T17:25:42.051043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229612+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:25:42.229612+0000","last_peered":"2024-09-13T17:25:42.229612+0000","last_clean":"2024-09-13T17:25:42.229612+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:25:42.229612+0000","last_undegraded":"2024-09-13T17:25:42.229612+0000","last_fullsized":"2024-09-13T17:25:42.229612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.233114+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:25:42.233114+0000","last_peered":"2024-09-13T17:25:42.233114+0000","last_clean":"2024-09-13T17:25:42.233114+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:25:42.233114+0000","last_undegraded":"2024-09-13T17:25:42.233114+0000","last_fullsized":"2024-09-13T17:25:42.233114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048848+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:25:42.048848+0000","last_peered":"2024-09-13T17:25:42.048848+0000","last_clean":"2024-09-13T17:25:42.048848+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:25:42.048848+0000","last_undegraded":"2024-09-13T17:25:42.048848+0000","last_fullsized":"2024-09-13T17:25:42.048848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":101,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:58.259154+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:25:58.259154+0000","last_peered":"2024-09-13T17:25:58.259154+0000","last_clean":"2024-09-13T17:25:58.259154+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:25:58.259154+0000","last_undegraded":"2024-09-13T17:25:58.259154+0000","last_fullsized":"2024-09-13T17:25:58.259154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053747+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:25:42.053747+0000","last_peered":"2024-09-13T17:25:42.053747+0000","last_clean":"2024-09-13T17:25:42.053747+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:25:42.053747+0000","last_undegraded":"2024-09-13T17:25:42.053747+0000","last_fullsized":"2024-09-13T17:25:42.053747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049147+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:25:42.049147+0000","last_peered":"2024-09-13T17:25:42.049147+0000","last_clean":"2024-09-13T17:25:42.049147+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:25:42.049147+0000","last_undegraded":"2024-09-13T17:25:42.049147+0000","last_fullsized":"2024-09-13T17:25:42.049147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051901+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:25:42.051901+0000","last_peered":"2024-09-13T17:25:42.051901+0000","last_clean":"2024-09-13T17:25:42.051901+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:25:42.051901+0000","last_undegraded":"2024-09-13T17:25:42.051901+0000","last_fullsized":"2024-09-13T17:25:42.051901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049119+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:25:42.049119+0000","last_peered":"2024-09-13T17:25:42.049119+0000","last_clean":"2024-09-13T17:25:42.049119+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:25:42.049119+0000","last_undegraded":"2024-09-13T17:25:42.049119+0000","last_fullsized":"2024-09-13T17:25:42.049119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048934+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:25:42.048934+0000","last_peered":"2024-09-13T17:25:42.048934+0000","last_clean":"2024-09-13T17:25:42.048934+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:25:42.048934+0000","last_undegraded":"2024-09-13T17:25:42.048934+0000","last_fullsized":"2024-09-13T17:25:42.048934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443260+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:25:42.443260+0000","last_peered":"2024-09-13T17:25:42.443260+0000","last_clean":"2024-09-13T17:25:42.443260+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:25:42.443260+0000","last_undegraded":"2024-09-13T17:25:42.443260+0000","last_fullsized":"2024-09-13T17:25:42.443260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051383+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:25:42.051383+0000","last_peered":"2024-09-13T17:25:42.051383+0000","last_clean":"2024-09-13T17:25:42.051383+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:25:42.051383+0000","last_undegraded":"2024-09-13T17:25:42.051383+0000","last_fullsized":"2024-09-13T17:25:42.051383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052549+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:25:42.052549+0000","last_peered":"2024-09-13T17:25:42.052549+0000","last_clean":"2024-09-13T17:25:42.052549+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:25:42.052549+0000","last_undegraded":"2024-09-13T17:25:42.052549+0000","last_fullsized":"2024-09-13T17:25:42.052549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049352+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:25:42.049352+0000","last_peered":"2024-09-13T17:25:42.049352+0000","last_clean":"2024-09-13T17:25:42.049352+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:25:42.049352+0000","last_undegraded":"2024-09-13T17:25:42.049352+0000","last_fullsized":"2024-09-13T17:25:42.049352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053653+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:25:42.053653+0000","last_peered":"2024-09-13T17:25:42.053653+0000","last_clean":"2024-09-13T17:25:42.053653+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:25:42.053653+0000","last_undegraded":"2024-09-13T17:25:42.053653+0000","last_fullsized":"2024-09-13T17:25:42.053653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.232744+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:25:42.232744+0000","last_peered":"2024-09-13T17:25:42.232744+0000","last_clean":"2024-09-13T17:25:42.232744+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:25:42.232744+0000","last_undegraded":"2024-09-13T17:25:42.232744+0000","last_fullsized":"2024-09-13T17:25:42.232744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053281+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:25:42.053281+0000","last_peered":"2024-09-13T17:25:42.053281+0000","last_clean":"2024-09-13T17:25:42.053281+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:25:42.053281+0000","last_undegraded":"2024-09-13T17:25:42.053281+0000","last_fullsized":"2024-09-13T17:25:42.053281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053997+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:25:42.053997+0000","last_peered":"2024-09-13T17:25:42.053997+0000","last_clean":"2024-09-13T17:25:42.053997+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:25:42.053997+0000","last_undegraded":"2024-09-13T17:25:42.053997+0000","last_fullsized":"2024-09-13T17:25:42.053997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051462+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:25:42.051462+0000","last_peered":"2024-09-13T17:25:42.051462+0000","last_clean":"2024-09-13T17:25:42.051462+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:25:42.051462+0000","last_undegraded":"2024-09-13T17:25:42.051462+0000","last_fullsized":"2024-09-13T17:25:42.051462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052052+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:25:42.052052+0000","last_peered":"2024-09-13T17:25:42.052052+0000","last_clean":"2024-09-13T17:25:42.052052+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:25:42.052052+0000","last_undegraded":"2024-09-13T17:25:42.052052+0000","last_fullsized":"2024-09-13T17:25:42.052052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.049260+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:25:42.049260+0000","last_peered":"2024-09-13T17:25:42.049260+0000","last_clean":"2024-09-13T17:25:42.049260+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:25:42.049260+0000","last_undegraded":"2024-09-13T17:25:42.049260+0000","last_fullsized":"2024-09-13T17:25:42.049260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048078+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:25:42.048078+0000","last_peered":"2024-09-13T17:25:42.048078+0000","last_clean":"2024-09-13T17:25:42.048078+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:25:42.048078+0000","last_undegraded":"2024-09-13T17:25:42.048078+0000","last_fullsized":"2024-09-13T17:25:42.048078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052670+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:25:42.052670+0000","last_peered":"2024-09-13T17:25:42.052670+0000","last_clean":"2024-09-13T17:25:42.052670+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:25:42.052670+0000","last_undegraded":"2024-09-13T17:25:42.052670+0000","last_fullsized":"2024-09-13T17:25:42.052670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053531+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:25:42.053531+0000","last_peered":"2024-09-13T17:25:42.053531+0000","last_clean":"2024-09-13T17:25:42.053531+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:25:42.053531+0000","last_undegraded":"2024-09-13T17:25:42.053531+0000","last_fullsized":"2024-09-13T17:25:42.053531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443108+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:25:42.443108+0000","last_peered":"2024-09-13T17:25:42.443108+0000","last_clean":"2024-09-13T17:25:42.443108+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:25:42.443108+0000","last_undegraded":"2024-09-13T17:25:42.443108+0000","last_fullsized":"2024-09-13T17:25:42.443108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443031+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:25:42.443031+0000","last_peered":"2024-09-13T17:25:42.443031+0000","last_clean":"2024-09-13T17:25:42.443031+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:25:42.443031+0000","last_undegraded":"2024-09-13T17:25:42.443031+0000","last_fullsized":"2024-09-13T17:25:42.443031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.051385+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:25:42.051385+0000","last_peered":"2024-09-13T17:25:42.051385+0000","last_clean":"2024-09-13T17:25:42.051385+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:25:42.051385+0000","last_undegraded":"2024-09-13T17:25:42.051385+0000","last_fullsized":"2024-09-13T17:25:42.051385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.048893+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:25:42.048893+0000","last_peered":"2024-09-13T17:25:42.048893+0000","last_clean":"2024-09-13T17:25:42.048893+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:25:42.048893+0000","last_undegraded":"2024-09-13T17:25:42.048893+0000","last_fullsized":"2024-09-13T17:25:42.048893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054254+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:25:42.054254+0000","last_peered":"2024-09-13T17:25:42.054254+0000","last_clean":"2024-09-13T17:25:42.054254+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:25:42.054254+0000","last_undegraded":"2024-09-13T17:25:42.054254+0000","last_fullsized":"2024-09-13T17:25:42.054254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229947+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:25:42.229947+0000","last_peered":"2024-09-13T17:25:42.229947+0000","last_clean":"2024-09-13T17:25:42.229947+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:25:42.229947+0000","last_undegraded":"2024-09-13T17:25:42.229947+0000","last_fullsized":"2024-09-13T17:25:42.229947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.054458+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:25:42.054458+0000","last_peered":"2024-09-13T17:25:42.054458+0000","last_clean":"2024-09-13T17:25:42.054458+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:25:42.054458+0000","last_undegraded":"2024-09-13T17:25:42.054458+0000","last_fullsized":"2024-09-13T17:25:42.054458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053189+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:25:42.053189+0000","last_peered":"2024-09-13T17:25:42.053189+0000","last_clean":"2024-09-13T17:25:42.053189+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:25:42.053189+0000","last_undegraded":"2024-09-13T17:25:42.053189+0000","last_fullsized":"2024-09-13T17:25:42.053189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053477+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:25:42.053477+0000","last_peered":"2024-09-13T17:25:42.053477+0000","last_clean":"2024-09-13T17:25:42.053477+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:25:42.053477+0000","last_undegraded":"2024-09-13T17:25:42.053477+0000","last_fullsized":"2024-09-13T17:25:42.053477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.229458+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:25:42.229458+0000","last_peered":"2024-09-13T17:25:42.229458+0000","last_clean":"2024-09-13T17:25:42.229458+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:25:42.229458+0000","last_undegraded":"2024-09-13T17:25:42.229458+0000","last_fullsized":"2024-09-13T17:25:42.229458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.053452+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:25:42.053452+0000","last_peered":"2024-09-13T17:25:42.053452+0000","last_clean":"2024-09-13T17:25:42.053452+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:25:42.053452+0000","last_undegraded":"2024-09-13T17:25:42.053452+0000","last_fullsized":"2024-09-13T17:25:42.053452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.047704+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:25:42.047704+0000","last_peered":"2024-09-13T17:25:42.047704+0000","last_clean":"2024-09-13T17:25:42.047704+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:25:42.047704+0000","last_undegraded":"2024-09-13T17:25:42.047704+0000","last_fullsized":"2024-09-13T17:25:42.047704+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443225+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:25:42.443225+0000","last_peered":"2024-09-13T17:25:42.443225+0000","last_clean":"2024-09-13T17:25:42.443225+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:25:42.443225+0000","last_undegraded":"2024-09-13T17:25:42.443225+0000","last_fullsized":"2024-09-13T17:25:42.443225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.443250+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:25:42.443250+0000","last_peered":"2024-09-13T17:25:42.443250+0000","last_clean":"2024-09-13T17:25:42.443250+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:25:42.443250+0000","last_undegraded":"2024-09-13T17:25:42.443250+0000","last_fullsized":"2024-09-13T17:25:42.443250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T17:25:42.052621+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:25:42.052621+0000","last_peered":"2024-09-13T17:25:42.052621+0000","last_clean":"2024-09-13T17:25:42.052621+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:25:42.052621+0000","last_undegraded":"2024-09-13T17:25:42.052621+0000","last_fullsized":"2024-09-13T17:25:42.052621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66193,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":464,"num_read_kb":304,"num_write":158,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285499,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":515208,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.72299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84999999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68600000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.313,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60699999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81499999999999995,"5min":1.05,"15min":1.05},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88400000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.89800000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92000000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.752}]},{"osd":10,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.73099999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678054,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1159168,"data_stored":1168850,"data_compressed":59819,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.438,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":1.101,"15min":1.101},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74399999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51300000000000001,"15min":0.52200000000000002},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.436}]},{"osd":5,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.44900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.83099999999999996,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.62},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.752,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":1.073,"15min":1.073},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82299999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.505}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72199999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.84399999999999997,"5min":1.147,"15min":1.147},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.53300000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.69999999999999996}]},{"osd":11,"last update":"Fri Sep 13 17:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.71699999999999997},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.79400000000000004,"5min":1.175,"15min":1.175},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.84399999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67500000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":515422,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45500000000000002,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64200000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.435},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.49299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68200000000000005,"5min":0.999,"15min":0.999},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.852,"5min":13.852,"15min":13.852},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":0.88300000000000001,"15min":0.77400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":17.055,"5min":17.055,"15min":17.055},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61399999999999999,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.52600000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57699999999999996,"15min":0.623},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.498},{"interface":"front","average":{"1min":1.21,"5min":0.77700000000000002,"15min":0.71699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.074999999999999,"5min":17.074999999999999,"15min":17.074999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67200000000000004,"15min":0.70899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88700000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.753,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":1.054,"15min":1.054},"last":0.55100000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86499999999999999,"5min":1.083,"15min":1.083},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65200000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.745,"15min":0.76500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92100000000000004,"5min":1.04,"15min":1.04},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.77900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66100000000000003}]},{"osd":11,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.68000000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":514989,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.44900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73999999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.41599999999999998,"15min":0.41799999999999998},"min":{"1min":0.26700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.33800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.40799999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.41399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40200000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46700000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.62},{"interface":"front","average":{"1min":0.75,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.56399999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.64200000000000002,"15min":0.63400000000000001},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.504,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61099999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.73099999999999998,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.44,"15min":0.44},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72799999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63},{"interface":"front","average":{"1min":0.755,"5min":0.66100000000000003,"15min":0.64500000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71599999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68100000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364852,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":1077248,"data_stored":1111156,"data_compressed":59819,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.29899999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.54600000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.56000000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.53800000000000003},"min":{"1min":0.48999999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.47599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.44600000000000001,"15min":0.42899999999999999},"min":{"1min":0.33100000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.67800000000000005,"5min":0.75,"15min":0.75},"last":0.627},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54900000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.82199999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.746},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.71899999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.46700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.621},"min":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Fri Sep 13 17:25:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84999999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86699999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.84999999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.878}]}]},{"osd":6,"up_from":45,"seq":193273528377,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":510598,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.65300000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40300000000000002,"15min":0.41299999999999998},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54100000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.751},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91500000000000004,"5min":1.008,"15min":1.008},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.41599999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.65900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.874,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":1.018,"15min":1.018},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85499999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.70499999999999996},"min":{"1min":0.48599999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.52200000000000002}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.046,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.60199999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.629}]},{"osd":11,"last update":"Fri Sep 13 17:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0780000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1106400,"data_compressed":59819,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.495,"15min":0.47499999999999998},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51900000000000002,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.39100000000000001,"15min":0.35699999999999998},"min":{"1min":0.29199999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.36799999999999999},"min":{"1min":0.307,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.53200000000000003,"15min":0.48399999999999999},"min":{"1min":0.46400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.443,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.56599999999999995,"15min":0.53100000000000003},"min":{"1min":0.47299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84699999999999998,"5min":1.3,"15min":1.3},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.57799999999999996},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":1.244,"15min":1.244},"last":0.748}]},{"osd":7,"last update":"Fri Sep 13 17:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.57599999999999996,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.65700000000000003}]},{"osd":8,"last update":"Fri Sep 13 17:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83699999999999997,"5min":4.327,"15min":4.327},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54600000000000004,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96799999999999997,"5min":1.212,"15min":1.212},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68999999999999995,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.73599999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":1019904,"data_stored":515592,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.34300000000000003},"min":{"1min":0.32300000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.497,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.29699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55700000000000005,"15min":0.5},"min":{"1min":0.48999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83599999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.53200000000000003,"15min":0.502},"min":{"1min":0.45800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.436}]},{"osd":5,"last update":"Fri Sep 13 17:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.498},"min":{"1min":0.38300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77600000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.505},"min":{"1min":0.42199999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.437,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.49,"5min":0.76300000000000001,"15min":0.63},"min":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":14.458,"5min":14.458,"15min":14.458},"last":0.46800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.317,"5min":0.309,"15min":0.309},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51300000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84099999999999997,"5min":0.872,"15min":0.872},"last":0.63},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73799999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.58699999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53500000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63,"15min":0.626},"min":{"1min":0.443,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":1.02,"15min":1.02},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56699999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.753},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79200000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":1019904,"data_stored":510792,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.35699999999999998},"min":{"1min":0.27000000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.36799999999999999},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56499999999999995,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":0.504}]},{"osd":2,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78500000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.753}]},{"osd":4,"last update":"Fri Sep 13 17:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86799999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.44},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.115,"15min":1.115},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.52600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89400000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65500000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92300000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98899999999999999,"5min":1.069,"15min":1.069},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.95099999999999996,"5min":1.05,"15min":1.05},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95099999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73699999999999999}]},{"osd":11,"last update":"Fri Sep 13 17:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.69299999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":1097728,"data_stored":573766,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.496,"15min":0.45200000000000001},"min":{"1min":0.32600000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.47799999999999998},"min":{"1min":0.371,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.35999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.747,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52700000000000002,"15min":0.48799999999999999},"min":{"1min":0.39800000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.53000000000000003,"15min":0.47599999999999998},"min":{"1min":0.42199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46999999999999997,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47299999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.55100000000000005,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.439}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.625,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.747}]},{"osd":10,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.57799999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66200000000000003}]},{"osd":11,"last update":"Fri Sep 13 17:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":1077248,"data_stored":572795,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84499999999999997,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.75}]},{"osd":2,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56899999999999995,"15min":0.51100000000000001},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.39300000000000002},"min":{"1min":0.34000000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.40899999999999997},"min":{"1min":0.33000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73999999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.374,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61499999999999999,"15min":0.56499999999999995},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.623,"15min":0.59699999999999998},"min":{"1min":0.504,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Sep 13 17:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.63300000000000001,"15min":0.60799999999999998},"min":{"1min":0.50800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Sep 13 17:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.58899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98099999999999998,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.57099999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Sep 13 17:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.878},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":511986,"data_compressed":23517,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.45800000000000002},"min":{"1min":0.40400000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55700000000000005,"15min":0.504},"min":{"1min":0.45200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.499},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.71899999999999997,"15min":0.56100000000000005},"min":{"1min":0.39900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.82199999999999995,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.70699999999999996,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":17.844999999999999,"5min":17.844999999999999,"15min":17.844999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60199999999999998,"15min":0.56599999999999995},"min":{"1min":0.439,"5min":0.379,"15min":0.379},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.628},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.40000000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.749}]},{"osd":8,"last update":"Fri Sep 13 17:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.57699999999999996}]},{"osd":9,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89700000000000002,"5min":1.099,"15min":1.099},"last":0.438}]},{"osd":10,"last update":"Fri Sep 13 17:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.51100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.68400000000000005}]},{"osd":11,"last update":"Fri Sep 13 17:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82099999999999995,"15min":0.81799999999999995},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90700000000000003,"5min":1.048,"15min":1.048},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34663,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:26:03.557 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-13T17:26:03.557 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T17:26:03.557 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-13T17:26:03.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-13T17:26:03.766 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:03.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:03 smithi044 ceph-mon[31175]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:03 smithi070 ceph-mon[34502]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:04.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:03 smithi100 ceph-mon[35055]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:04.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:04 smithi044 ceph-mon[31175]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:04.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:04 smithi070 ceph-mon[34502]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:05.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:04 smithi100 ceph-mon[35055]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:05.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 -- 172.21.15.44:0/3421559141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 msgr2=0x7f37640feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:05.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 --2- 172.21.15.44:0/3421559141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37640feea0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3760009900 tx=0x7f376002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:05.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 -- 172.21.15.44:0/3421559141 shutdown_connections 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 --2- 172.21.15.44:0/3421559141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376410c670 0x7f376410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 --2- 172.21.15.44:0/3421559141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640ff470 0x7f376410c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 --2- 172.21.15.44:0/3421559141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37640feea0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 -- 172.21.15.44:0/3421559141 >> 172.21.15.44:0/3421559141 conn(0x7f37640fa9b0 msgr2=0x7f37640fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 -- 172.21.15.44:0/3421559141 shutdown_connections 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.444+0000 7f376b50c640 1 -- 172.21.15.44:0/3421559141 wait complete. 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 Processor -- start 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 -- start start 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37641983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ff470 0x7f3764198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376410c670 0x7f376419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419d320 con 0x7f37640feac0 2024-09-13T17:26:05.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419d490 con 0x7f376410c670 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f376b50c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419d600 con 0x7f37640ff470 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37641983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37641983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34336/0 (socket says 172.21.15.44:34336) 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 -- 172.21.15.44:0/1740720813 learned_addr learned my addr 172.21.15.44:0/1740720813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f3769aa9640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376410c670 0x7f376419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f3768aa7640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ff470 0x7f3764198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 -- 172.21.15.44:0/1740720813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ff470 msgr2=0x7f3764198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ff470 0x7f3764198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 -- 172.21.15.44:0/1740720813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376410c670 msgr2=0x7f376419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376410c670 0x7f376419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f37692a8640 1 -- 172.21.15.44:0/1740720813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009560 con 0x7f37640feac0 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.445+0000 7f3768aa7640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ff470 0x7f3764198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:05.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.446+0000 7f37692a8640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37641983e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37600057c0 tx=0x7f3760032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.446+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760032e10 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.446+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37600046c0 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.446+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003850 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.446+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376419d880 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.446+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37641a4fd0 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.447+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37640687e0 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.448+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3760005210 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.449+0000 7f375a7fc640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1110 0x7f37500a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.449+0000 7f3768aa7640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1110 0x7f37500a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.449+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f37600eddb0 con 0x7f37640feac0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.449+0000 7f3768aa7640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1110 0x7f37500a35d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3754005d00 tx=0x7f3754009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:05.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.453+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f37600b70a0 con 0x7f37640feac0 2024-09-13T17:26:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.661+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3764107830 con 0x7f37640feac0 2024-09-13T17:26:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.661+0000 7f375a7fc640 1 -- 172.21.15.44:0/1740720813 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f37600ba6c0 con 0x7f37640feac0 2024-09-13T17:26:05.663 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:05.663 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T17:26:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1110 msgr2=0x7f37500a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1110 0x7f37500a35d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3754005d00 tx=0x7f3754009550 comp rx=0 tx=0).stop 2024-09-13T17:26:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 msgr2=0x7f37641983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37641983e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37600057c0 tx=0x7f3760032c20 comp rx=0 tx=0).stop 2024-09-13T17:26:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 shutdown_connections 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1110 0x7f37500a35d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ff470 0x7f3764198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376410c670 0x7f376419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 --2- 172.21.15.44:0/1740720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37640feac0 0x7f37641983e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 >> 172.21.15.44:0/1740720813 conn(0x7f37640fa9b0 msgr2=0x7f37640fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 shutdown_connections 2024-09-13T17:26:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:05.663+0000 7f376b50c640 1 -- 172.21.15.44:0/1740720813 wait complete. 2024-09-13T17:26:05.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:05 smithi044 ceph-mon[31175]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:05 smithi070 ceph-mon[34502]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:05 smithi100 ceph-mon[35055]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:06.394 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-13T17:26:06.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.503+0000 7f71eaf08640 1 -- 172.21.15.44:0/3761614770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 msgr2=0x7f71e4102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.503+0000 7f71eaf08640 1 --2- 172.21.15.44:0/3761614770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4102c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- 172.21.15.44:0/3761614770 shutdown_connections 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 --2- 172.21.15.44:0/3761614770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e4105ab0 0x7f71e4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 --2- 172.21.15.44:0/3761614770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71e4103180 0x7f71e4105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 --2- 172.21.15.44:0/3761614770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4102c40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- 172.21.15.44:0/3761614770 >> 172.21.15.44:0/3761614770 conn(0x7f71e40fa430 msgr2=0x7f71e40fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- 172.21.15.44:0/3761614770 shutdown_connections 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- 172.21.15.44:0/3761614770 wait complete. 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 Processor -- start 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- start start 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e4103180 0x7f71e41962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71e4105ab0 0x7f71e419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e419d850 con 0x7f71e4100850 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e419d9c0 con 0x7f71e4105ab0 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.504+0000 7f71eaf08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e419db80 con 0x7f71e4103180 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34344/0 (socket says 172.21.15.44:34344) 2024-09-13T17:26:06.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 -- 172.21.15.44:0/4230990652 learned_addr learned my addr 172.21.15.44:0/4230990652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e94a5640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71e4105ab0 0x7f71e419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e3fff640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e4103180 0x7f71e41962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 -- 172.21.15.44:0/4230990652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e4103180 msgr2=0x7f71e41962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e4103180 0x7f71e41962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 -- 172.21.15.44:0/4230990652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71e4105ab0 msgr2=0x7f71e419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71e4105ab0 0x7f71e419d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 -- 172.21.15.44:0/4230990652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4009560 con 0x7f71e4100850 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e8ca4640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4195d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f71d4005ab0 tx=0x7f71d4005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4031d70 con 0x7f71e4100850 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71d4005510 con 0x7f71e4100850 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e419de00 con 0x7f71e4100850 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.505+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4003b30 con 0x7f71e4100850 2024-09-13T17:26:06.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.506+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e419e220 con 0x7f71e4100850 2024-09-13T17:26:06.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.507+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e40fbf20 con 0x7f71e4100850 2024-09-13T17:26:06.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.508+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f71d4003cf0 con 0x7f71e4100850 2024-09-13T17:26:06.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.508+0000 7f71e1ffb640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71d00a11a0 0x7f71d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.509+0000 7f71e3fff640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71d00a11a0 0x7f71d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.509+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f71d40edfa0 con 0x7f71e4100850 2024-09-13T17:26:06.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.509+0000 7f71e3fff640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71d00a11a0 0x7f71d00a3660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71cc005e10 tx=0x7f71cc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:06.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.513+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f71d40b7290 con 0x7f71e4100850 2024-09-13T17:26:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.685+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71e410b600 con 0x7f71e4100850 2024-09-13T17:26:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.687+0000 7f71e1ffb640 1 -- 172.21.15.44:0/4230990652 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24492 (secure 0 0 0) 0x7f71d40ba8b0 con 0x7f71e4100850 2024-09-13T17:26:06.689 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:06.690 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":92,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:25:43.045876+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71d00a11a0 msgr2=0x7f71d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71d00a11a0 0x7f71d00a3660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71cc005e10 tx=0x7f71cc005da0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 msgr2=0x7f71e4195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4195d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f71d4005ab0 tx=0x7f71d4005320 comp rx=0 tx=0).stop 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 shutdown_connections 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f71d00a11a0 0x7f71d00a3660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e4103180 0x7f71e41962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71e4105ab0 0x7f71e419d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 --2- 172.21.15.44:0/4230990652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e4100850 0x7f71e4195d60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 >> 172.21.15.44:0/4230990652 conn(0x7f71e40fa430 msgr2=0x7f71e4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 shutdown_connections 2024-09-13T17:26:06.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.689+0000 7f71eaf08640 1 -- 172.21.15.44:0/4230990652 wait complete. 2024-09-13T17:26:06.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:06.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.806+0000 7f7b71a80640 1 -- 172.21.15.44:0/4072655800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fec60 msgr2=0x7f7b6c0ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.806+0000 7f7b71a80640 1 --2- 172.21.15.44:0/4072655800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fec60 0x7f7b6c0ff060 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009900 tx=0x7f7b5c02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 -- 172.21.15.44:0/4072655800 shutdown_connections 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 --2- 172.21.15.44:0/4072655800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 --2- 172.21.15.44:0/4072655800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c100810 0x7f7b6c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 --2- 172.21.15.44:0/4072655800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fec60 0x7f7b6c0ff060 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 -- 172.21.15.44:0/4072655800 >> 172.21.15.44:0/4072655800 conn(0x7f7b6c0fa3e0 msgr2=0x7f7b6c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 -- 172.21.15.44:0/4072655800 shutdown_connections 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 -- 172.21.15.44:0/4072655800 wait complete. 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 Processor -- start 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.807+0000 7f7b71a80640 1 -- start start 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c0fec60 0x7f7b6c068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c100810 0x7f7b6c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19d790 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19d900 con 0x7f7b6c0fec60 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19dac0 con 0x7f7b6c100810 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34368/0 (socket says 172.21.15.44:34368) 2024-09-13T17:26:06.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 -- 172.21.15.44:0/1415693274 learned_addr learned my addr 172.21.15.44:0/1415693274 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6b7fe640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c0fec60 0x7f7b6c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6bfff640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c100810 0x7f7b6c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 -- 172.21.15.44:0/1415693274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c100810 msgr2=0x7f7b6c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c100810 0x7f7b6c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 -- 172.21.15.44:0/1415693274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c0fec60 msgr2=0x7f7b6c068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c0fec60 0x7f7b6c068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 -- 172.21.15.44:0/1415693274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009560 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b6affd640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c19a990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c970 tx=0x7f7b6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60007bb0 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b60007d10 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c19dd40 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.808+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60004d70 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.809+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c19e1a0 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.810+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c106190 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.811+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7b60005020 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.811+0000 7f7b68ff9640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7b540a1120 0x7f7b540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:06.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.812+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f7b600ccb20 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.812+0000 7f7b6b7fe640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7b540a1120 0x7f7b540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:06.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.812+0000 7f7b6b7fe640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7b540a1120 0x7f7b540a35e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c02aa00 tx=0x7f7b5c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:06.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.816+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7b60095e10 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:06 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1740720813' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T17:26:06.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:06 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1740720813' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T17:26:06.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.986+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b6c02c3e0 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.987+0000 7f7b68ff9640 1 -- 172.21.15.44:0/1415693274 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b6001d020 con 0x7f7b6c0ffe50 2024-09-13T17:26:06.989 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.988+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7b540a1120 msgr2=0x7f7b540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.988+0000 7f7b71a80640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7b540a1120 0x7f7b540a35e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c02aa00 tx=0x7f7b5c005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.988+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 msgr2=0x7f7b6c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.988+0000 7f7b71a80640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c19a990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c970 tx=0x7f7b6000ce30 comp rx=0 tx=0).stop 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 shutdown_connections 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7b540a1120 0x7f7b540a35e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c100810 0x7f7b6c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0ffe50 0x7f7b6c19a990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 --2- 172.21.15.44:0/1415693274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c0fec60 0x7f7b6c068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 >> 172.21.15.44:0/1415693274 conn(0x7f7b6c0fa3e0 msgr2=0x7f7b6c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 shutdown_connections 2024-09-13T17:26:06.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:06.989+0000 7f7b71a80640 1 -- 172.21.15.44:0/1415693274 wait complete. 2024-09-13T17:26:07.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:07.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:06 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1740720813' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 -- 172.21.15.44:0/2797260332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc105a70 msgr2=0x7f57cc107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 --2- 172.21.15.44:0/2797260332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc105a70 0x7f57cc107ee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009ca0 tx=0x7f57c402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 -- 172.21.15.44:0/2797260332 shutdown_connections 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 --2- 172.21.15.44:0/2797260332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc105a70 0x7f57cc107ee0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 --2- 172.21.15.44:0/2797260332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc103140 0x7f57cc105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 --2- 172.21.15.44:0/2797260332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc100810 0x7f57cc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 -- 172.21.15.44:0/2797260332 >> 172.21.15.44:0/2797260332 conn(0x7f57cc0fa3f0 msgr2=0x7f57cc0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 -- 172.21.15.44:0/2797260332 shutdown_connections 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.105+0000 7f57d386b640 1 -- 172.21.15.44:0/2797260332 wait complete. 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 Processor -- start 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 -- start start 2024-09-13T17:26:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 0x7f57cc195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc103140 0x7f57cc196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc105a70 0x7f57cc19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19d8b0 con 0x7f57cc100810 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19da20 con 0x7f57cc105a70 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d386b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19dbe0 con 0x7f57cc103140 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 0x7f57cc195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 0x7f57cc195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34386/0 (socket says 172.21.15.44:34386) 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 -- 172.21.15.44:0/2437411899 learned_addr learned my addr 172.21.15.44:0/2437411899 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d0e06640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc103140 0x7f57cc196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1e08640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc105a70 0x7f57cc19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 -- 172.21.15.44:0/2437411899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc103140 msgr2=0x7f57cc196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc103140 0x7f57cc196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 -- 172.21.15.44:0/2437411899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc105a70 msgr2=0x7f57cc19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc105a70 0x7f57cc19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1607640 1 -- 172.21.15.44:0/2437411899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009900 con 0x7f57cc100810 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d1e08640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc105a70 0x7f57cc19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.106+0000 7f57d0e06640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc103140 0x7f57cc196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.107+0000 7f57d1607640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 0x7f57cc195dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57c800e9f0 tx=0x7f57c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.107+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8009720 con 0x7f57cc100810 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.107+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57c800cd80 con 0x7f57cc100810 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.107+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc19dec0 con 0x7f57cc100810 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.107+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8010430 con 0x7f57cc100810 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.107+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc19e290 con 0x7f57cc100810 2024-09-13T17:26:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.108+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc103520 con 0x7f57cc100810 2024-09-13T17:26:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.109+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f57c8004f90 con 0x7f57cc100810 2024-09-13T17:26:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.110+0000 7f57c27fc640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f57b80a1120 0x7f57b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.110+0000 7f57d0e06640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f57b80a1120 0x7f57b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.110+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f57c8014070 con 0x7f57cc100810 2024-09-13T17:26:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.110+0000 7f57d0e06640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f57b80a1120 0x7f57b80a35e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57cc1972d0 tx=0x7f57bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.113+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f57c8096770 con 0x7f57cc100810 2024-09-13T17:26:07.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.285+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57cc10b720 con 0x7f57cc100810 2024-09-13T17:26:07.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.286+0000 7f57c27fc640 1 -- 172.21.15.44:0/2437411899 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57c8099d90 con 0x7f57cc100810 2024-09-13T17:26:07.288 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f57b80a1120 msgr2=0x7f57b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f57b80a1120 0x7f57b80a35e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57cc1972d0 tx=0x7f57bc005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 msgr2=0x7f57cc195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 0x7f57cc195dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57c800e9f0 tx=0x7f57c800edf0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 shutdown_connections 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f57b80a1120 0x7f57b80a35e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc103140 0x7f57cc196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57cc105a70 0x7f57cc19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 --2- 172.21.15.44:0/2437411899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57cc100810 0x7f57cc195dc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 >> 172.21.15.44:0/2437411899 conn(0x7f57cc0fa3f0 msgr2=0x7f57cc102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 shutdown_connections 2024-09-13T17:26:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.288+0000 7f57d386b640 1 -- 172.21.15.44:0/2437411899 wait complete. 2024-09-13T17:26:07.301 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:07.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 -- 172.21.15.44:0/2556873145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460100870 msgr2=0x7f5460102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 --2- 172.21.15.44:0/2556873145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460100870 0x7f5460102c60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5450009900 tx=0x7f545002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:07.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 -- 172.21.15.44:0/2556873145 shutdown_connections 2024-09-13T17:26:07.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 --2- 172.21.15.44:0/2556873145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54601031a0 0x7f5460105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 --2- 172.21.15.44:0/2556873145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460105ad0 0x7f5460107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 --2- 172.21.15.44:0/2556873145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460100870 0x7f5460102c60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 -- 172.21.15.44:0/2556873145 >> 172.21.15.44:0/2556873145 conn(0x7f54600fa3d0 msgr2=0x7f54600fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 -- 172.21.15.44:0/2556873145 shutdown_connections 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.404+0000 7f54661bc640 1 -- 172.21.15.44:0/2556873145 wait complete. 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 Processor -- start 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 -- start start 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100870 0x7f5460197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54601031a0 0x7f5460198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 0x7f546019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546019f9e0 con 0x7f5460105ad0 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546019fb50 con 0x7f5460100870 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f54661bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546019fd10 con 0x7f54601031a0 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 0x7f546019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 0x7f546019f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34406/0 (socket says 172.21.15.44:34406) 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 -- 172.21.15.44:0/441250081 learned_addr learned my addr 172.21.15.44:0/441250081 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545f7fe640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100870 0x7f5460197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f5457fff640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54601031a0 0x7f5460198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 -- 172.21.15.44:0/441250081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54601031a0 msgr2=0x7f5460198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54601031a0 0x7f5460198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 -- 172.21.15.44:0/441250081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100870 msgr2=0x7f5460197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100870 0x7f5460197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 -- 172.21.15.44:0/441250081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5450009560 con 0x7f5460105ad0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.405+0000 7f545ffff640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 0x7f546019f4a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f545800e9d0 tx=0x7f545800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.406+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545800cd80 con 0x7f5460105ad0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.406+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5458005020 con 0x7f5460105ad0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.406+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546019fff0 con 0x7f5460105ad0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.406+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580116d0 con 0x7f5460105ad0 2024-09-13T17:26:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.406+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54601a0420 con 0x7f5460105ad0 2024-09-13T17:26:07.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.407+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54601075c0 con 0x7f5460105ad0 2024-09-13T17:26:07.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.408+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5458005190 con 0x7f5460105ad0 2024-09-13T17:26:07.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.408+0000 7f545d7fa640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f544c0a1120 0x7f544c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.408+0000 7f545f7fe640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f544c0a1120 0x7f544c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.408+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f54580ccf80 con 0x7f5460105ad0 2024-09-13T17:26:07.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.409+0000 7f545f7fe640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f544c0a1120 0x7f544c0a35e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f545002aa00 tx=0x7f5450005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.412+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5458096270 con 0x7f5460105ad0 2024-09-13T17:26:07.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.585+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f546002c410 con 0x7f5460105ad0 2024-09-13T17:26:07.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.586+0000 7f545d7fa640 1 -- 172.21.15.44:0/441250081 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f545801f020 con 0x7f5460105ad0 2024-09-13T17:26:07.588 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f544c0a1120 msgr2=0x7f544c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f544c0a1120 0x7f544c0a35e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f545002aa00 tx=0x7f5450005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 msgr2=0x7f546019f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 0x7f546019f4a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f545800e9d0 tx=0x7f545800ee90 comp rx=0 tx=0).stop 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 shutdown_connections 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f544c0a1120 0x7f544c0a35e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54601031a0 0x7f5460198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5460105ad0 0x7f546019f4a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 --2- 172.21.15.44:0/441250081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100870 0x7f5460197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 >> 172.21.15.44:0/441250081 conn(0x7f54600fa3d0 msgr2=0x7f54600feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 shutdown_connections 2024-09-13T17:26:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.588+0000 7f54661bc640 1 -- 172.21.15.44:0/441250081 wait complete. 2024-09-13T17:26:07.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:07 smithi044 ceph-mon[31175]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:07.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:07 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4230990652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:07.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:07 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1415693274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:07.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:07 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2437411899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:07.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:07.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 -- 172.21.15.44:0/2693523038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90181010e0 msgr2=0x7f9018101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 --2- 172.21.15.44:0/2693523038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90181010e0 0x7f9018101560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9000009900 tx=0x7f900002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 -- 172.21.15.44:0/2693523038 shutdown_connections 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 --2- 172.21.15.44:0/2693523038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9018101aa0 0x7f9018108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 --2- 172.21.15.44:0/2693523038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90181010e0 0x7f9018101560 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 --2- 172.21.15.44:0/2693523038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90180fe070 0x7f90180fe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 -- 172.21.15.44:0/2693523038 >> 172.21.15.44:0/2693523038 conn(0x7f90180f9b70 msgr2=0x7f90180fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 -- 172.21.15.44:0/2693523038 shutdown_connections 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.704+0000 7f901d325640 1 -- 172.21.15.44:0/2693523038 wait complete. 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 Processor -- start 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 -- start start 2024-09-13T17:26:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90180fe070 0x7f9018100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181010e0 0x7f90180ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 0x7f90180ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180ffd30 con 0x7f9018101aa0 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180ffea0 con 0x7f90180fe070 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f901d325640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901819fce0 con 0x7f90181010e0 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f90177fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 0x7f90180ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f90177fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 0x7f90180ff5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34410/0 (socket says 172.21.15.44:34410) 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f90177fe640 1 -- 172.21.15.44:0/3508247181 learned_addr learned my addr 172.21.15.44:0/3508247181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.705+0000 7f900ffff640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181010e0 0x7f90180ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f9016ffd640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90180fe070 0x7f9018100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f90177fe640 1 -- 172.21.15.44:0/3508247181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181010e0 msgr2=0x7f90180ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f90177fe640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181010e0 0x7f90180ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f90177fe640 1 -- 172.21.15.44:0/3508247181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90180fe070 msgr2=0x7f9018100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f90177fe640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90180fe070 0x7f9018100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f90177fe640 1 -- 172.21.15.44:0/3508247181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000009560 con 0x7f9018101aa0 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f90177fe640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 0x7f90180ff5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f901000eb40 tx=0x7f901000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010004b20 con 0x7f9018101aa0 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9010004c80 con 0x7f9018101aa0 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901819ff90 con 0x7f9018101aa0 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010010690 con 0x7f9018101aa0 2024-09-13T17:26:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.706+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90181a0410 con 0x7f9018101aa0 2024-09-13T17:26:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.707+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9018107370 con 0x7f9018101aa0 2024-09-13T17:26:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.708+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9010005020 con 0x7f9018101aa0 2024-09-13T17:26:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.710+0000 7f9014ff9640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90040a1120 0x7f90040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.710+0000 7f9016ffd640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90040a1120 0x7f90040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.710+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f9010014070 con 0x7f9018101aa0 2024-09-13T17:26:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.710+0000 7f9016ffd640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90040a1120 0x7f90040a35e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9008005fd0 tx=0x7f9008005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.714+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f90100968f0 con 0x7f9018101aa0 2024-09-13T17:26:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.883+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f901802c410 con 0x7f9018101aa0 2024-09-13T17:26:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.884+0000 7f9014ff9640 1 -- 172.21.15.44:0/3508247181 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9010099f10 con 0x7f9018101aa0 2024-09-13T17:26:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.885+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90040a1120 msgr2=0x7f90040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.885+0000 7f901d325640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90040a1120 0x7f90040a35e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9008005fd0 tx=0x7f9008005850 comp rx=0 tx=0).stop 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.885+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 msgr2=0x7f90180ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.885+0000 7f901d325640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 0x7f90180ff5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f901000eb40 tx=0x7f901000c2d0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 shutdown_connections 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f90040a1120 0x7f90040a35e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181010e0 0x7f90180ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9018101aa0 0x7f90180ff5d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 --2- 172.21.15.44:0/3508247181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90180fe070 0x7f9018100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:07.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 >> 172.21.15.44:0/3508247181 conn(0x7f90180f9b70 msgr2=0x7f9018104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:07.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 shutdown_connections 2024-09-13T17:26:07.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.886+0000 7f901d325640 1 -- 172.21.15.44:0/3508247181 wait complete. 2024-09-13T17:26:07.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:07 smithi070 ceph-mon[34502]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:07 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4230990652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:07 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1415693274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:07 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2437411899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 -- 172.21.15.44:0/3689875340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278103180 msgr2=0x7f4278105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 --2- 172.21.15.44:0/3689875340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278103180 0x7f4278105570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 -- 172.21.15.44:0/3689875340 shutdown_connections 2024-09-13T17:26:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 --2- 172.21.15.44:0/3689875340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4278105ab0 0x7f4278107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 --2- 172.21.15.44:0/3689875340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278103180 0x7f4278105570 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 --2- 172.21.15.44:0/3689875340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4278100850 0x7f4278102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 -- 172.21.15.44:0/3689875340 >> 172.21.15.44:0/3689875340 conn(0x7f42780fa430 msgr2=0x7f42780fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 -- 172.21.15.44:0/3689875340 shutdown_connections 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:07.999+0000 7f427f02b640 1 -- 172.21.15.44:0/3689875340 wait complete. 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 Processor -- start 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 -- start start 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4278100850 0x7f4278195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4278103180 0x7f4278196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 0x7f427819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819d8c0 con 0x7f4278105ab0 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819da30 con 0x7f4278100850 2024-09-13T17:26:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427f02b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819dbf0 con 0x7f4278103180 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 0x7f427819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 0x7f427819d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34422/0 (socket says 172.21.15.44:34422) 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 -- 172.21.15.44:0/108496506 learned_addr learned my addr 172.21.15.44:0/108496506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427cdc7640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4278100850 0x7f4278195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f4277fff640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4278103180 0x7f4278196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 -- 172.21.15.44:0/108496506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4278103180 msgr2=0x7f4278196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4278103180 0x7f4278196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 -- 172.21.15.44:0/108496506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4278100850 msgr2=0x7f4278195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4278100850 0x7f4278195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.000+0000 7f427d5c8640 1 -- 172.21.15.44:0/108496506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4268009560 con 0x7f4278105ab0 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.001+0000 7f427d5c8640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 0x7f427819d380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f427000dac0 tx=0x7f427000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.001+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000be30 con 0x7f4278105ab0 2024-09-13T17:26:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.001+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4270003e00 con 0x7f4278105ab0 2024-09-13T17:26:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.001+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427819ded0 con 0x7f4278105ab0 2024-09-13T17:26:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.001+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42700106c0 con 0x7f4278105ab0 2024-09-13T17:26:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.001+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427819e350 con 0x7f4278105ab0 2024-09-13T17:26:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.002+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42780fbf20 con 0x7f4278105ab0 2024-09-13T17:26:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.003+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4270003970 con 0x7f4278105ab0 2024-09-13T17:26:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.004+0000 7f4275ffb640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42640a11a0 0x7f42640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.004+0000 7f427cdc7640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42640a11a0 0x7f42640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.004+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f42700cc9d0 con 0x7f4278105ab0 2024-09-13T17:26:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.004+0000 7f427cdc7640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42640a11a0 0x7f42640a3660 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4260009bd0 tx=0x7f4260005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.008+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4270095cc0 con 0x7f4278105ab0 2024-09-13T17:26:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:07 smithi100 ceph-mon[35055]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:07 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4230990652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:07 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1415693274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:07 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2437411899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.216+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42780fb990 con 0x7f4278105ab0 2024-09-13T17:26:08.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.216+0000 7f4275ffb640 1 -- 172.21.15.44:0/108496506 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f42700992e0 con 0x7f4278105ab0 2024-09-13T17:26:08.222 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:08.222 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:47.147746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:08.226 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:26:08.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42640a11a0 msgr2=0x7f42640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42640a11a0 0x7f42640a3660 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4260009bd0 tx=0x7f4260005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 msgr2=0x7f427819d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 0x7f427819d380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f427000dac0 tx=0x7f427000df80 comp rx=0 tx=0).stop 2024-09-13T17:26:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 shutdown_connections 2024-09-13T17:26:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42640a11a0 0x7f42640a3660 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4278103180 0x7f4278196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4278105ab0 0x7f427819d380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 --2- 172.21.15.44:0/108496506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4278100850 0x7f4278195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 >> 172.21.15.44:0/108496506 conn(0x7f42780fa430 msgr2=0x7f4278101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 shutdown_connections 2024-09-13T17:26:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.218+0000 7f427f02b640 1 -- 172.21.15.44:0/108496506 wait complete. 2024-09-13T17:26:08.232 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.332+0000 7f376c2e5640 1 -- 172.21.15.44:0/2785595484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 msgr2=0x7f37641070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.332+0000 7f376c2e5640 1 --2- 172.21.15.44:0/2785595484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f37641070e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ca0 tx=0x7f375c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- 172.21.15.44:0/2785595484 shutdown_connections 2024-09-13T17:26:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 --2- 172.21.15.44:0/2785595484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640ffe90 0x7f3764100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 --2- 172.21.15.44:0/2785595484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f37641070e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 --2- 172.21.15.44:0/2785595484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640feca0 0x7f37640ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- 172.21.15.44:0/2785595484 >> 172.21.15.44:0/2785595484 conn(0x7f37640fa440 msgr2=0x7f37640fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- 172.21.15.44:0/2785595484 shutdown_connections 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- 172.21.15.44:0/2785595484 wait complete. 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 Processor -- start 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- start start 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640feca0 0x7f3764195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640ffe90 0x7f3764196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f376419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419d730 con 0x7f3764100850 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419d8a0 con 0x7f37640ffe90 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376c2e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419da60 con 0x7f37640feca0 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376a882640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f376419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376a882640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f376419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34434/0 (socket says 172.21.15.44:34434) 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.333+0000 7f376a882640 1 -- 172.21.15.44:0/1892343223 learned_addr learned my addr 172.21.15.44:0/1892343223 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:08.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a081640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640feca0 0x7f3764195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f3769880640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640ffe90 0x7f3764196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a882640 1 -- 172.21.15.44:0/1892343223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640feca0 msgr2=0x7f3764195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a882640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640feca0 0x7f3764195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a882640 1 -- 172.21.15.44:0/1892343223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640ffe90 msgr2=0x7f3764196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a882640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640ffe90 0x7f3764196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a882640 1 -- 172.21.15.44:0/1892343223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c009900 con 0x7f3764100850 2024-09-13T17:26:08.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376a882640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f376419d1c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f375c02aeb0 tx=0x7f375c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c0307c0 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f375c040070 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376419dce0 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c0043c0 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.334+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376419e1a0 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.335+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3764106180 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.337+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f375c042030 con 0x7f3764100850 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.337+0000 7f375b7fe640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1120 0x7f37500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.337+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f375c0ee400 con 0x7f3764100850 2024-09-13T17:26:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.337+0000 7f376a081640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1120 0x7f37500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.338+0000 7f376a081640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1120 0x7f37500a35e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3764197120 tx=0x7f3760005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.341+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f375c0b7770 con 0x7f3764100850 2024-09-13T17:26:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.512+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f376402c410 con 0x7f3764100850 2024-09-13T17:26:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.514+0000 7f375b7fe640 1 -- 172.21.15.44:0/1892343223 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24492 (secure 0 0 0) 0x7f375c0b96a0 con 0x7f3764100850 2024-09-13T17:26:08.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:08.516 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":92,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:25:43.045876+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1120 msgr2=0x7f37500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1120 0x7f37500a35e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3764197120 tx=0x7f3760005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 msgr2=0x7f376419d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f376419d1c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f375c02aeb0 tx=0x7f375c003c20 comp rx=0 tx=0).stop 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 shutdown_connections 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f37500a1120 0x7f37500a35e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3764100850 0x7f376419d1c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37640ffe90 0x7f3764196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 --2- 172.21.15.44:0/1892343223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37640feca0 0x7f3764195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 >> 172.21.15.44:0/1892343223 conn(0x7f37640fa440 msgr2=0x7f3764103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 shutdown_connections 2024-09-13T17:26:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.516+0000 7f376c2e5640 1 -- 172.21.15.44:0/1892343223 wait complete. 2024-09-13T17:26:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:08.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 -- 172.21.15.44:0/1313052761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 msgr2=0x7fd5e4102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 --2- 172.21.15.44:0/1313052761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4102ca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009900 tx=0x7fd5d402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 -- 172.21.15.44:0/1313052761 shutdown_connections 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 --2- 172.21.15.44:0/1313052761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e41031e0 0x7fd5e41055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 --2- 172.21.15.44:0/1313052761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e4105b10 0x7fd5e4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 --2- 172.21.15.44:0/1313052761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4102ca0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 -- 172.21.15.44:0/1313052761 >> 172.21.15.44:0/1313052761 conn(0x7fd5e40fa3d0 msgr2=0x7fd5e40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 -- 172.21.15.44:0/1313052761 shutdown_connections 2024-09-13T17:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.634+0000 7fd5e9157640 1 -- 172.21.15.44:0/1313052761 wait complete. 2024-09-13T17:26:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 Processor -- start 2024-09-13T17:26:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 -- start start 2024-09-13T17:26:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e41031e0 0x7fd5e4196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e4105b10 0x7fd5e419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e419d7e0 con 0x7fd5e41008b0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e419d950 con 0x7fd5e4105b10 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e9157640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e419db10 con 0x7fd5e41031e0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34442/0 (socket says 172.21.15.44:34442) 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.635+0000 7fd5e2d9d640 1 -- 172.21.15.44:0/49976918 learned_addr learned my addr 172.21.15.44:0/49976918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e359e640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e4105b10 0x7fd5e419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e259c640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e41031e0 0x7fd5e4196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e2d9d640 1 -- 172.21.15.44:0/49976918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e41031e0 msgr2=0x7fd5e4196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e2d9d640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e41031e0 0x7fd5e4196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e2d9d640 1 -- 172.21.15.44:0/49976918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e4105b10 msgr2=0x7fd5e419d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e2d9d640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e4105b10 0x7fd5e419d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e2d9d640 1 -- 172.21.15.44:0/49976918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4009560 con 0x7fd5e41008b0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e2d9d640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4195d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40057c0 tx=0x7fd5d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4031d70 con 0x7fd5e41008b0 2024-09-13T17:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5d40044e0 con 0x7fd5e41008b0 2024-09-13T17:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e419dd90 con 0x7fd5e41008b0 2024-09-13T17:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4003b80 con 0x7fd5e41008b0 2024-09-13T17:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.636+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e419e190 con 0x7fd5e41008b0 2024-09-13T17:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.637+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e40fbec0 con 0x7fd5e41008b0 2024-09-13T17:26:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.638+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd5d4003ce0 con 0x7fd5e41008b0 2024-09-13T17:26:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.639+0000 7fd5cbfff640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd5cc0a1120 0x7fd5cc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.640+0000 7fd5e259c640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd5cc0a1120 0x7fd5cc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.640+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fd5d40ee440 con 0x7fd5e41008b0 2024-09-13T17:26:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.640+0000 7fd5e259c640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd5cc0a1120 0x7fd5cc0a35e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4069030 tx=0x7fd5d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.643+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd5d40b76f0 con 0x7fd5e41008b0 2024-09-13T17:26:08.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.816+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5e402c480 con 0x7fd5e41008b0 2024-09-13T17:26:08.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.816+0000 7fd5cbfff640 1 -- 172.21.15.44:0/49976918 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5d40b9620 con 0x7fd5e41008b0 2024-09-13T17:26:08.819 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd5cc0a1120 msgr2=0x7fd5cc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd5cc0a1120 0x7fd5cc0a35e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4069030 tx=0x7fd5d800b390 comp rx=0 tx=0).stop 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 msgr2=0x7fd5e4195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4195d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40057c0 tx=0x7fd5d40042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 shutdown_connections 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd5cc0a1120 0x7fd5cc0a35e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e41031e0 0x7fd5e4196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e4105b10 0x7fd5e419d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 --2- 172.21.15.44:0/49976918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5e41008b0 0x7fd5e4195d00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 >> 172.21.15.44:0/49976918 conn(0x7fd5e40fa3d0 msgr2=0x7fd5e4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 shutdown_connections 2024-09-13T17:26:08.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.819+0000 7fd5e9157640 1 -- 172.21.15.44:0/49976918 wait complete. 2024-09-13T17:26:08.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:08 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/441250081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:08 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3508247181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.831 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:08 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/108496506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:08.831 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:08 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1892343223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:08.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:08.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 -- 172.21.15.44:0/209501441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 msgr2=0x7f9b00101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/209501441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b00101400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009900 tx=0x7f9af402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:08.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 -- 172.21.15.44:0/209501441 shutdown_connections 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/209501441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 0x7f9b001081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/209501441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b00101400 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/209501441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b000ffd90 0x7f9b00100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 -- 172.21.15.44:0/209501441 >> 172.21.15.44:0/209501441 conn(0x7f9b000fb510 msgr2=0x7f9b000fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 -- 172.21.15.44:0/209501441 shutdown_connections 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 -- 172.21.15.44:0/209501441 wait complete. 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 Processor -- start 2024-09-13T17:26:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.935+0000 7f9b05f5e640 1 -- start start 2024-09-13T17:26:08.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9b05f5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b000ffd90 0x7f9b00074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9b05f5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b000728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9b05f5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 0x7f9b00072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9b05f5e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00073540 con 0x7f9b00100f80 2024-09-13T17:26:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9b05f5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b000736b0 con 0x7f9b00101940 2024-09-13T17:26:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9b05f5e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0019fcf0 con 0x7f9b000ffd90 2024-09-13T17:26:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b000728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b000728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34454/0 (socket says 172.21.15.44:34454) 2024-09-13T17:26:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 -- 172.21.15.44:0/2252929678 learned_addr learned my addr 172.21.15.44:0/2252929678 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afffff640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 0x7f9b00072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9aff7fe640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b000ffd90 0x7f9b00074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 -- 172.21.15.44:0/2252929678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b000ffd90 msgr2=0x7f9b00074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b000ffd90 0x7f9b00074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 -- 172.21.15.44:0/2252929678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 msgr2=0x7f9b00072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 0x7f9b00072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 -- 172.21.15.44:0/2252929678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009560 con 0x7f9b00100f80 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afffff640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 0x7f9b00072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afeffd640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b000728a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005ab0 tx=0x7f9af40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.936+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4031d70 con 0x7f9b00100f80 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.937+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9af40044e0 con 0x7f9b00100f80 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.937+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0019ff40 con 0x7f9b00100f80 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.937+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4003b30 con 0x7f9b00100f80 2024-09-13T17:26:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.937+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b001a0370 con 0x7f9b00100f80 2024-09-13T17:26:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.937+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b00107290 con 0x7f9b00100f80 2024-09-13T17:26:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.939+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9af4003c90 con 0x7f9b00100f80 2024-09-13T17:26:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.939+0000 7f9afcff9640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9ae80a11a0 0x7f9ae80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.939+0000 7f9aff7fe640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9ae80a11a0 0x7f9ae80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.939+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f9af40ee2e0 con 0x7f9b00100f80 2024-09-13T17:26:08.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.940+0000 7f9aff7fe640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9ae80a11a0 0x7f9ae80a3660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9af00058c0 tx=0x7f9af000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:08.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:08.943+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9af40b7680 con 0x7f9b00100f80 2024-09-13T17:26:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:08 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/441250081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:08 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3508247181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:08 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/108496506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:08.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:08 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1892343223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:08 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/441250081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:08 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3508247181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:08 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/108496506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:09.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:08 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1892343223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.114+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b0002c410 con 0x7f9b00100f80 2024-09-13T17:26:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.114+0000 7f9afcff9640 1 -- 172.21.15.44:0/2252929678 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9af404a020 con 0x7f9b00100f80 2024-09-13T17:26:09.117 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9ae80a11a0 msgr2=0x7f9ae80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9ae80a11a0 0x7f9ae80a3660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9af00058c0 tx=0x7f9af000a2d0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 msgr2=0x7f9b000728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b000728a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005ab0 tx=0x7f9af40042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 shutdown_connections 2024-09-13T17:26:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f9ae80a11a0 0x7f9ae80a3660 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b00101940 0x7f9b00072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b00100f80 0x7f9b000728a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 --2- 172.21.15.44:0/2252929678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b000ffd90 0x7f9b00074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 >> 172.21.15.44:0/2252929678 conn(0x7f9b000fb510 msgr2=0x7f9b00104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 shutdown_connections 2024-09-13T17:26:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.117+0000 7f9b05f5e640 1 -- 172.21.15.44:0/2252929678 wait complete. 2024-09-13T17:26:09.130 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:09.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 -- 172.21.15.44:0/2208060300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00100f90 msgr2=0x7f7b00101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/2208060300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00100f90 0x7f7b00101410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009860 tx=0x7f7af402a340 comp rx=0 tx=0).stop 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 -- 172.21.15.44:0/2208060300 shutdown_connections 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/2208060300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00100f90 0x7f7b00101410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/2208060300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00101950 0x7f7b001081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/2208060300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b000ffda0 0x7f7b001001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 -- 172.21.15.44:0/2208060300 >> 172.21.15.44:0/2208060300 conn(0x7f7b000fb520 msgr2=0x7f7b000fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 -- 172.21.15.44:0/2208060300 shutdown_connections 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.233+0000 7f7b05eb7640 1 -- 172.21.15.44:0/2208060300 wait complete. 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 Processor -- start 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 -- start start 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b000ffda0 0x7f7b00074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 0x7f7b000728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b00101950 0x7f7b00072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00073540 con 0x7f7b00100f90 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000736b0 con 0x7f7b00101950 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7b05eb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0019fd30 con 0x7f7b000ffda0 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7afeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 0x7f7b000728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7afeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 0x7f7b000728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34460/0 (socket says 172.21.15.44:34460) 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7afeffd640 1 -- 172.21.15.44:0/1924899303 learned_addr learned my addr 172.21.15.44:0/1924899303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7aff7fe640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b000ffda0 0x7f7b00074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7afffff640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b00101950 0x7f7b00072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.234+0000 7f7afeffd640 1 -- 172.21.15.44:0/1924899303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b000ffda0 msgr2=0x7f7b00074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afeffd640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b000ffda0 0x7f7b00074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afeffd640 1 -- 172.21.15.44:0/1924899303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b00101950 msgr2=0x7f7b00072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afeffd640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b00101950 0x7f7b00072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afeffd640 1 -- 172.21.15.44:0/1924899303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009490 con 0x7f7b00100f90 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afffff640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b00101950 0x7f7b00072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afeffd640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 0x7f7b000728a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009980 tx=0x7f7af4030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4032e50 con 0x7f7b00100f90 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af4030e90 con 0x7f7b00100f90 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0019ff80 con 0x7f7b00100f90 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af403b430 con 0x7f7b00100f90 2024-09-13T17:26:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.235+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b001a0350 con 0x7f7b00100f90 2024-09-13T17:26:09.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.236+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b001072c0 con 0x7f7b00100f90 2024-09-13T17:26:09.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.237+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7af4039040 con 0x7f7b00100f90 2024-09-13T17:26:09.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.238+0000 7f7afcff9640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7ae80a11a0 0x7f7ae80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.238+0000 7f7aff7fe640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7ae80a11a0 0x7f7ae80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.238+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f7af40ed060 con 0x7f7b00100f90 2024-09-13T17:26:09.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.238+0000 7f7aff7fe640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7ae80a11a0 0x7f7ae80a3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b00100df0 tx=0x7f7af0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:09.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.242+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7af40b6460 con 0x7f7b00100f90 2024-09-13T17:26:09.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.412+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b0002c480 con 0x7f7b00100f90 2024-09-13T17:26:09.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.413+0000 7f7afcff9640 1 -- 172.21.15.44:0/1924899303 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7af40370a0 con 0x7f7b00100f90 2024-09-13T17:26:09.415 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:09.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.414+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7ae80a11a0 msgr2=0x7f7ae80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.414+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7ae80a11a0 0x7f7ae80a3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b00100df0 tx=0x7f7af0005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:09.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 msgr2=0x7f7b000728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 0x7f7b000728a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009980 tx=0x7f7af4030890 comp rx=0 tx=0).stop 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 shutdown_connections 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f7ae80a11a0 0x7f7ae80a3660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b00101950 0x7f7b00072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b00100f90 0x7f7b000728a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 --2- 172.21.15.44:0/1924899303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b000ffda0 0x7f7b00074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 >> 172.21.15.44:0/1924899303 conn(0x7f7b000fb520 msgr2=0x7f7b00104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 shutdown_connections 2024-09-13T17:26:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.415+0000 7f7b05eb7640 1 -- 172.21.15.44:0/1924899303 wait complete. 2024-09-13T17:26:09.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:09.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.530+0000 7fe16f0e9640 1 -- 172.21.15.44:0/978612797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe168101950 msgr2=0x7fe1681081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.530+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/978612797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe168101950 0x7fe1681081e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe160009ca0 tx=0x7fe16002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- 172.21.15.44:0/978612797 shutdown_connections 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/978612797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe168101950 0x7fe1681081e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/978612797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 0x7fe168101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/978612797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1680ffda0 0x7fe1681001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- 172.21.15.44:0/978612797 >> 172.21.15.44:0/978612797 conn(0x7fe1680fb520 msgr2=0x7fe1680fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- 172.21.15.44:0/978612797 shutdown_connections 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- 172.21.15.44:0/978612797 wait complete. 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 Processor -- start 2024-09-13T17:26:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- start start 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 0x7fe168195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 0x7fe1681961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe168101950 0x7fe16819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819d7c0 con 0x7fe1680ffda0 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819d930 con 0x7fe168100f90 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.531+0000 7fe16f0e9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819daf0 con 0x7fe168101950 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 0x7fe168195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 0x7fe168195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58234/0 (socket says 172.21.15.44:58234) 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 -- 172.21.15.44:0/3204325628 learned_addr learned my addr 172.21.15.44:0/3204325628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16d686640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe168101950 0x7fe16819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe167fff640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 0x7fe1681961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 -- 172.21.15.44:0/3204325628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe168101950 msgr2=0x7fe16819d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe168101950 0x7fe16819d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 -- 172.21.15.44:0/3204325628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 msgr2=0x7fe1681961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 0x7fe1681961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 -- 172.21.15.44:0/3204325628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160009900 con 0x7fe1680ffda0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe167fff640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 0x7fe1681961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16ce85640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 0x7fe168195cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00d950 tx=0x7fe15c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c009c30 con 0x7fe1680ffda0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe15c009d90 con 0x7fe1680ffda0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16819ddd0 con 0x7fe1680ffda0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00f620 con 0x7fe1680ffda0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.532+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16819e220 con 0x7fe1680ffda0 2024-09-13T17:26:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.533+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe168107230 con 0x7fe1680ffda0 2024-09-13T17:26:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.535+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe15c010980 con 0x7fe1680ffda0 2024-09-13T17:26:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.535+0000 7fe165ffb640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1580a11a0 0x7fe1580a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.536+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fe15c0ccb00 con 0x7fe1680ffda0 2024-09-13T17:26:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.536+0000 7fe167fff640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1580a11a0 0x7fe1580a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.536+0000 7fe167fff640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1580a11a0 0x7fe1580a3660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe154008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.540+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe15c095df0 con 0x7fe1680ffda0 2024-09-13T17:26:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.714+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe16802c480 con 0x7fe1680ffda0 2024-09-13T17:26:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.714+0000 7fe165ffb640 1 -- 172.21.15.44:0/3204325628 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe15c0170a0 con 0x7fe1680ffda0 2024-09-13T17:26:09.716 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.716+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1580a11a0 msgr2=0x7fe1580a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.716+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1580a11a0 0x7fe1580a3660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe154008040 comp rx=0 tx=0).stop 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 msgr2=0x7fe168195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 0x7fe168195cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00d950 tx=0x7fe15c00de10 comp rx=0 tx=0).stop 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 shutdown_connections 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fe1580a11a0 0x7fe1580a3660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe168101950 0x7fe16819d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe168100f90 0x7fe1681961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 --2- 172.21.15.44:0/3204325628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1680ffda0 0x7fe168195cb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 >> 172.21.15.44:0/3204325628 conn(0x7fe1680fb520 msgr2=0x7fe168104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 shutdown_connections 2024-09-13T17:26:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.717+0000 7fe16f0e9640 1 -- 172.21.15.44:0/3204325628 wait complete. 2024-09-13T17:26:09.728 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:09 smithi044 ceph-mon[31175]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:26:09.728 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:09 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/49976918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.728 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:09 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2252929678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.729 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:09 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1924899303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:09.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 -- 172.21.15.44:0/577661207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 msgr2=0x7fab74107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 --2- 172.21.15.44:0/577661207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab74107f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009ca0 tx=0x7fab6c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 -- 172.21.15.44:0/577661207 shutdown_connections 2024-09-13T17:26:09.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 --2- 172.21.15.44:0/577661207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab74107f20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 --2- 172.21.15.44:0/577661207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74103180 0x7fab74105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 --2- 172.21.15.44:0/577661207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 0x7fab74102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 -- 172.21.15.44:0/577661207 >> 172.21.15.44:0/577661207 conn(0x7fab740fa430 msgr2=0x7fab740fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 -- 172.21.15.44:0/577661207 shutdown_connections 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.832+0000 7fab7b019640 1 -- 172.21.15.44:0/577661207 wait complete. 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 Processor -- start 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 -- start start 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 0x7fab74195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74103180 0x7fab741961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab7419d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7419d790 con 0x7fab74105ab0 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7419d900 con 0x7fab74103180 2024-09-13T17:26:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab7b019640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7419dac0 con 0x7fab74100850 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab7419d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab7419d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58248/0 (socket says 172.21.15.44:58248) 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 -- 172.21.15.44:0/1838959567 learned_addr learned my addr 172.21.15.44:0/1838959567 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab78db5640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 0x7fab74195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab73fff640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74103180 0x7fab741961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 -- 172.21.15.44:0/1838959567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 msgr2=0x7fab74195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 0x7fab74195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 -- 172.21.15.44:0/1838959567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74103180 msgr2=0x7fab741961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74103180 0x7fab741961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab795b6640 1 -- 172.21.15.44:0/1838959567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c009900 con 0x7fab74105ab0 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.833+0000 7fab78db5640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 0x7fab74195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:09.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.834+0000 7fab795b6640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab7419d220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fab6c02aeb0 tx=0x7fab6c02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:09.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.834+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c003bf0 con 0x7fab74105ab0 2024-09-13T17:26:09.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.834+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab6c040070 con 0x7fab74105ab0 2024-09-13T17:26:09.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.834+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7419dd40 con 0x7fab74105ab0 2024-09-13T17:26:09.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.834+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0044a0 con 0x7fab74105ab0 2024-09-13T17:26:09.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.834+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7419e1f0 con 0x7fab74105ab0 2024-09-13T17:26:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.835+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab740fbf20 con 0x7fab74105ab0 2024-09-13T17:26:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.836+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fab6c042030 con 0x7fab74105ab0 2024-09-13T17:26:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.837+0000 7fab71ffb640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fab640a1120 0x7fab640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.837+0000 7fab78db5640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fab640a1120 0x7fab640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.837+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fab6c0ee670 con 0x7fab74105ab0 2024-09-13T17:26:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.838+0000 7fab78db5640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fab640a1120 0x7fab640a35e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab68005fd0 tx=0x7fab68005f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:09.841+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fab6c0b7b30 con 0x7fab74105ab0 2024-09-13T17:26:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:09 smithi070 ceph-mon[34502]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:26:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:09 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/49976918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:09 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2252929678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:09 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1924899303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:09 smithi100 ceph-mon[35055]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-13T17:26:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:09 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/49976918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:09 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2252929678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:09 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1924899303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.046+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab7410b560 con 0x7fab74105ab0 2024-09-13T17:26:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.046+0000 7fab71ffb640 1 -- 172.21.15.44:0/1838959567 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fab6c0bb150 con 0x7fab74105ab0 2024-09-13T17:26:10.049 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:10.049 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:47.147746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:10.049 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fab640a1120 msgr2=0x7fab640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fab640a1120 0x7fab640a35e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab68005fd0 tx=0x7fab68005f40 comp rx=0 tx=0).stop 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 msgr2=0x7fab7419d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab7419d220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fab6c02aeb0 tx=0x7fab6c02f310 comp rx=0 tx=0).stop 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 shutdown_connections 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fab640a1120 0x7fab640a35e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab74105ab0 0x7fab7419d220 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74103180 0x7fab741961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 --2- 172.21.15.44:0/1838959567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab74100850 0x7fab74195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 >> 172.21.15.44:0/1838959567 conn(0x7fab740fa430 msgr2=0x7fab74101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.048+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 shutdown_connections 2024-09-13T17:26:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.049+0000 7fab7b019640 1 -- 172.21.15.44:0/1838959567 wait complete. 2024-09-13T17:26:10.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 -- 172.21.15.44:0/2159265255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 msgr2=0x7fc220100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2159265255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220100310 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc214009900 tx=0x7fc21402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 -- 172.21.15.44:0/2159265255 shutdown_connections 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2159265255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc220100850 0x7fc2201070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2159265255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220100310 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2159265255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2200feca0 0x7fc2200ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 -- 172.21.15.44:0/2159265255 >> 172.21.15.44:0/2159265255 conn(0x7fc2200fa440 msgr2=0x7fc2200fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 -- 172.21.15.44:0/2159265255 shutdown_connections 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 -- 172.21.15.44:0/2159265255 wait complete. 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 Processor -- start 2024-09-13T17:26:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.162+0000 7fc225aaa640 1 -- start start 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2200feca0 0x7fc220109560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc220100850 0x7fc2201080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220108840 con 0x7fc2200ffe90 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201089b0 con 0x7fc220100850 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22019fcc0 con 0x7fc2200feca0 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58278/0 (socket says 172.21.15.44:58278) 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 -- 172.21.15.44:0/2952222544 learned_addr learned my addr 172.21.15.44:0/2952222544 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21f7fe640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2200feca0 0x7fc220109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21ffff640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc220100850 0x7fc2201080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 -- 172.21.15.44:0/2952222544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2200feca0 msgr2=0x7fc220109560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2200feca0 0x7fc220109560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 -- 172.21.15.44:0/2952222544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc220100850 msgr2=0x7fc2201080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc220100850 0x7fc2201080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 -- 172.21.15.44:0/2952222544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009560 con 0x7fc2200ffe90 2024-09-13T17:26:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21ffff640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc220100850 0x7fc2201080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21effd640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220107ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc214009a20 tx=0x7fc214004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214031d70 con 0x7fc2200ffe90 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc214004b50 con 0x7fc2200ffe90 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22019ff10 con 0x7fc2200ffe90 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.163+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214003bf0 con 0x7fc2200ffe90 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.164+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2201a0340 con 0x7fc2200ffe90 2024-09-13T17:26:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.165+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2201060d0 con 0x7fc2200ffe90 2024-09-13T17:26:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.166+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc214003d70 con 0x7fc2200ffe90 2024-09-13T17:26:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.166+0000 7fc21cff9640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc2080a11a0 0x7fc2080a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.166+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fc2140ee230 con 0x7fc2200ffe90 2024-09-13T17:26:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.166+0000 7fc21f7fe640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc2080a11a0 0x7fc2080a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.167+0000 7fc21f7fe640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc2080a11a0 0x7fc2080a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2200ffcf0 tx=0x7fc21000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.170+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc2140b7520 con 0x7fc2200ffe90 2024-09-13T17:26:10.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.345+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc22002c410 con 0x7fc2200ffe90 2024-09-13T17:26:10.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.346+0000 7fc21cff9640 1 -- 172.21.15.44:0/2952222544 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24492 (secure 0 0 0) 0x7fc2140bab40 con 0x7fc2200ffe90 2024-09-13T17:26:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:10.349 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":92,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:25:43.045876+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc2080a11a0 msgr2=0x7fc2080a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc2080a11a0 0x7fc2080a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2200ffcf0 tx=0x7fc21000a2d0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 msgr2=0x7fc220107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220107ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc214009a20 tx=0x7fc214004380 comp rx=0 tx=0).stop 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 shutdown_connections 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc2080a11a0 0x7fc2080a3660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc220100850 0x7fc2201080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2200ffe90 0x7fc220107ba0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 --2- 172.21.15.44:0/2952222544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2200feca0 0x7fc220109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 >> 172.21.15.44:0/2952222544 conn(0x7fc2200fa440 msgr2=0x7fc2201032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 shutdown_connections 2024-09-13T17:26:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.348+0000 7fc225aaa640 1 -- 172.21.15.44:0/2952222544 wait complete. 2024-09-13T17:26:10.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 -- 172.21.15.44:0/1529364133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70100910 msgr2=0x7fda70102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 --2- 172.21.15.44:0/1529364133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70100910 0x7fda70102d00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fda60009900 tx=0x7fda6002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 -- 172.21.15.44:0/1529364133 shutdown_connections 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 --2- 172.21.15.44:0/1529364133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda70103240 0x7fda70105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 --2- 172.21.15.44:0/1529364133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda70105b70 0x7fda70107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 --2- 172.21.15.44:0/1529364133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70100910 0x7fda70102d00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 -- 172.21.15.44:0/1529364133 >> 172.21.15.44:0/1529364133 conn(0x7fda700fa430 msgr2=0x7fda700fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 -- 172.21.15.44:0/1529364133 shutdown_connections 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.462+0000 7fda7683a640 1 -- 172.21.15.44:0/1529364133 wait complete. 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 Processor -- start 2024-09-13T17:26:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 -- start start 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda70100910 0x7fda70195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 0x7fda70196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda70105b70 0x7fda7019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7019d8b0 con 0x7fda70103240 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7019da20 con 0x7fda70105b70 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda7683a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7019dbe0 con 0x7fda70100910 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 0x7fda70196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 0x7fda70196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58282/0 (socket says 172.21.15.44:58282) 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 -- 172.21.15.44:0/872515486 learned_addr learned my addr 172.21.15.44:0/872515486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda74dd7640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda70105b70 0x7fda7019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6ffff640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda70100910 0x7fda70195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 -- 172.21.15.44:0/872515486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda70100910 msgr2=0x7fda70195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda70100910 0x7fda70195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 -- 172.21.15.44:0/872515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda70105b70 msgr2=0x7fda7019d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda70105b70 0x7fda7019d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.463+0000 7fda6f7fe640 1 -- 172.21.15.44:0/872515486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60009560 con 0x7fda70103240 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.464+0000 7fda6f7fe640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 0x7fda70196300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fda6400e9f0 tx=0x7fda6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.464+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640096c0 con 0x7fda70103240 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.464+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6400ccc0 con 0x7fda70103240 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.464+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7019dec0 con 0x7fda70103240 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.464+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64004e60 con 0x7fda70103240 2024-09-13T17:26:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.464+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7019e2f0 con 0x7fda70103240 2024-09-13T17:26:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.465+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda700fbf20 con 0x7fda70103240 2024-09-13T17:26:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.466+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fda64005000 con 0x7fda70103240 2024-09-13T17:26:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.467+0000 7fda6d7fa640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda580a1120 0x7fda580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.467+0000 7fda6ffff640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda580a1120 0x7fda580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.467+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fda64014070 con 0x7fda70103240 2024-09-13T17:26:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.467+0000 7fda6ffff640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda580a1120 0x7fda580a35e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda60005ec0 tx=0x7fda60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.470+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fda64096760 con 0x7fda70103240 2024-09-13T17:26:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.677+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda7010b570 con 0x7fda70103240 2024-09-13T17:26:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.678+0000 7fda6d7fa640 1 -- 172.21.15.44:0/872515486 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fda64099d80 con 0x7fda70103240 2024-09-13T17:26:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:10.681 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:47.147746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:26:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda580a1120 msgr2=0x7fda580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda580a1120 0x7fda580a35e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda60005ec0 tx=0x7fda60000bc0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 msgr2=0x7fda70196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 0x7fda70196300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fda6400e9f0 tx=0x7fda6400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 shutdown_connections 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fda580a1120 0x7fda580a35e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda70105b70 0x7fda7019d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda70103240 0x7fda70196300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 --2- 172.21.15.44:0/872515486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda70100910 0x7fda70195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 >> 172.21.15.44:0/872515486 conn(0x7fda700fa430 msgr2=0x7fda70101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 shutdown_connections 2024-09-13T17:26:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.680+0000 7fda7683a640 1 -- 172.21.15.44:0/872515486 wait complete. 2024-09-13T17:26:10.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:10 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3204325628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:10.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:10 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1838959567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:10.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:10 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2952222544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:10.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-13T17:24:21.722369+0000', 'modified': '2024-09-13T17:24:47.147746+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24517, 'mds_1': 24496, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6833/459553980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 459553980}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 459553980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6837/4202473697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 4202473697}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 4202473697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.70:6839/2731404816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6838', 'nonce': 2731404816}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 2731404816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6835/4065662345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 4065662345}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 4065662345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.100:6837/288521582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 288521582}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 288521582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.100:6839/3416533473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 3416533473}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 3416533473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:26:10.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-13T17:26:10.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.794+0000 7f4df50c9640 1 -- 172.21.15.44:0/926426218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00ffea0 msgr2=0x7f4df0100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.794+0000 7f4df50c9640 1 --2- 172.21.15.44:0/926426218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00ffea0 0x7f4df0100320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009900 tx=0x7f4de402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 -- 172.21.15.44:0/926426218 shutdown_connections 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 --2- 172.21.15.44:0/926426218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4df0100860 0x7f4df01070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 --2- 172.21.15.44:0/926426218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00ffea0 0x7f4df0100320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 --2- 172.21.15.44:0/926426218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df00fecb0 0x7f4df00ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 -- 172.21.15.44:0/926426218 >> 172.21.15.44:0/926426218 conn(0x7f4df00fa430 msgr2=0x7f4df00fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 -- 172.21.15.44:0/926426218 shutdown_connections 2024-09-13T17:26:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 -- 172.21.15.44:0/926426218 wait complete. 2024-09-13T17:26:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 Processor -- start 2024-09-13T17:26:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.795+0000 7f4df50c9640 1 -- start start 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4df50c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df00fecb0 0x7f4df0102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4df50c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4df00ffea0 0x7f4df0101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4df50c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 0x7f4df01017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4df50c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0101f20 con 0x7f4df0100860 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4df50c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0102090 con 0x7f4df00fecb0 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4df50c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df019fe00 con 0x7f4df00ffea0 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 0x7f4df01017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 0x7f4df01017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58296/0 (socket says 172.21.15.44:58296) 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 -- 172.21.15.44:0/351735243 learned_addr learned my addr 172.21.15.44:0/351735243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4dee59c640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4df00ffea0 0x7f4df0101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4deed9d640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df00fecb0 0x7f4df0102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 -- 172.21.15.44:0/351735243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4df00ffea0 msgr2=0x7f4df0101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4df00ffea0 0x7f4df0101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 -- 172.21.15.44:0/351735243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df00fecb0 msgr2=0x7f4df0102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df00fecb0 0x7f4df0102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 -- 172.21.15.44:0/351735243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de4009560 con 0x7f4df0100860 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4def59e640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 0x7f4df01017c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4de800cd10 tx=0x7f4de800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.796+0000 7f4dd7fff640 1 -- 172.21.15.44:0/351735243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de800f450 con 0x7f4df0100860 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.797+0000 7f4dd7fff640 1 -- 172.21.15.44:0/351735243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4de800fa40 con 0x7f4df0100860 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.797+0000 7f4dd7fff640 1 -- 172.21.15.44:0/351735243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8018950 con 0x7f4df0100860 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.797+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df01a00b0 con 0x7f4df0100860 2024-09-13T17:26:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.797+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df01a0540 con 0x7f4df0100860 2024-09-13T17:26:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4dd7fff640 1 -- 172.21.15.44:0/351735243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4de800fbb0 con 0x7f4df0100860 2024-09-13T17:26:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 Processor -- start 2024-09-13T17:26:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 -- start start 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 0x557a29971900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a29976900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29976ed0 0x557a29979390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a298cc7c0 con 0x557a29971500 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a298dbb70 con 0x557a29976ed0 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4df7dec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a298d9300 con 0x557a29971e40 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 0x557a29971900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 0x557a29971900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58304/0 (socket says 172.21.15.44:58304) 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4dee59c640 1 -- 172.21.15.44:0/3175420471 learned_addr learned my addr 172.21.15.44:0/3175420471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4dee59c640 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29976ed0 0x557a29979390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.799+0000 7f4deed9d640 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a29976900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dd7fff640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4dd80a1120 0x7f4dd80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dee59c640 1 -- 172.21.15.44:0/3175420471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 msgr2=0x557a29976900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dee59c640 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a29976900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dee59c640 1 -- 172.21.15.44:0/3175420471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29976ed0 msgr2=0x557a29979390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dee59c640 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29976ed0 0x557a29979390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dee59c640 1 -- 172.21.15.44:0/3175420471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a298614b0 con 0x557a29971500 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4def59e640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4dd80a1120 0x7f4dd80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dee59c640 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 0x557a29971900 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009230 tx=0x7f4de4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babb2e1742b42606 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dd7fff640 1 -- 172.21.15.44:0/351735243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f4de80cd230 con 0x7f4df0100860 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dd57fa640 1 -- 172.21.15.44:0/3175420471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40053f0 con 0x557a29971500 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dd57fa640 1 -- 172.21.15.44:0/3175420471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4de4043070 con 0x557a29971500 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.800+0000 7f4dd57fa640 1 -- 172.21.15.44:0/3175420471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4003ae0 con 0x557a29971500 2024-09-13T17:26:10.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- 172.21.15.44:0/3175420471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 msgr2=0x557a29971900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 0x557a29971900 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009230 tx=0x7f4de4004250 comp rx=0 tx=0).stop 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4def59e640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4dd80a1120 0x7f4dd80a35e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4de8099ee0 tx=0x7f4de800cf50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- 172.21.15.44:0/3175420471 shutdown_connections 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a29976900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29976ed0 0x557a29979390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- 172.21.15.44:0/3175420471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971500 0x557a29971900 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- 172.21.15.44:0/3175420471 >> 172.21.15.44:0/3175420471 conn(0x557a2996cb70 msgr2=0x557a2996cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- 172.21.15.44:0/3175420471 shutdown_connections 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- 172.21.15.44:0/3175420471 wait complete. 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- start start 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 Processor -- start 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971500 0x557a2997b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971e40 0x557a2997b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 0x557a29982690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a298d4bf0 con 0x557a29976ed0 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29893250 con 0x557a29971e40 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29892ef0 con 0x557a29971500 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.801+0000 7f4df7dec740 10 client.0 _ll_register_callbacks cb 0x557a298c5740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 0x557a29982690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 0x557a29982690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58312/0 (socket says 172.21.15.44:58312) 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 -- 172.21.15.44:0/2254463959 learned_addr learned my addr 172.21.15.44:0/2254463959 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971500 0x557a2997b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4deed9d640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971e40 0x557a2997b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 -- 172.21.15.44:0/2254463959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971500 msgr2=0x557a2997b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971500 0x557a2997b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 -- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971e40 msgr2=0x557a2997b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971e40 0x557a2997b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 -- 172.21.15.44:0/2254463959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df01079f0 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4deed9d640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971e40 0x557a2997b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dee59c640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 0x557a29982690 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4de402a930 tx=0x7f4de4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8707627ecf8b8a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/2254463959 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4003dc0 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/2254463959 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f4de40046c0 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a295c83c0 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/2254463959 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de4003f20 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4df7dec740 10 client.15228 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.802+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a298d3270 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/2254463959 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557a295c83c0 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/2254463959 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4de4031a10 con 0x557a29976ed0 2024-09-13T17:26:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4df7dec740 10 client.15228 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4df7dec740 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4df7dec740 20 client.15228 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4df7dec740 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a299872f0 0x557a2998b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4df7dec740 4 client.15228 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a295d96c0 con 0x557a299872f0 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.803+0000 7f4deed9d640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a299872f0 0x557a2998b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4deed9d640 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a299872f0 0x557a2998b470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=558f8aafaf77853d server_cookie=e1cefedeb2454193 in_seq=0 out_seq=0 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4dd2ff5640 10 client.15228 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/2254463959 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557a295d96c0 con 0x557a299872f0 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4dd2ff5640 10 client.15228 handle_command_reply: tid=0 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 client.15228 shutdown 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 msgr2=0x557a29982690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 0x557a29982690 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4de402a930 tx=0x7f4de4005850 comp rx=0 tx=0).stop 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 shutdown_connections 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a299872f0 0x557a2998b470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29976ed0 0x557a29982690 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971e40 0x557a2997b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 --2- 172.21.15.44:0/2254463959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971500 0x557a2997b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 >> 172.21.15.44:0/2254463959 conn(0x557a2996cb70 msgr2=0x557a2996e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.804+0000 7f4dd2ff5640 0 client.15228 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.805+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 shutdown_connections 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.805+0000 7f4df7dec740 1 -- 172.21.15.44:0/2254463959 wait complete. 2024-09-13T17:26:10.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.805+0000 7f4df7dec740 20 client.15228 trim_cache size 0 max 16384 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.819+0000 7f4df7dec740 1 Processor -- start 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.819+0000 7f4df7dec740 1 -- start start 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a2996cb70 0x557a2998f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a299825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 0x557a2998f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4df7dec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29858460 con 0x557a29971e40 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4df7dec740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29989280 con 0x557a29971500 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4df7dec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a295c83c0 con 0x557a2996cb70 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 0x557a2998f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 0x557a2998f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58324/0 (socket says 172.21.15.44:58324) 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 -- 172.21.15.44:0/1778219379 learned_addr learned my addr 172.21.15.44:0/1778219379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a2996cb70 0x557a2998f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4deed9d640 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a299825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 -- 172.21.15.44:0/1778219379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a2996cb70 msgr2=0x557a2998f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a2996cb70 0x557a2998f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 -- 172.21.15.44:0/1778219379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 msgr2=0x557a299825b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a299825b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 -- 172.21.15.44:0/1778219379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a298755f0 con 0x557a29971e40 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 0x557a2998f9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4de4040040 tx=0x7f4de402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c44823f5a74d590 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dd5ffb640 1 -- 172.21.15.44:0/1778219379 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de404a430 con 0x557a29971e40 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dd5ffb640 1 -- 172.21.15.44:0/1778219379 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4de4053030 con 0x557a29971e40 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.820+0000 7f4dd5ffb640 1 -- 172.21.15.44:0/1778219379 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4031470 con 0x557a29971e40 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 -- 172.21.15.44:0/1778219379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 msgr2=0x557a2998f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 0x557a2998f9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4de4040040 tx=0x7f4de402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 -- 172.21.15.44:0/1778219379 shutdown_connections 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a29971e40 0x557a2998f9d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a299825b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1778219379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a2996cb70 0x557a2998f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 -- 172.21.15.44:0/1778219379 >> 172.21.15.44:0/1778219379 conn(0x557a29979630 msgr2=0x557a2997b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 -- 172.21.15.44:0/1778219379 shutdown_connections 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 -- 172.21.15.44:0/1778219379 wait complete. 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 -- start start 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.821+0000 7f4df7dec740 1 Processor -- start 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 0x557a29982490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a2997f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a2997f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2997fc10 con 0x557a2996cb70 2024-09-13T17:26:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a299dd130 con 0x557a29971500 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a299dd2a0 con 0x557a29971e40 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 10 client.0 _ll_register_callbacks cb 0x7f4de40041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 0x557a29982490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 0x557a29982490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58336/0 (socket says 172.21.15.44:58336) 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 -- 172.21.15.44:0/1150902926 learned_addr learned my addr 172.21.15.44:0/1150902926 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4deed9d640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a2997f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a2997f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 -- 172.21.15.44:0/1150902926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 msgr2=0x557a2997f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a2997f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 -- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 msgr2=0x557a2997f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a2997f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 -- 172.21.15.44:0/1150902926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a298d3270 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4deed9d640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a2997f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dee59c640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 0x557a29982490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4de4031ec0 tx=0x7f4de4003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a50abd8e367437 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/1150902926 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40035f0 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/1150902926 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f4de4000e40 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.822+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4de4043070 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/1150902926 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de40046c0 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 10 client.15234 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a2989ad00 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/1150902926 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4de4043070 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/1150902926 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4de404a5e0 con 0x557a2996cb70 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 10 client.15234 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T17:26:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 20 client.15234 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a29976ed0 0x557a299de590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 4 client.15234 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a2962cd10 con 0x557a29976ed0 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.823+0000 7f4deed9d640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a29976ed0 0x557a299de590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4dd2ff5640 10 client.15234 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4deed9d640 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a29976ed0 0x557a299de590 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f27ea5cee78be16 server_cookie=db528546a0114cfc in_seq=0 out_seq=0 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4dd2ff5640 1 -- 172.21.15.44:0/1150902926 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557a2962cd10 con 0x557a29976ed0 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4dd2ff5640 10 client.15234 handle_command_reply: tid=0 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 client.15234 shutdown 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 msgr2=0x557a29982490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 0x557a29982490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4de4031ec0 tx=0x7f4de4003810 comp rx=0 tx=0).stop 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 shutdown_connections 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557a29976ed0 0x557a299de590 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a29971e40 0x557a2997f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a29971500 0x557a2997f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 --2- 172.21.15.44:0/1150902926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a2996cb70 0x557a29982490 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4dd2ff5640 0 client.15234 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 >> 172.21.15.44:0/1150902926 conn(0x557a29979630 msgr2=0x557a2998f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 shutdown_connections 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 1 -- 172.21.15.44:0/1150902926 wait complete. 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.824+0000 7f4df7dec740 20 client.15234 trim_cache size 0 max 16384 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:26:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24449, 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 84.681762974999998, 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 24, 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 92, 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 90, 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 124.398245199 2024-09-13T17:26:10.834 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4dd80a1120 msgr2=0x7f4dd80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4dd80a1120 0x7f4dd80a35e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4de8099ee0 tx=0x7f4de800cf50 comp rx=0 tx=0).stop 2024-09-13T17:26:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 msgr2=0x7f4df01017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 0x7f4df01017c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4de800cd10 tx=0x7f4de800a620 comp rx=0 tx=0).stop 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 shutdown_connections 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f4dd80a1120 0x7f4dd80a35e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4df00ffea0 0x7f4df0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df0100860 0x7f4df01017c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 --2- 172.21.15.44:0/351735243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df00fecb0 0x7f4df0102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 >> 172.21.15.44:0/351735243 conn(0x7f4df00fa430 msgr2=0x7f4df00686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 shutdown_connections 2024-09-13T17:26:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.825+0000 7f4df50c9640 1 -- 172.21.15.44:0/351735243 wait complete. 2024-09-13T17:26:10.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- 172.21.15.44:0/447435465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f700ffe60 msgr2=0x7f5f701002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- 172.21.15.44:0/447435465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f700ffe60 0x7f5f701002e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009900 tx=0x7f5f6402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- 172.21.15.44:0/447435465 shutdown_connections 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- 172.21.15.44:0/447435465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f70100820 0x7f5f701070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- 172.21.15.44:0/447435465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f700ffe60 0x7f5f701002e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- 172.21.15.44:0/447435465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700fec70 0x7f5f700ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- 172.21.15.44:0/447435465 >> 172.21.15.44:0/447435465 conn(0x7f5f700fa410 msgr2=0x7f5f700fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- 172.21.15.44:0/447435465 shutdown_connections 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- 172.21.15.44:0/447435465 wait complete. 2024-09-13T17:26:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 Processor -- start 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- start start 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700fec70 0x7f5f70195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700ffe60 0x7f5f701961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 0x7f5f7019d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7019d750 con 0x7f5f70100820 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7019d8c0 con 0x7f5f700fec70 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.939+0000 7f5f76fec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7019da80 con 0x7f5f700ffe60 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 0x7f5f7019d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 0x7f5f7019d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58348/0 (socket says 172.21.15.44:58348) 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 -- 172.21.15.44:0/2012630683 learned_addr learned my addr 172.21.15.44:0/2012630683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700ffe60 0x7f5f701961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700fec70 0x7f5f70195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 -- 172.21.15.44:0/2012630683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700ffe60 msgr2=0x7f5f701961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700ffe60 0x7f5f701961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 -- 172.21.15.44:0/2012630683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700fec70 msgr2=0x7f5f70195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700fec70 0x7f5f70195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 -- 172.21.15.44:0/2012630683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009560 con 0x7f5f70100820 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f75589640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 0x7f5f7019d1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800eaa0 tx=0x7f5f6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f6dffb640 1 -- 172.21.15.44:0/2012630683 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800cdb0 con 0x7f5f70100820 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f6dffb640 1 -- 172.21.15.44:0/2012630683 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f68005020 con 0x7f5f70100820 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7019dd60 con 0x7f5f70100820 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.940+0000 7f5f6dffb640 1 -- 172.21.15.44:0/2012630683 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f680116d0 con 0x7f5f70100820 2024-09-13T17:26:10.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.941+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7019e200 con 0x7f5f70100820 2024-09-13T17:26:10.951 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:10 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3204325628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:10.952 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:10 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1838959567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:10.952 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:10 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2952222544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f6dffb640 1 -- 172.21.15.44:0/2012630683 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5f68005190 con 0x7f5f70100820 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 Processor -- start 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 -- start start 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 0x558eb0ab2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2e40 0x558eb0ab7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f6dffb640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5f580a11a0 0x7f5f580a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0aba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0a0d7c0 con 0x558eb0ab2500 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0a1cb70 con 0x558eb0ab7ed0 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f79de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0a1a300 con 0x558eb0ab2e40 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f6dffb640 1 -- 172.21.15.44:0/2012630683 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f5f680ccec0 con 0x7f5f70100820 2024-09-13T17:26:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.943+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 0x558eb0ab2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0aba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 0x558eb0ab2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58352/0 (socket says 172.21.15.44:58352) 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f6ffff640 1 -- 172.21.15.44:0/3602138585 learned_addr learned my addr 172.21.15.44:0/3602138585 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f74d88640 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2e40 0x558eb0ab7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5f580a11a0 0x7f5f580a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 -- 172.21.15.44:0/3602138585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2e40 msgr2=0x558eb0ab7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2e40 0x558eb0ab7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 -- 172.21.15.44:0/3602138585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 msgr2=0x558eb0ab2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 0x558eb0ab2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 -- 172.21.15.44:0/3602138585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eb09a24b0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 0x558eb0ab2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5f580a11a0 0x7f5f580a3660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f64005ec0 tx=0x7f5f64005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f75589640 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0aba390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f680ca530 tx=0x7f5f68099880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c43ca1d5c0d538 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f56ffd640 1 -- 172.21.15.44:0/3602138585 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68005490 con 0x558eb0ab7ed0 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.944+0000 7f5f56ffd640 1 -- 172.21.15.44:0/3602138585 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f680a31b0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f56ffd640 1 -- 172.21.15.44:0/3602138585 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6809b0b0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 -- 172.21.15.44:0/3602138585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 msgr2=0x558eb0aba390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0aba390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f680ca530 tx=0x7f5f68099880 comp rx=0 tx=0).stop 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 -- 172.21.15.44:0/3602138585 shutdown_connections 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2e40 0x558eb0ab7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0aba390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 --2- 172.21.15.44:0/3602138585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2500 0x558eb0ab2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 -- 172.21.15.44:0/3602138585 >> 172.21.15.44:0/3602138585 conn(0x558eb0aadb70 msgr2=0x558eb0aadf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 -- 172.21.15.44:0/3602138585 shutdown_connections 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.945+0000 7f5f79de1740 1 -- 172.21.15.44:0/3602138585 wait complete. 2024-09-13T17:26:10.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 -- start start 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 Processor -- start 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 0x558eb0abc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0a15bf0 con 0x558eb0ab2e40 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb09d4250 con 0x558eb0ab7ed0 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb09d3ef0 con 0x558eb0ab2500 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f79de1740 10 client.0 _ll_register_callbacks cb 0x558eb0a06740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 0x558eb0abc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 0x558eb0abc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58358/0 (socket says 172.21.15.44:58358) 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f6ffff640 1 -- 172.21.15.44:0/2451531393 learned_addr learned my addr 172.21.15.44:0/2451531393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.946+0000 7f5f75589640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f6ffff640 1 -- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 msgr2=0x558eb0abc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f6ffff640 1 -- 172.21.15.44:0/2451531393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 msgr2=0x558eb0ac36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f6ffff640 1 -- 172.21.15.44:0/2451531393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7019e200 con 0x558eb0ab2e40 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 0x558eb0abc6a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009230 tx=0x7f5f64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6c5f4a0c17be61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2451531393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64003800 con 0x558eb0ab2e40 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2451531393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f5f64003960 con 0x558eb0ab2e40 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558eb07093c0 con 0x558eb0ab2e40 2024-09-13T17:26:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2451531393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f64003b20 con 0x558eb0ab2e40 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f79de1740 10 client.15246 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558eb0a14270 con 0x558eb0ab2e40 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2451531393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558eb07093c0 con 0x558eb0ab2e40 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2451531393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f6404dc50 con 0x558eb0ab2e40 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.947+0000 7f5f79de1740 10 client.15246 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f79de1740 10 client.15246 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f79de1740 20 client.15246 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0ac7b60 0x558eb0acc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f79de1740 4 client.15246 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558eb071a6c0 con 0x558eb0ac7b60 2024-09-13T17:26:10.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0ac7b60 0x558eb0acc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f4ffff640 10 client.15246 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.948+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0ac7b60 0x558eb0acc800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a07b10eb5d3dd6bb server_cookie=eac959bf6724b2d3 in_seq=0 out_seq=0 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2451531393 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558eb071a6c0 con 0x558eb0ac7b60 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f4ffff640 10 client.15246 handle_command_reply: tid=0 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 client.15246 shutdown 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 msgr2=0x558eb0abc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 0x558eb0abc6a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009230 tx=0x7f5f64009260 comp rx=0 tx=0).stop 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 shutdown_connections 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0ac7b60 0x558eb0acc800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f4ffff640 0 client.15246 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab2e40 0x558eb0abc6a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2451531393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 >> 172.21.15.44:0/2451531393 conn(0x558eb0aadb70 msgr2=0x558eb0aaf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 shutdown_connections 2024-09-13T17:26:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 1 -- 172.21.15.44:0/2451531393 wait complete. 2024-09-13T17:26:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.949+0000 7f5f79de1740 20 client.15246 trim_cache size 0 max 16384 2024-09-13T17:26:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 Processor -- start 2024-09-13T17:26:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 -- start start 2024-09-13T17:26:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0ac36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0abc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0aba790 con 0x558eb0ab7ed0 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0999460 con 0x558eb0ab2e40 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f79de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0abd1d0 con 0x558eb0ab2500 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.964+0000 7f5f75589640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0abc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0abc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58360/0 (socket says 172.21.15.44:58360) 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 -- 172.21.15.44:0/2036772849 learned_addr learned my addr 172.21.15.44:0/2036772849 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0ac36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 -- 172.21.15.44:0/2036772849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 msgr2=0x558eb0abbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 -- 172.21.15.44:0/2036772849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 msgr2=0x558eb0ac36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0ac36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 -- 172.21.15.44:0/2036772849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eb09b65f0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0ac36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f75589640 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0abc450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5f68098040 tx=0x7f5f68000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6864cd3ec22c02e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f577fe640 1 -- 172.21.15.44:0/2036772849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68005490 con 0x558eb0ab7ed0 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f577fe640 1 -- 172.21.15.44:0/2036772849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f68099f20 con 0x558eb0ab7ed0 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.965+0000 7f5f577fe640 1 -- 172.21.15.44:0/2036772849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68099410 con 0x558eb0ab7ed0 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- 172.21.15.44:0/2036772849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 msgr2=0x558eb0abc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0abc450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5f68098040 tx=0x7f5f68000f40 comp rx=0 tx=0).stop 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- 172.21.15.44:0/2036772849 shutdown_connections 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0abc450 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0ac36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2036772849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- 172.21.15.44:0/2036772849 >> 172.21.15.44:0/2036772849 conn(0x558eb0aadb70 msgr2=0x558eb0ab3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- 172.21.15.44:0/2036772849 shutdown_connections 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- 172.21.15.44:0/2036772849 wait complete. 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- start start 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 Processor -- start 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0abde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0ac3020 con 0x558eb0ab7ed0 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.966+0000 7f5f79de1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0ac3190 con 0x558eb0ab2e40 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f79de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb0ac3350 con 0x558eb0ab2500 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f79de1740 10 client.0 _ll_register_callbacks cb 0x7f5f6800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58376/0 (socket says 172.21.15.44:58376) 2024-09-13T17:26:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 -- 172.21.15.44:0/2685831038 learned_addr learned my addr 172.21.15.44:0/2685831038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f6ffff640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0abde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 -- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 msgr2=0x558eb0abd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 -- 172.21.15.44:0/2685831038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 msgr2=0x558eb0abde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0abde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 -- 172.21.15.44:0/2685831038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558eb09a24b0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f75589640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac2ae0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f680a1dc0 tx=0x7f5f680984d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67eb58d3787d9a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2685831038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f680a2060 con 0x558eb0ab7ed0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2685831038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f5f680a22f0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f680a31b0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2685831038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f6809dca0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f79de1740 10 client.15252 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.967+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558eb09dbd00 con 0x558eb0ab7ed0 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2685831038 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f680a31b0 con 0x558eb0ab7ed0 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2685831038 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f68018030 con 0x558eb0ab7ed0 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f79de1740 10 client.15252 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f79de1740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f79de1740 20 client.15252 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0b22350 0x558eb0b1ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f79de1740 4 client.15252 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558eb076dd10 con 0x558eb0b22350 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.968+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0b22350 0x558eb0b1ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f4ffff640 10 client.15252 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f74d88640 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0b22350 0x558eb0b1ee30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abcd83ed53ed6892 server_cookie=8a386038cd8b1327 in_seq=0 out_seq=0 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f4ffff640 1 -- 172.21.15.44:0/2685831038 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558eb076dd10 con 0x558eb0b22350 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f4ffff640 10 client.15252 handle_command_reply: tid=0 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 client.15252 shutdown 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 msgr2=0x558eb0ac2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac2ae0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f680a1dc0 tx=0x7f5f680984d0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 shutdown_connections 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558eb0b22350 0x558eb0b1ee30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eb0ab7ed0 0x558eb0ac2ae0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558eb0ab2e40 0x558eb0abde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 --2- 172.21.15.44:0/2685831038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558eb0ab2500 0x558eb0abd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 >> 172.21.15.44:0/2685831038 conn(0x558eb0aadb70 msgr2=0x558eb0ab6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f4ffff640 0 client.15252 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 shutdown_connections 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 1 -- 172.21.15.44:0/2685831038 wait complete. 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.969+0000 7f5f79de1740 20 client.15252 trim_cache size 0 max 16384 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24496, 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:26:10.979 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:26:10.980 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 85.383652982000001, 2024-09-13T17:26:10.980 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 24, 2024-09-13T17:26:10.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 92, 2024-09-13T17:26:10.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 90, 2024-09-13T17:26:10.980 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 127.105958374 2024-09-13T17:26:10.980 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5f580a11a0 msgr2=0x7f5f580a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5f580a11a0 0x7f5f580a3660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f64005ec0 tx=0x7f5f64005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 msgr2=0x7f5f7019d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 0x7f5f7019d1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800eaa0 tx=0x7f5f6800ef60 comp rx=0 tx=0).stop 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 shutdown_connections 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f5f580a11a0 0x7f5f580a3660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700ffe60 0x7f5f701961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f70100820 0x7f5f7019d1e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 --2- 172.21.15.44:0/2012630683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700fec70 0x7f5f70195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 >> 172.21.15.44:0/2012630683 conn(0x7f5f700fa410 msgr2=0x7f5f700675d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 shutdown_connections 2024-09-13T17:26:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:10.970+0000 7f5f76fec640 1 -- 172.21.15.44:0/2012630683 wait complete. 2024-09-13T17:26:10.983 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T17:26:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:10 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3204325628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:10 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1838959567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:10 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2952222544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 -- 172.21.15.44:0/2748689069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fffb0 msgr2=0x7fa060106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2748689069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fffb0 0x7fa060106840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa05800adb0 tx=0x7fa05802b7a0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 -- 172.21.15.44:0/2748689069 shutdown_connections 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2748689069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0600ff5f0 0x7fa0600ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2748689069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fffb0 0x7fa060106840 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2748689069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600fe400 0x7fa0600fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 -- 172.21.15.44:0/2748689069 >> 172.21.15.44:0/2748689069 conn(0x7fa0600f9b60 msgr2=0x7fa0600fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 -- 172.21.15.44:0/2748689069 shutdown_connections 2024-09-13T17:26:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 -- 172.21.15.44:0/2748689069 wait complete. 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 Processor -- start 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.084+0000 7fa067d1f640 1 -- start start 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 0x7fa060195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600ff5f0 0x7fa060196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0600fffb0 0x7fa06019d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06019d730 con 0x7fa0600fe400 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06019d8a0 con 0x7fa0600ff5f0 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06019da60 con 0x7fa0600fffb0 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 0x7fa060195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 0x7fa060195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58388/0 (socket says 172.21.15.44:58388) 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 -- 172.21.15.44:0/2757481488 learned_addr learned my addr 172.21.15.44:0/2757481488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0600fffb0 0x7fa06019d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600ff5f0 0x7fa060196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 -- 172.21.15.44:0/2757481488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0600fffb0 msgr2=0x7fa06019d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0600fffb0 0x7fa06019d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 -- 172.21.15.44:0/2757481488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600ff5f0 msgr2=0x7fa060196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600ff5f0 0x7fa060196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 -- 172.21.15.44:0/2757481488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05800aa10 con 0x7fa0600fe400 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600ff5f0 0x7fa060196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa065abb640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 0x7fa060195c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d950 tx=0x7fa05c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa056ffd640 1 -- 172.21.15.44:0/2757481488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c009c30 con 0x7fa0600fe400 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa056ffd640 1 -- 172.21.15.44:0/2757481488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa05c009d90 con 0x7fa0600fe400 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06019dd40 con 0x7fa0600fe400 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa056ffd640 1 -- 172.21.15.44:0/2757481488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00f6b0 con 0x7fa0600fe400 2024-09-13T17:26:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.085+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06019e1a0 con 0x7fa0600fe400 2024-09-13T17:26:11.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.087+0000 7fa056ffd640 1 -- 172.21.15.44:0/2757481488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa05c010460 con 0x7fa0600fe400 2024-09-13T17:26:11.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 Processor -- start 2024-09-13T17:26:11.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 -- start start 2024-09-13T17:26:11.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dc8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dcd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6dd0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6d237c0 con 0x5649d6dcded0 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6d32b70 con 0x5649d6dc8e40 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.088+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6d30300 con 0x5649d6dc8500 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6dd0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6dd0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58392/0 (socket says 172.21.15.44:58392) 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 -- 172.21.15.44:0/3443124671 learned_addr learned my addr 172.21.15.44:0/3443124671 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa056ffd640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa04c0a1120 0x7fa04c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa065abb640 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dcd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 -- 172.21.15.44:0/3443124671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 msgr2=0x5649d6dc8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dc8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 -- 172.21.15.44:0/3443124671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 msgr2=0x5649d6dcd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dcd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 -- 172.21.15.44:0/3443124671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d6cb84b0 con 0x5649d6dcded0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa04c0a1120 0x7fa04c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa056ffd640 1 -- 172.21.15.44:0/2757481488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fa05c0cd190 con 0x7fa0600fe400 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa0652ba640 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6dd0390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa05000baf0 tx=0x7fa05000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fadee38a92e69a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa04bfff640 1 -- 172.21.15.44:0/3443124671 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050017420 con 0x5649d6dcded0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa04bfff640 1 -- 172.21.15.44:0/3443124671 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa05000b5a0 con 0x5649d6dcded0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.089+0000 7fa04bfff640 1 -- 172.21.15.44:0/3443124671 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000f9d0 con 0x5649d6dcded0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 -- 172.21.15.44:0/3443124671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 msgr2=0x5649d6dd0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa04c0a1120 0x7fa04c0a35e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa058005cc0 tx=0x7fa058005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6dd0390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa05000baf0 tx=0x7fa05000d340 comp rx=0 tx=0).stop 2024-09-13T17:26:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 -- 172.21.15.44:0/3443124671 shutdown_connections 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6dd0390 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dcd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/3443124671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dc8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 -- 172.21.15.44:0/3443124671 >> 172.21.15.44:0/3443124671 conn(0x5649d6dc3b70 msgr2=0x5649d6dc3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 -- 172.21.15.44:0/3443124671 shutdown_connections 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 -- 172.21.15.44:0/3443124671 wait complete. 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 -- start start 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.090+0000 7fa06aa3f740 1 Processor -- start 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 0x5649d6dd20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dd25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dcded0 0x5649d6dd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6d2bbf0 con 0x5649d6dc8500 2024-09-13T17:26:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6cea250 con 0x5649d6dc8e40 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6ce9ef0 con 0x5649d6dcded0 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa06aa3f740 10 client.0 _ll_register_callbacks cb 0x5649d6d1c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 0x5649d6dd20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 0x5649d6dd20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58408/0 (socket says 172.21.15.44:58408) 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 -- 172.21.15.44:0/2312763083 learned_addr learned my addr 172.21.15.44:0/2312763083 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dd25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dcded0 0x5649d6dd9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 -- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dcded0 msgr2=0x5649d6dd9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dcded0 0x5649d6dd9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 -- 172.21.15.44:0/2312763083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 msgr2=0x5649d6dd25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dd25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 -- 172.21.15.44:0/2312763083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05000b720 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa065abb640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 0x5649d6dd20a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005fd0 tx=0x7fa05c010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ef47fc5d036be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.091+0000 7fa0497fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa0497fa640 1 -- 172.21.15.44:0/2312763083 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c09df10 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa0497fa640 1 -- 172.21.15.44:0/2312763083 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fa05c0a2560 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d6a306c0 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa0497fa640 1 -- 172.21.15.44:0/2312763083 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa05c0ca9f0 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5649d6d2a270 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa0497fa640 1 -- 172.21.15.44:0/2312763083 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5649d6a306c0 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa0497fa640 1 -- 172.21.15.44:0/2312763083 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa05c0a20d0 con 0x5649d6dc8500 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T17:26:11.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 20 client.15264 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6ddde90 0x5649d6de2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 4 client.15264 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.092+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649d6a83d10 con 0x5649d6ddde90 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.093+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6ddde90 0x5649d6de2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.093+0000 7fa0497fa640 10 client.15264 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.093+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6ddde90 0x5649d6de2620 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fbd8ad66ae2346 server_cookie=9751e4c1a5bdccd7 in_seq=0 out_seq=0 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa0497fa640 1 -- 172.21.15.44:0/2312763083 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5649d6a83d10 con 0x5649d6ddde90 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa0497fa640 10 client.15264 handle_command_reply: tid=0 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 client.15264 shutdown 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 msgr2=0x5649d6dd20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 0x5649d6dd20a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005fd0 tx=0x7fa05c010c90 comp rx=0 tx=0).stop 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 shutdown_connections 2024-09-13T17:26:11.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6ddde90 0x5649d6de2620 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dcded0 0x5649d6dd9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6dd25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2312763083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8500 0x5649d6dd20a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa0497fa640 0 client.15264 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 >> 172.21.15.44:0/2312763083 conn(0x5649d6dc3b70 msgr2=0x5649d6dc5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 shutdown_connections 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2312763083 wait complete. 2024-09-13T17:26:11.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.094+0000 7fa06aa3f740 20 client.15264 trim_cache size 0 max 16384 2024-09-13T17:26:11.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 Processor -- start 2024-09-13T17:26:11.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 -- start start 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dd2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6de6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6de6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6caf460 con 0x5649d6dcded0 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6a83d10 con 0x5649d6dc8e40 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6a306c0 con 0x5649d6dc8500 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6de6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6de6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58422/0 (socket says 172.21.15.44:58422) 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 -- 172.21.15.44:0/2104067399 learned_addr learned my addr 172.21.15.44:0/2104067399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 -- 172.21.15.44:0/2104067399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 msgr2=0x5649d6dd2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6de6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa065abb640 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dd2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dd2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 -- 172.21.15.44:0/2104067399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 msgr2=0x5649d6de6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6de6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 -- 172.21.15.44:0/2104067399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d6ccc5f0 con 0x5649d6dcded0 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6de6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.109+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6de6b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa05800adb0 tx=0x7fa05803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feab6b7b55b16b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa054ff9640 1 -- 172.21.15.44:0/2104067399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058004040 con 0x5649d6dcded0 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa054ff9640 1 -- 172.21.15.44:0/2104067399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa058041070 con 0x5649d6dcded0 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa054ff9640 1 -- 172.21.15.44:0/2104067399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058030c20 con 0x5649d6dcded0 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2104067399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 msgr2=0x5649d6de6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6de6b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa05800adb0 tx=0x7fa05803f000 comp rx=0 tx=0).stop 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2104067399 shutdown_connections 2024-09-13T17:26:11.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dcded0 0x5649d6de6b80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dc8e40 0x5649d6de6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2104067399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6dd2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2104067399 >> 172.21.15.44:0/2104067399 conn(0x5649d6dc3b70 msgr2=0x5649d6dd1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2104067399 shutdown_connections 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2104067399 wait complete. 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 -- start start 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.110+0000 7fa06aa3f740 1 Processor -- start 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6e31570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 0x5649d6e31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dcded0 0x5649d6e38af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6e39030 con 0x5649d6dc8e40 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6e39170 con 0x5649d6dcded0 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d6e39300 con 0x5649d6dc8500 2024-09-13T17:26:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa06aa3f740 10 client.0 _ll_register_callbacks cb 0x7fa05000b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 0x5649d6e31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 0x5649d6e31ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58438/0 (socket says 172.21.15.44:58438) 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 -- 172.21.15.44:0/2975139551 learned_addr learned my addr 172.21.15.44:0/2975139551 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0662bc640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dcded0 0x5649d6e38af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa065abb640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6e31570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 -- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 msgr2=0x5649d6e31570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6e31570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 -- 172.21.15.44:0/2975139551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dcded0 msgr2=0x5649d6e38af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dcded0 0x5649d6e38af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 -- 172.21.15.44:0/2975139551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060107140 con 0x5649d6dc8e40 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa065abb640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6e31570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0652ba640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 0x5649d6e31ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5649d6dc49f0 tx=0x7fa05000f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bec15e241956fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.111+0000 7fa0497fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:11.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa0497fa640 1 -- 172.21.15.44:0/2975139551 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050027040 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa0497fa640 1 -- 172.21.15.44:0/2975139551 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fa050013070 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d6a1f3c0 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa0497fa640 1 -- 172.21.15.44:0/2975139551 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa05000b5a0 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 10 client.15270 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5649d6cf1d00 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa0497fa640 1 -- 172.21.15.44:0/2975139551 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5649d6a1f3c0 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa0497fa640 1 -- 172.21.15.44:0/2975139551 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa05001f040 con 0x5649d6dc8e40 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 10 client.15270 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 20 client.15270 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6e3e0d0 0x5649d6e3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 4 client.15270 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:26:11.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5649d6dde8b0 con 0x5649d6e3e0d0 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.112+0000 7fa065abb640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6e3e0d0 0x5649d6e3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa065abb640 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6e3e0d0 0x5649d6e3aa20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39018343409c39c9 server_cookie=d5a4ac6385d42ae5 in_seq=0 out_seq=0 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa0497fa640 10 client.15270 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa0497fa640 1 -- 172.21.15.44:0/2975139551 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7fa05c099c20 con 0x5649d6e3e0d0 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa0497fa640 10 client.15270 handle_command_reply: tid=0 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 client.15270 shutdown 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 msgr2=0x5649d6e31ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 0x5649d6e31ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5649d6dc49f0 tx=0x7fa05000f440 comp rx=0 tx=0).stop 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 shutdown_connections 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5649d6e3e0d0 0x5649d6e3aa20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649d6dcded0 0x5649d6e38af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d6dc8e40 0x5649d6e31ab0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa0497fa640 0 client.15270 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 --2- 172.21.15.44:0/2975139551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5649d6dc8500 0x5649d6e31570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 >> 172.21.15.44:0/2975139551 conn(0x5649d6dc3b70 msgr2=0x5649d6de6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 shutdown_connections 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 1 -- 172.21.15.44:0/2975139551 wait complete. 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.113+0000 7fa06aa3f740 20 client.15270 trim_cache size 0 max 16384 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T17:26:11.122 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24517, 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 109.389304439, 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 24, 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 92, 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 90, 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 122.284492593 2024-09-13T17:26:11.123 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa04c0a1120 msgr2=0x7fa04c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa04c0a1120 0x7fa04c0a35e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa058005cc0 tx=0x7fa058005c30 comp rx=0 tx=0).stop 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 msgr2=0x7fa060195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 0x7fa060195c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d950 tx=0x7fa05c00de10 comp rx=0 tx=0).stop 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 shutdown_connections 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa04c0a1120 0x7fa04c0a35e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0600fffb0 0x7fa06019d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0600ff5f0 0x7fa060196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 --2- 172.21.15.44:0/2757481488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0600fe400 0x7fa060195c50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:11.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 >> 172.21.15.44:0/2757481488 conn(0x7fa0600f9b60 msgr2=0x7fa060102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 shutdown_connections 2024-09-13T17:26:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:11.114+0000 7fa067d1f640 1 -- 172.21.15.44:0/2757481488 wait complete. 2024-09-13T17:26:11.127 INFO:tasks.cephadm:Setup complete, yielding 2024-09-13T17:26:11.127 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-13T17:26:11.146 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:26:11.146 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-13T17:26:11.347 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:11.725 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:11 smithi044 ceph-mon[31175]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:11.725 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:11 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/872515486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:11.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:11 smithi070 ceph-mon[34502]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:11.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:11 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/872515486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:11 smithi100 ceph-mon[35055]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:12.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:11 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/872515486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 -- 172.21.15.44:0/4106414886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d00ffe00 msgr2=0x7f61d0100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 --2- 172.21.15.44:0/4106414886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d00ffe00 0x7f61d0100200 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 -- 172.21.15.44:0/4106414886 shutdown_connections 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 --2- 172.21.15.44:0/4106414886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61d0100f60 0x7f61d01013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 --2- 172.21.15.44:0/4106414886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 0x7f61d0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 --2- 172.21.15.44:0/4106414886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d00ffe00 0x7f61d0100200 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 -- 172.21.15.44:0/4106414886 >> 172.21.15.44:0/4106414886 conn(0x7f61d00fb5d0 msgr2=0x7f61d00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 -- 172.21.15.44:0/4106414886 shutdown_connections 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.233+0000 7f61d74d8640 1 -- 172.21.15.44:0/4106414886 wait complete. 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 Processor -- start 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 -- start start 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61d00ffe00 0x7f61d0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 0x7f61d00728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 0x7f61d0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0073570 con 0x7f61d0100f60 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00736e0 con 0x7f61d0101920 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d74d8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d019fd90 con 0x7f61d00ffe00 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 0x7f61d00728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 0x7f61d00728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58452/0 (socket says 172.21.15.44:58452) 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 -- 172.21.15.44:0/2056576504 learned_addr learned my addr 172.21.15.44:0/2056576504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d5274640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61d00ffe00 0x7f61d0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d5a75640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 0x7f61d0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 -- 172.21.15.44:0/2056576504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61d00ffe00 msgr2=0x7f61d0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61d00ffe00 0x7f61d0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 -- 172.21.15.44:0/2056576504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 msgr2=0x7f61d0072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:13.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 0x7f61d0072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d4a73640 1 -- 172.21.15.44:0/2056576504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009560 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.234+0000 7f61d5a75640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 0x7f61d0072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.235+0000 7f61d4a73640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 0x7f61d00728a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f61c000b4b0 tx=0x7f61c000b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.235+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0010610 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.235+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61c0010c00 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.235+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d019ffe0 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.235+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0017bb0 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.235+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d01a04a0 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.236+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d0107330 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.237+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f61c0017d50 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.237+0000 7f61c67fc640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f61bc0a1100 0x7f61bc0a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.238+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f61c00cc210 con 0x7f61d0100f60 2024-09-13T17:26:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.238+0000 7f61d5274640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f61bc0a1100 0x7f61bc0a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.238+0000 7f61d5274640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f61bc0a1100 0x7f61bc0a35c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f61cc02aa00 tx=0x7f61cc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.242+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f61c0095600 con 0x7f61d0100f60 2024-09-13T17:26:13.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.429+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61d002c4f0 con 0x7f61bc0a1100 2024-09-13T17:26:13.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.431+0000 7f61c67fc640 1 -- 172.21.15.44:0/2056576504 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f61d002c4f0 con 0x7f61bc0a1100 2024-09-13T17:26:13.433 INFO:teuthology.orchestra.run.smithi044.stdout:Backend: cephadm 2024-09-13T17:26:13.433 INFO:teuthology.orchestra.run.smithi044.stdout:Available: Yes 2024-09-13T17:26:13.434 INFO:teuthology.orchestra.run.smithi044.stdout:Paused: No 2024-09-13T17:26:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f61bc0a1100 msgr2=0x7f61bc0a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f61bc0a1100 0x7f61bc0a35c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f61cc02aa00 tx=0x7f61cc005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 msgr2=0x7f61d00728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 0x7f61d00728a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f61c000b4b0 tx=0x7f61c000b970 comp rx=0 tx=0).stop 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 shutdown_connections 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f61bc0a1100 0x7f61bc0a35c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d0101920 0x7f61d0072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d0100f60 0x7f61d00728a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 --2- 172.21.15.44:0/2056576504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61d00ffe00 0x7f61d0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 >> 172.21.15.44:0/2056576504 conn(0x7f61d00fb5d0 msgr2=0x7f61d0104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 shutdown_connections 2024-09-13T17:26:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:13.433+0000 7f61d74d8640 1 -- 172.21.15.44:0/2056576504 wait complete. 2024-09-13T17:26:13.746 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:13 smithi044 ceph-mon[31175]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:13.746 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:13 smithi044 ceph-mon[31175]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-09-13T17:26:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:13 smithi070 ceph-mon[34502]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:13 smithi070 ceph-mon[34502]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-09-13T17:26:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:13 smithi100 ceph-mon[35055]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:13 smithi100 ceph-mon[35055]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-09-13T17:26:14.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-13T17:26:14.303 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:14.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:14 smithi044 ceph-mon[31175]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:14.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:14 smithi070 ceph-mon[34502]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:15.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:14 smithi100 ceph-mon[35055]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:15.728 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:15 smithi044 ceph-mon[31175]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:26:15.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 -- 172.21.15.44:0/4160947851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 msgr2=0x7f0428102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:15.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 --2- 172.21.15.44:0/4160947851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428102c60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0418009ca0 tx=0x7f041802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:15.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 -- 172.21.15.44:0/4160947851 shutdown_connections 2024-09-13T17:26:15.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 --2- 172.21.15.44:0/4160947851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0428105ad0 0x7f0428107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:15.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 --2- 172.21.15.44:0/4160947851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04281031a0 0x7f0428105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:15.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 --2- 172.21.15.44:0/4160947851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428102c60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 -- 172.21.15.44:0/4160947851 >> 172.21.15.44:0/4160947851 conn(0x7f04280fa490 msgr2=0x7f04280fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 -- 172.21.15.44:0/4160947851 shutdown_connections 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 -- 172.21.15.44:0/4160947851 wait complete. 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 Processor -- start 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.932+0000 7f04301f6640 1 -- start start 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f04301f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f04301f6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04281031a0 0x7f0428196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f04301f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0428105ad0 0x7f042819d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f04301f6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042819d830 con 0x7f0428100870 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f04301f6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042819d9a0 con 0x7f04281031a0 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f04301f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042819db60 con 0x7f0428105ad0 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:15.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58474/0 (socket says 172.21.15.44:58474) 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 -- 172.21.15.44:0/649289233 learned_addr learned my addr 172.21.15.44:0/649289233 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042d791640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04281031a0 0x7f0428196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042e793640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0428105ad0 0x7f042819d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 -- 172.21.15.44:0/649289233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0428105ad0 msgr2=0x7f042819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0428105ad0 0x7f042819d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 -- 172.21.15.44:0/649289233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04281031a0 msgr2=0x7f0428196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04281031a0 0x7f0428196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 -- 172.21.15.44:0/649289233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418009900 con 0x7f0428100870 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f042df92640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428195d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0418009dc0 tx=0x7f0418003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.933+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180307c0 con 0x7f0428100870 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.934+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0418040070 con 0x7f0428100870 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.934+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042819dde0 con 0x7f0428100870 2024-09-13T17:26:15.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.934+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180044f0 con 0x7f0428100870 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.934+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042819e2d0 con 0x7f0428100870 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.935+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04280fbf80 con 0x7f0428100870 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.936+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0418042030 con 0x7f0428100870 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.936+0000 7f0426ffd640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f04100a1100 0x7f04100a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.937+0000 7f042d791640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f04100a1100 0x7f04100a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.937+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f04180ee3d0 con 0x7f0428100870 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.937+0000 7f042d791640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f04100a1100 0x7f04100a35c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0428068a60 tx=0x7f041c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:15.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:15.940+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f04180b77c0 con 0x7f0428100870 2024-09-13T17:26:15.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:15 smithi070 ceph-mon[34502]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:26:16.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:15 smithi100 ceph-mon[35055]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-13T17:26:16.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.119+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f042802c4f0 con 0x7f04100a1100 2024-09-13T17:26:16.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.131+0000 7f0426ffd640 1 -- 172.21.15.44:0/649289233 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f042802c4f0 con 0x7f04100a1100 2024-09-13T17:26:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-13T17:26:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:mds.a smithi044 running (2m) 26s ago 2m 16.6M - 18.2.4-860-g6c6b083f 8a91f241d7d4 5c56a4017604 2024-09-13T17:26:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:mds.b smithi070 running (119s) 54s ago 118s 16.2M - 18.2.4-860-g6c6b083f 8a91f241d7d4 e228f8664847 2024-09-13T17:26:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:mds.c smithi100 running (2m) 54s ago 2m 16.2M - 18.2.4-860-g6c6b083f 8a91f241d7d4 f9613c3f418c 2024-09-13T17:26:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:mds.d smithi044 running (2m) 26s ago 2m 14.2M - 18.2.4-860-g6c6b083f 8a91f241d7d4 94e9650c1045 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.e smithi070 running (2m) 54s ago 2m 13.3M - 18.2.4-860-g6c6b083f 8a91f241d7d4 ab98e581e5b3 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.f smithi100 running (2m) 54s ago 2m 17.5M - 18.2.4-860-g6c6b083f 8a91f241d7d4 34b7c635ed72 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.g smithi044 running (2m) 26s ago 2m 13.9M - 18.2.4-860-g6c6b083f 8a91f241d7d4 584279830ebf 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.h smithi070 running (2m) 54s ago 2m 17.4M - 18.2.4-860-g6c6b083f 8a91f241d7d4 3c9933e75de7 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.i smithi100 running (2m) 54s ago 2m 25.8M - 18.2.4-860-g6c6b083f 8a91f241d7d4 54717c131dc7 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.j smithi044 running (2m) 26s ago 2m 15.8M - 18.2.4-860-g6c6b083f 8a91f241d7d4 caa882e3ae12 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.k smithi070 running (2m) 54s ago 2m 14.2M - 18.2.4-860-g6c6b083f 8a91f241d7d4 fb48a9aeda9f 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mds.l smithi100 running (2m) 54s ago 2m 16.3M - 18.2.4-860-g6c6b083f 8a91f241d7d4 feff28de37ff 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.x smithi044 *:9283,8765 running (9m) 26s ago 9m 497M - 18.2.4-860-g6c6b083f 8a91f241d7d4 09db5f4cccf0 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.y smithi070 *:8443,8765 running (7m) 54s ago 7m 442M - 18.2.4-860-g6c6b083f 8a91f241d7d4 e0a4f1005dfb 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.z smithi100 *:8443,8765 running (7m) 54s ago 7m 442M - 18.2.4-860-g6c6b083f 8a91f241d7d4 f5f8c6077225 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a smithi044 running (9m) 26s ago 9m 49.8M 2048M 18.2.4-860-g6c6b083f 8a91f241d7d4 1c73e264c322 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b smithi070 running (7m) 54s ago 7m 39.0M 2048M 18.2.4-860-g6c6b083f 8a91f241d7d4 2596fd5d3a2a 2024-09-13T17:26:16.134 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c smithi100 running (7m) 54s ago 7m 40.1M 2048M 18.2.4-860-g6c6b083f 8a91f241d7d4 7b5c424e19e5 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 smithi044 running (7m) 26s ago 7m 47.2M 4273M 18.2.4-860-g6c6b083f 8a91f241d7d4 2f3550ac5cc0 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 smithi070 running (6m) 54s ago 6m 57.4M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 5eb1a2557fe6 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 smithi100 running (6m) 54s ago 6m 57.1M 4271M 18.2.4-860-g6c6b083f 8a91f241d7d4 03562ee101f7 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 smithi044 running (6m) 26s ago 5m 55.5M 4273M 18.2.4-860-g6c6b083f 8a91f241d7d4 c6ad6fa2b107 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 smithi070 running (5m) 54s ago 5m 53.9M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 5a5040a244e5 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 smithi100 running (5m) 54s ago 5m 44.8M 4271M 18.2.4-860-g6c6b083f 8a91f241d7d4 37d5cc92d168 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 smithi044 running (4m) 26s ago 4m 41.8M 4273M 18.2.4-860-g6c6b083f 8a91f241d7d4 543b5d1d4f06 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 smithi070 running (4m) 54s ago 4m 54.9M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 31d5bbd383f5 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 smithi100 running (4m) 54s ago 4m 44.1M 4271M 18.2.4-860-g6c6b083f 8a91f241d7d4 e6abddf68c7f 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 smithi044 running (3m) 26s ago 3m 44.2M 4273M 18.2.4-860-g6c6b083f 8a91f241d7d4 97a6d2ae3fa9 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 smithi070 running (3m) 54s ago 3m 54.1M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 eacdb723fa3c 2024-09-13T17:26:16.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 smithi100 running (2m) 54s ago 2m 42.2M 4271M 18.2.4-860-g6c6b083f 8a91f241d7d4 b9233b7d026a 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f04100a1100 msgr2=0x7f04100a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f04100a1100 0x7f04100a35c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0428068a60 tx=0x7f041c00b390 comp rx=0 tx=0).stop 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 msgr2=0x7f0428195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428195d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0418009dc0 tx=0x7f0418003c20 comp rx=0 tx=0).stop 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 shutdown_connections 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f04100a1100 0x7f04100a35c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0428105ad0 0x7f042819d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:16.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04281031a0 0x7f0428196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:16.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 --2- 172.21.15.44:0/649289233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0428100870 0x7f0428195d40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:16.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 >> 172.21.15.44:0/649289233 conn(0x7f04280fa490 msgr2=0x7f0428101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:16.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 shutdown_connections 2024-09-13T17:26:16.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:16.133+0000 7f04301f6640 1 -- 172.21.15.44:0/649289233 wait complete. 2024-09-13T17:26:16.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:16 smithi044 ceph-mon[31175]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-13T17:26:16.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:16 smithi044 ceph-mon[31175]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:16.920 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-13T17:26:16.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:16 smithi070 ceph-mon[34502]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-13T17:26:16.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:16 smithi070 ceph-mon[34502]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:17.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:16 smithi100 ceph-mon[35055]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-13T17:26:17.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:16 smithi100 ceph-mon[35055]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:18.888 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:18 smithi044 ceph-mon[31175]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.885+0000 7f678064e640 1 -- 172.21.15.44:0/889151905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67780ffe00 msgr2=0x7f6778100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.885+0000 7f678064e640 1 --2- 172.21.15.44:0/889151905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67780ffe00 0x7f6778100200 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6774009900 tx=0x7f677402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 -- 172.21.15.44:0/889151905 shutdown_connections 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 --2- 172.21.15.44:0/889151905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6778100f60 0x7f67781013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 --2- 172.21.15.44:0/889151905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6778101920 0x7f6778108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 --2- 172.21.15.44:0/889151905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67780ffe00 0x7f6778100200 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 -- 172.21.15.44:0/889151905 >> 172.21.15.44:0/889151905 conn(0x7f67780fb5d0 msgr2=0x7f67780fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 -- 172.21.15.44:0/889151905 shutdown_connections 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 -- 172.21.15.44:0/889151905 wait complete. 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 Processor -- start 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.886+0000 7f678064e640 1 -- start start 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f678064e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67780ffe00 0x7f6778195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f678064e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6778100f60 0x7f67781961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f678064e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6778101920 0x7f677819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f678064e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677819d770 con 0x7f6778100f60 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f678064e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677819d8e0 con 0x7f6778101920 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f678064e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677819daa0 con 0x7f67780ffe00 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677e3ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67780ffe00 0x7f6778195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6778100f60 0x7f67781961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677e3ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67780ffe00 0x7f6778195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:59004/0 (socket says 172.21.15.44:59004) 2024-09-13T17:26:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677e3ea640 1 -- 172.21.15.44:0/1841262522 learned_addr learned my addr 172.21.15.44:0/1841262522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677ebeb640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6778101920 0x7f677819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 -- 172.21.15.44:0/1841262522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67780ffe00 msgr2=0x7f6778195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67780ffe00 0x7f6778195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 -- 172.21.15.44:0/1841262522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6778101920 msgr2=0x7f677819d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6778101920 0x7f677819d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 -- 172.21.15.44:0/1841262522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774009560 con 0x7f6778100f60 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.887+0000 7f677dbe9640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6778100f60 0x7f67781961c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f676800b3c0 tx=0x7f676800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.888+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768011830 con 0x7f6778100f60 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.888+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6768011e20 con 0x7f6778100f60 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.888+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677819dd80 con 0x7f6778100f60 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.888+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768010e50 con 0x7f6778100f60 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.888+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677819e1c0 con 0x7f6778100f60 2024-09-13T17:26:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.889+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6778107280 con 0x7f6778100f60 2024-09-13T17:26:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.890+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6768011990 con 0x7f6778100f60 2024-09-13T17:26:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.891+0000 7f676f7fe640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67640a1100 0x7f67640a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.891+0000 7f677e3ea640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67640a1100 0x7f67640a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.891+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f67680cc2c0 con 0x7f6778100f60 2024-09-13T17:26:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.892+0000 7f677e3ea640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67640a1100 0x7f67640a35c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6774000c00 tx=0x7f6774005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:18.895+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6768095710 con 0x7f6778100f60 2024-09-13T17:26:18.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:18 smithi070 ceph-mon[34502]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:19.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:18 smithi100 ceph-mon[35055]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:19.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.075+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f677802c4f0 con 0x7f67640a1100 2024-09-13T17:26:19.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.080+0000 7f676f7fe640 1 -- 172.21.15.44:0/1841262522 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f677802c4f0 con 0x7f67640a1100 2024-09-13T17:26:19.083 INFO:teuthology.orchestra.run.smithi044.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-13T17:26:19.083 INFO:teuthology.orchestra.run.smithi044.stdout:mds.all 12/12 57s ago 2m smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi100=c;smithi100=f;smithi100=i;smithi100=l;count:12 2024-09-13T17:26:19.083 INFO:teuthology.orchestra.run.smithi044.stdout:mgr 3/3 57s ago 7m smithi044=x;smithi070=y;smithi100=z;count:3 2024-09-13T17:26:19.083 INFO:teuthology.orchestra.run.smithi044.stdout:mon 3/3 57s ago 7m smithi044:172.21.15.44=a;smithi070:172.21.15.70=b;smithi100:172.21.15.100=c;count:3 2024-09-13T17:26:19.083 INFO:teuthology.orchestra.run.smithi044.stdout:osd 12 57s ago - 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67640a1100 msgr2=0x7f67640a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67640a1100 0x7f67640a35c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6774000c00 tx=0x7f6774005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6778100f60 msgr2=0x7f67781961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6778100f60 0x7f67781961c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f676800b3c0 tx=0x7f676800b880 comp rx=0 tx=0).stop 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 shutdown_connections 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f67640a1100 0x7f67640a35c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6778101920 0x7f677819d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6778100f60 0x7f67781961c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.082+0000 7f678064e640 1 --2- 172.21.15.44:0/1841262522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67780ffe00 0x7f6778195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.083+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 >> 172.21.15.44:0/1841262522 conn(0x7f67780fb5d0 msgr2=0x7f6778104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.083+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 shutdown_connections 2024-09-13T17:26:19.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:19.083+0000 7f678064e640 1 -- 172.21.15.44:0/1841262522 wait complete. 2024-09-13T17:26:19.841 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:19 smithi044 ceph-mon[31175]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:19.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-13T17:26:19.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:19 smithi070 ceph-mon[34502]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:20.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:19 smithi100 ceph-mon[35055]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:20.081 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:20.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:20 smithi044 ceph-mon[31175]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:20.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:20 smithi070 ceph-mon[34502]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:21.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:20 smithi100 ceph-mon[35055]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:21.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 -- 172.21.15.44:0/2627503609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c100290 msgr2=0x7f875c100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:21.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 --2- 172.21.15.44:0/2627503609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c100290 0x7f875c100690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8758009930 tx=0x7f875802a420 comp rx=0 tx=0).stop 2024-09-13T17:26:21.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 -- 172.21.15.44:0/2627503609 shutdown_connections 2024-09-13T17:26:21.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 --2- 172.21.15.44:0/2627503609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c0ff990 0x7f875c107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:21.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 --2- 172.21.15.44:0/2627503609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875c0fefd0 0x7f875c0ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 --2- 172.21.15.44:0/2627503609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c100290 0x7f875c100690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 -- 172.21.15.44:0/2627503609 >> 172.21.15.44:0/2627503609 conn(0x7f875c0f9c10 msgr2=0x7f875c0fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 -- 172.21.15.44:0/2627503609 shutdown_connections 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.871+0000 7f8763bc5640 1 -- 172.21.15.44:0/2627503609 wait complete. 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 Processor -- start 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 -- start start 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875c0fefd0 0x7f875c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 0x7f875c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c100290 0x7f875c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c19d860 con 0x7f875c0ff990 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c19d9d0 con 0x7f875c0fefd0 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8763bc5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c19db90 con 0x7f875c100290 2024-09-13T17:26:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8761160640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 0x7f875c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8761160640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 0x7f875c1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55760/0 (socket says 172.21.15.44:55760) 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.872+0000 7f8761160640 1 -- 172.21.15.44:0/1627406813 learned_addr learned my addr 172.21.15.44:0/1627406813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761961640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875c0fefd0 0x7f875c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761160640 1 -- 172.21.15.44:0/1627406813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c100290 msgr2=0x7f875c19d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8762162640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c100290 0x7f875c19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761160640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c100290 0x7f875c19d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761160640 1 -- 172.21.15.44:0/1627406813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875c0fefd0 msgr2=0x7f875c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761160640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875c0fefd0 0x7f875c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761160640 1 -- 172.21.15.44:0/1627406813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009560 con 0x7f875c0ff990 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8761160640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 0x7f875c1962a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f874c00d950 tx=0x7f874c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c009c30 con 0x7f875c0ff990 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f874c009d90 con 0x7f875c0ff990 2024-09-13T17:26:21.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00f620 con 0x7f875c0ff990 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c19de70 con 0x7f875c0ff990 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.873+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c19e290 con 0x7f875c0ff990 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.874+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c10a4c0 con 0x7f875c0ff990 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.875+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f874c0109f0 con 0x7f875c0ff990 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.876+0000 7f8752ffd640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87480a10b0 0x7f87480a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.876+0000 7f8761961640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87480a10b0 0x7f87480a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.876+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f874c0cc840 con 0x7f875c0ff990 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.876+0000 7f8761961640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87480a10b0 0x7f87480a3570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8758009900 tx=0x7f8758000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:21.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:21.880+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f874c095c30 con 0x7f875c0ff990 2024-09-13T17:26:22.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.057+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f875c19e480 con 0x7f87480a10b0 2024-09-13T17:26:22.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.059+0000 7f8752ffd640 1 -- 172.21.15.44:0/1627406813 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f875c19e480 con 0x7f87480a10b0 2024-09-13T17:26:22.061 INFO:teuthology.orchestra.run.smithi044.stdout:HOST ADDR LABELS STATUS 2024-09-13T17:26:22.061 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 172.21.15.44 2024-09-13T17:26:22.061 INFO:teuthology.orchestra.run.smithi044.stdout:smithi070 172.21.15.70 2024-09-13T17:26:22.061 INFO:teuthology.orchestra.run.smithi044.stdout:smithi100 172.21.15.100 2024-09-13T17:26:22.061 INFO:teuthology.orchestra.run.smithi044.stdout:3 hosts in cluster 2024-09-13T17:26:22.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87480a10b0 msgr2=0x7f87480a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:22.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87480a10b0 0x7f87480a3570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8758009900 tx=0x7f8758000bc0 comp rx=0 tx=0).stop 2024-09-13T17:26:22.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 msgr2=0x7f875c1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:22.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 0x7f875c1962a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f874c00d950 tx=0x7f874c00de10 comp rx=0 tx=0).stop 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 shutdown_connections 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f87480a10b0 0x7f87480a3570 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f875c100290 0x7f875c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f875c0ff990 0x7f875c1962a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 --2- 172.21.15.44:0/1627406813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875c0fefd0 0x7f875c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 >> 172.21.15.44:0/1627406813 conn(0x7f875c0f9c10 msgr2=0x7f875c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 shutdown_connections 2024-09-13T17:26:22.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:22.061+0000 7f8763bc5640 1 -- 172.21.15.44:0/1627406813 wait complete. 2024-09-13T17:26:22.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-13T17:26:22.931 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:22.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:22 smithi070 ceph-mon[34502]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-13T17:26:22.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:22 smithi070 ceph-mon[34502]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:23.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:22 smithi100 ceph-mon[35055]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-13T17:26:23.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:22 smithi100 ceph-mon[35055]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:23.052 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:22 smithi044 ceph-mon[31175]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-13T17:26:23.053 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:22 smithi044 ceph-mon[31175]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 -- 172.21.15.44:0/3285109443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 msgr2=0x7f1e80108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 --2- 172.21.15.44:0/3285109443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e80108240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009900 tx=0x7f1e7802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 -- 172.21.15.44:0/3285109443 shutdown_connections 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 --2- 172.21.15.44:0/3285109443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e80108240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 --2- 172.21.15.44:0/3285109443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e80100f60 0x7f1e801013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 --2- 172.21.15.44:0/3285109443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e800ffe00 0x7f1e80100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 -- 172.21.15.44:0/3285109443 >> 172.21.15.44:0/3285109443 conn(0x7f1e800fb5d0 msgr2=0x7f1e800fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 -- 172.21.15.44:0/3285109443 shutdown_connections 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.741+0000 7f1e85b93640 1 -- 172.21.15.44:0/3285109443 wait complete. 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 Processor -- start 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 -- start start 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e800ffe00 0x7f1e80197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e80100f60 0x7f1e80198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e8019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8019f830 con 0x7f1e80101920 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8019f9a0 con 0x7f1e80100f60 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e85b93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8019fb60 con 0x7f1e800ffe00 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e7ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e8019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e7ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e8019f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55768/0 (socket says 172.21.15.44:55768) 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e7ffff640 1 -- 172.21.15.44:0/1744406557 learned_addr learned my addr 172.21.15.44:0/1744406557 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e7f7fe640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e800ffe00 0x7f1e80197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.742+0000 7f1e7effd640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e80100f60 0x7f1e80198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7ffff640 1 -- 172.21.15.44:0/1744406557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e800ffe00 msgr2=0x7f1e80197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7ffff640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e800ffe00 0x7f1e80197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7ffff640 1 -- 172.21.15.44:0/1744406557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e80100f60 msgr2=0x7f1e80198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7ffff640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e80100f60 0x7f1e80198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7ffff640 1 -- 172.21.15.44:0/1744406557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78009560 con 0x7f1e80101920 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7ffff640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e8019f2c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e7802aa00 tx=0x7f1e780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78031d70 con 0x7f1e80101920 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e780044e0 con 0x7f1e80101920 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8019fde0 con 0x7f1e80101920 2024-09-13T17:26:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78003b60 con 0x7f1e80101920 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.743+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e801a01c0 con 0x7f1e80101920 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.744+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8010b580 con 0x7f1e80101920 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.745+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1e78004c50 con 0x7f1e80101920 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.746+0000 7f1e7cff9640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e680a1080 0x7f1e680a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.746+0000 7f1e7f7fe640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e680a1080 0x7f1e680a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.746+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f1e780ee120 con 0x7f1e80101920 2024-09-13T17:26:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.747+0000 7f1e7f7fe640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e680a1080 0x7f1e680a3540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1e80199220 tx=0x7f1e700075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:24.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.749+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1e780b7620 con 0x7f1e80101920 2024-09-13T17:26:24.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:24 smithi044 ceph-mon[31175]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.929+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 --> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e80073e00 con 0x7f1e680a1080 2024-09-13T17:26:24.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.931+0000 7f1e7cff9640 1 -- 172.21.15.44:0/1744406557 <== mgr.14150 v2:172.21.15.44:6800/322597156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f1e80073e00 con 0x7f1e680a1080 2024-09-13T17:26:24.934 INFO:teuthology.orchestra.run.smithi044.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-13T17:26:24.934 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-13T17:26:24.934 INFO:teuthology.orchestra.run.smithi044.stdout:smithi070 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002P480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-13T17:26:24.934 INFO:teuthology.orchestra.run.smithi044.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-13T17:26:24.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.933+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e680a1080 msgr2=0x7f1e680a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:24.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.933+0000 7f1e85b93640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e680a1080 0x7f1e680a3540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1e80199220 tx=0x7f1e700075b0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.933+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 msgr2=0x7f1e8019f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:24.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.933+0000 7f1e85b93640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e8019f2c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e7802aa00 tx=0x7f1e780042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 shutdown_connections 2024-09-13T17:26:24.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f1e680a1080 0x7f1e680a3540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e80101920 0x7f1e8019f2c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e80100f60 0x7f1e80198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 --2- 172.21.15.44:0/1744406557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e800ffe00 0x7f1e80197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:24.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 >> 172.21.15.44:0/1744406557 conn(0x7f1e800fb5d0 msgr2=0x7f1e800726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:24.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 shutdown_connections 2024-09-13T17:26:24.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:24.934+0000 7f1e85b93640 1 -- 172.21.15.44:0/1744406557 wait complete. 2024-09-13T17:26:24.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:24 smithi070 ceph-mon[34502]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:25.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:24 smithi100 ceph-mon[35055]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-13T17:26:25.662 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-13T17:26:25.672 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:26:25.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-13T17:26:25.886 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:25 smithi070 ceph-mon[34502]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:26.007 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:25 smithi044 ceph-mon[31175]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:26.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:25 smithi100 ceph-mon[35055]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:26.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:26 smithi070 ceph-mon[34502]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:26:27.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:26 smithi100 ceph-mon[35055]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:26:27.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:26 smithi044 ceph-mon[31175]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 -- 172.21.15.44:0/3913038865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 msgr2=0x7f75141013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 --2- 172.21.15.44:0/3913038865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75141013e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 -- 172.21.15.44:0/3913038865 shutdown_connections 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 --2- 172.21.15.44:0/3913038865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7514101920 0x7f7514108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 --2- 172.21.15.44:0/3913038865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75141013e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 --2- 172.21.15.44:0/3913038865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75140ffe00 0x7f7514100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 -- 172.21.15.44:0/3913038865 >> 172.21.15.44:0/3913038865 conn(0x7f75140fb5b0 msgr2=0x7f75140fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 -- 172.21.15.44:0/3913038865 shutdown_connections 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.725+0000 7f7519d44640 1 -- 172.21.15.44:0/3913038865 wait complete. 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 Processor -- start 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 -- start start 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75140ffe00 0x7f7514074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75140728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7514101920 0x7f7514072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514073570 con 0x7f7514100f60 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75140736e0 con 0x7f7514101920 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7519d44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751419fd80 con 0x7f75140ffe00 2024-09-13T17:26:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.726+0000 7f7512ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75140728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75140728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55798/0 (socket says 172.21.15.44:55798) 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 -- 172.21.15.44:0/703495976 learned_addr learned my addr 172.21.15.44:0/703495976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7513fff640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7514101920 0x7f7514072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f75137fe640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75140ffe00 0x7f7514074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 -- 172.21.15.44:0/703495976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75140ffe00 msgr2=0x7f7514074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75140ffe00 0x7f7514074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 -- 172.21.15.44:0/703495976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7514101920 msgr2=0x7f7514072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7514101920 0x7f7514072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 -- 172.21.15.44:0/703495976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508009560 con 0x7f7514100f60 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7513fff640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7514101920 0x7f7514072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f75137fe640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75140ffe00 0x7f7514074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7512ffd640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75140728a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7508006010 tx=0x7f75080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508031d70 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75080044e0 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508003b10 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751419ffd0 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.727+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75141a0380 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.728+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514107330 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.729+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7508003c70 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.730+0000 7f7510ff9640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f74fc0a1100 0x7f74fc0a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.730+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f75080edd70 con 0x7f7514100f60 2024-09-13T17:26:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.730+0000 7f75137fe640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f74fc0a1100 0x7f74fc0a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:27.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.732+0000 7f75137fe640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f74fc0a1100 0x7f74fc0a35c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7504009b40 tx=0x7f7504005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:27.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.735+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f75080b7160 con 0x7f7514100f60 2024-09-13T17:26:27.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.944+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f751402c520 con 0x7f7514100f60 2024-09-13T17:26:27.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.944+0000 7f7510ff9640 1 -- 172.21.15.44:0/703495976 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2893 (secure 0 0 0) 0x7f7508040b60 con 0x7f7514100f60 2024-09-13T17:26:27.946 INFO:teuthology.orchestra.run.smithi044.stdout:e24 2024-09-13T17:26:27.946 INFO:teuthology.orchestra.run.smithi044.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:legacy client fscid: 1 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:Filesystem 'cephfs' (1) 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:fs_name cephfs 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 24 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-09-13T17:24:21.722369+0000 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-09-13T17:24:47.147746+0000 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:tableserver 0 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:root 0 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:session_timeout 300 2024-09-13T17:26:27.947 INFO:teuthology.orchestra.run.smithi044.stdout:session_autoclose 300 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:max_file_size 1099511627776 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:max_xattr_size 65536 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:required_client_features {} 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure 0 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure_osd_epoch 0 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:max_mds 3 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:in 0,1,2 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:up {0=24517,1=24496,2=24449} 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:failed 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:damaged 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:stopped 2024-09-13T17:26:27.948 INFO:teuthology.orchestra.run.smithi044.stdout:data_pools [3,4] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:metadata_pool 2 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:inline_data disabled 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:balancer 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:bal_rank_mask -1 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:standby_count_wanted 0 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.i{0:24517} state up:active seq 5 addr [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f{0:24484} state up:standby-replay seq 1 addr [v2:172.21.15.100:6834/4065662345,v1:172.21.15.100:6835/4065662345] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c{1:24496} state up:active seq 12 addr [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b{1:24482} state up:standby-replay seq 1 addr [v2:172.21.15.70:6838/2731404816,v1:172.21.15.70:6839/2731404816] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.h{2:24449} state up:active seq 11 addr [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.l{2:24439} state up:standby-replay seq 1 addr [v2:172.21.15.100:6832/459553980,v1:172.21.15.100:6833/459553980] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:27.949 INFO:teuthology.orchestra.run.smithi044.stdout:Standby daemons: 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.44:6834/673823135,v1:172.21.15.44:6835/673823135] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.44:6836/3952931988,v1:172.21.15.44:6837/3952931988] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.44:6838/4189498967,v1:172.21.15.44:6839/4189498967] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a{-1:14643} state up:standby seq 1 addr [v2:172.21.15.44:6840/1193858292,v1:172.21.15.44:6841/1193858292] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.k{-1:24389} state up:standby seq 1 addr [v2:172.21.15.70:6832/982414748,v1:172.21.15.70:6833/982414748] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.950 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e{-1:24416} state up:standby seq 1 addr [v2:172.21.15.70:6834/4022128703,v1:172.21.15.70:6835/4022128703] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f74fc0a1100 msgr2=0x7f74fc0a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f74fc0a1100 0x7f74fc0a35c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7504009b40 tx=0x7f7504005e50 comp rx=0 tx=0).stop 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 msgr2=0x7f75140728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75140728a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7508006010 tx=0x7f75080042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 shutdown_connections 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f74fc0a1100 0x7f74fc0a35c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7514101920 0x7f7514072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7514100f60 0x7f75140728a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 --2- 172.21.15.44:0/703495976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75140ffe00 0x7f7514074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:27.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 >> 172.21.15.44:0/703495976 conn(0x7f75140fb5b0 msgr2=0x7f7514104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:27.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 shutdown_connections 2024-09-13T17:26:27.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:27.946+0000 7f7519d44640 1 -- 172.21.15.44:0/703495976 wait complete. 2024-09-13T17:26:28.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-13T17:26:28.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:28 smithi070 ceph-mon[34502]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-13T17:26:28.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:28 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/703495976' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T17:26:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:29.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:28 smithi100 ceph-mon[35055]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-13T17:26:29.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:28 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/703495976' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T17:26:29.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:28 smithi044 ceph-mon[31175]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-13T17:26:29.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:28 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/703495976' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T17:26:30.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 -- 172.21.15.44:0/1783114111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 msgr2=0x7f80d40ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:30.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/1783114111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d40ff0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009900 tx=0x7f80d002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 -- 172.21.15.44:0/1783114111 shutdown_connections 2024-09-13T17:26:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/1783114111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 0x7f80d4107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/1783114111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80d40ffee0 0x7f80d4100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/1783114111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d40ff0e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 -- 172.21.15.44:0/1783114111 >> 172.21.15.44:0/1783114111 conn(0x7f80d40fa490 msgr2=0x7f80d40fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 -- 172.21.15.44:0/1783114111 shutdown_connections 2024-09-13T17:26:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.719+0000 7f80dc4d0640 1 -- 172.21.15.44:0/1783114111 wait complete. 2024-09-13T17:26:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 Processor -- start 2024-09-13T17:26:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 -- start start 2024-09-13T17:26:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d4102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:30.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80d40ffee0 0x7f80d41012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:30.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 0x7f80d4101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:30.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4101f90 con 0x7f80d40fece0 2024-09-13T17:26:30.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4102100 con 0x7f80d40ffee0 2024-09-13T17:26:30.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80dc4d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419fd40 con 0x7f80d41008a0 2024-09-13T17:26:30.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d4102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:30.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d4102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34720/0 (socket says 172.21.15.44:34720) 2024-09-13T17:26:30.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 -- 172.21.15.44:0/3810045597 learned_addr learned my addr 172.21.15.44:0/3810045597 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:30.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80d9a6b640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80d40ffee0 0x7f80d41012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:30.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80daa6d640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 0x7f80d4101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:30.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 -- 172.21.15.44:0/3810045597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 msgr2=0x7f80d4101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:30.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 0x7f80d4101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 -- 172.21.15.44:0/3810045597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80d40ffee0 msgr2=0x7f80d41012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:30.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80d40ffee0 0x7f80d41012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80da26c640 1 -- 172.21.15.44:0/3810045597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0009560 con 0x7f80d40fece0 2024-09-13T17:26:30.767 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:30 smithi044 ceph-mon[31175]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:30.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.720+0000 7f80daa6d640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 0x7f80d4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:30.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.721+0000 7f80da26c640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d4102c70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f80d0006010 tx=0x7f80d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:30.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.721+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0031d70 con 0x7f80d40fece0 2024-09-13T17:26:30.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.721+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d00044e0 con 0x7f80d40fece0 2024-09-13T17:26:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.721+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0003a30 con 0x7f80d40fece0 2024-09-13T17:26:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.721+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d419ff90 con 0x7f80d40fece0 2024-09-13T17:26:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.721+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d41a0350 con 0x7f80d40fece0 2024-09-13T17:26:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.722+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d410a460 con 0x7f80d40fece0 2024-09-13T17:26:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.723+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f80d0003bd0 con 0x7f80d40fece0 2024-09-13T17:26:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.724+0000 7f80cb7fe640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80c00a1100 0x7f80c00a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.724+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f80d00edc70 con 0x7f80d40fece0 2024-09-13T17:26:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.724+0000 7f80d9a6b640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80c00a1100 0x7f80c00a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.724+0000 7f80d9a6b640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80c00a1100 0x7f80c00a35c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f80d41029f0 tx=0x7f80c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.728+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f80d00b7060 con 0x7f80d40fece0 2024-09-13T17:26:30.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.904+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f80d41a05a0 con 0x7f80d40fece0 2024-09-13T17:26:30.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.905+0000 7f80cb7fe640 1 -- 172.21.15.44:0/3810045597 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5611 (secure 0 0 0) 0x7f80d00ba680 con 0x7f80d40fece0 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 92 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:fsid ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-09-13T17:16:55.305803+0000 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-09-13T17:25:43.045876+0000 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:crush_version 27 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:full_ratio 0.95 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:backfillfull_ratio 0.9 2024-09-13T17:26:30.906 INFO:teuthology.orchestra.run.smithi044.stdout:nearfull_ratio 0.85 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:require_min_compat_client luminous 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:min_compat_client luminous 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:require_osd_release reef 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:stretch_mode_enabled false 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:max_osd 12 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6802/637354587,v1:172.21.15.44:6803/637354587] [v2:172.21.15.44:6804/637354587,v1:172.21.15.44:6805/637354587] exists,up ff8d9259-48ec-4ece-86c1-326e4bc5ab62 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6800/3261074014,v1:172.21.15.70:6801/3261074014] [v2:172.21.15.70:6802/3261074014,v1:172.21.15.70:6803/3261074014] exists,up 8eb850b3-92e7-40d5-a891-9c21f9f591dd 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/4273545793,v1:172.21.15.100:6801/4273545793] [v2:172.21.15.100:6802/4273545793,v1:172.21.15.100:6803/4273545793] exists,up 1cfc25ea-477f-4e25-b425-2481547b68c0 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 up in weight 1 up_from 24 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6810/581674337,v1:172.21.15.44:6811/581674337] [v2:172.21.15.44:6812/581674337,v1:172.21.15.44:6813/581674337] exists,up 65f620c4-46d1-40d5-9346-56d4f1cb4841 2024-09-13T17:26:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6808/2214036944,v1:172.21.15.70:6809/2214036944] [v2:172.21.15.70:6810/2214036944,v1:172.21.15.70:6811/2214036944] exists,up d9180bdb-24d0-42c8-9d13-51f4ae9812ba 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 up in weight 1 up_from 37 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/2555541619,v1:172.21.15.100:6809/2555541619] [v2:172.21.15.100:6810/2555541619,v1:172.21.15.100:6811/2555541619] exists,up 55e04db8-8d0a-42bc-a507-aa94db69fa62 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 up in weight 1 up_from 45 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6818/3425095841,v1:172.21.15.44:6819/3425095841] [v2:172.21.15.44:6820/3425095841,v1:172.21.15.44:6821/3425095841] exists,up 18a8c0b4-0801-40db-88a0-ee66c59f28bf 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 up in weight 1 up_from 50 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6816/4007129132,v1:172.21.15.70:6817/4007129132] [v2:172.21.15.70:6818/4007129132,v1:172.21.15.70:6819/4007129132] exists,up f7c62e54-ff2e-4578-8eb7-93a848d283c7 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 up in weight 1 up_from 58 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/2612262054,v1:172.21.15.100:6817/2612262054] [v2:172.21.15.100:6818/2612262054,v1:172.21.15.100:6819/2612262054] exists,up c07c1084-7ab4-4531-bb14-893e93b0e1b8 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 up in weight 1 up_from 66 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6826/280929733,v1:172.21.15.44:6827/280929733] [v2:172.21.15.44:6828/280929733,v1:172.21.15.44:6829/280929733] exists,up b38cbad2-3c65-4acc-bc76-68184079cc48 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6824/752114837,v1:172.21.15.70:6825/752114837] [v2:172.21.15.70:6826/752114837,v1:172.21.15.70:6827/752114837] exists,up 27a2977e-4fde-4a5f-9be6-9811159c13d4 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 up in weight 1 up_from 79 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/1155650564,v1:172.21.15.100:6825/1155650564] [v2:172.21.15.100:6826/1155650564,v1:172.21.15.100:6827/1155650564] exists,up 1fdeaf9a-fb39-48a9-a42c-76de2ae09f34 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.5 [8,11] 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.e [2,1] 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.1d [2,4,8,11] 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.23 [4,8] 2024-09-13T17:26:30.908 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.25 [5,9] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.36 [4,9] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.3c [4,9] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 4.c [9,10] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 4.e [5,8] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 4.16 [9,8] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 4.17 [9,8] 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2209604760 expires 2024-09-14T17:17:41.472796+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/2265862199 expires 2024-09-14T17:17:41.472796+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/2265862199 expires 2024-09-14T17:17:41.472796+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/815607991 expires 2024-09-14T17:17:17.599897+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/1683772085 expires 2024-09-14T17:17:17.599897+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/1586278527 expires 2024-09-14T17:17:41.472796+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/3711196986 expires 2024-09-14T17:17:17.599897+0000 2024-09-13T17:26:30.909 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/771352345 expires 2024-09-14T17:17:41.472796+0000 2024-09-13T17:26:30.910 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/3711196986 expires 2024-09-14T17:17:17.599897+0000 2024-09-13T17:26:30.910 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2038542363 expires 2024-09-14T17:17:17.599897+0000 2024-09-13T17:26:30.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80c00a1100 msgr2=0x7f80c00a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80c00a1100 0x7f80c00a35c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f80d41029f0 tx=0x7f80c400b390 comp rx=0 tx=0).stop 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 msgr2=0x7f80d4102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d4102c70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f80d0006010 tx=0x7f80d00042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 shutdown_connections 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f80c00a1100 0x7f80c00a35c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d41008a0 0x7f80d4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80d40ffee0 0x7f80d41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 --2- 172.21.15.44:0/3810045597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40fece0 0x7f80d4102c70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 >> 172.21.15.44:0/3810045597 conn(0x7f80d40fa490 msgr2=0x7f80d4067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 shutdown_connections 2024-09-13T17:26:30.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:30.908+0000 7f80dc4d0640 1 -- 172.21.15.44:0/3810045597 wait complete. 2024-09-13T17:26:30.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:30 smithi070 ceph-mon[34502]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:31.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:30 smithi100 ceph-mon[35055]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:31.699 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-13T17:26:31.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:31.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 -- 172.21.15.44:0/3447879695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 msgr2=0x7fa24c107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:31.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 --2- 172.21.15.44:0/3447879695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c107ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa244009ca0 tx=0x7fa24402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:31.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 -- 172.21.15.44:0/3447879695 shutdown_connections 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 --2- 172.21.15.44:0/3447879695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa24c103150 0x7fa24c105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 --2- 172.21.15.44:0/3447879695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c107ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 --2- 172.21.15.44:0/3447879695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c100820 0x7fa24c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 -- 172.21.15.44:0/3447879695 >> 172.21.15.44:0/3447879695 conn(0x7fa24c0fa440 msgr2=0x7fa24c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 -- 172.21.15.44:0/3447879695 shutdown_connections 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.824+0000 7fa2536d7640 1 -- 172.21.15.44:0/3447879695 wait complete. 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 Processor -- start 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 -- start start 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa24c100820 0x7fa24c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c103150 0x7fa24c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c19d710 con 0x7fa24c105a80 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c19d880 con 0x7fa24c103150 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa2536d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c19da40 con 0x7fa24c100820 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c19d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34744/0 (socket says 172.21.15.44:34744) 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 -- 172.21.15.44:0/684998437 learned_addr learned my addr 172.21.15.44:0/684998437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa250c72640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c103150 0x7fa24c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251473640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa24c100820 0x7fa24c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 -- 172.21.15.44:0/684998437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa24c100820 msgr2=0x7fa24c195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa24c100820 0x7fa24c195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 -- 172.21.15.44:0/684998437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c103150 msgr2=0x7fa24c196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c103150 0x7fa24c196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 -- 172.21.15.44:0/684998437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009900 con 0x7fa24c105a80 2024-09-13T17:26:31.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa250c72640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c103150 0x7fa24c196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.825+0000 7fa251c74640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c19d1a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa24402aeb0 tx=0x7fa244003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.826+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2440307c0 con 0x7fa24c105a80 2024-09-13T17:26:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.826+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa244040070 con 0x7fa24c105a80 2024-09-13T17:26:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.826+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c19dcc0 con 0x7fa24c105a80 2024-09-13T17:26:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.826+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2440044b0 con 0x7fa24c105a80 2024-09-13T17:26:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.826+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c19e1d0 con 0x7fa24c105a80 2024-09-13T17:26:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.827+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24c0fbf30 con 0x7fa24c105a80 2024-09-13T17:26:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.828+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa244042030 con 0x7fa24c105a80 2024-09-13T17:26:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.829+0000 7fa2427fc640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa2380a1110 0x7fa2380a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.829+0000 7fa251473640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa2380a1110 0x7fa2380a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.829+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fa2440ee460 con 0x7fa24c105a80 2024-09-13T17:26:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.829+0000 7fa251473640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa2380a1110 0x7fa2380a35d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa248005fd0 tx=0x7fa248005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:31.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:31.833+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa2440b7940 con 0x7fa24c105a80 2024-09-13T17:26:31.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:31 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3810045597' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T17:26:32.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.005+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa24c02c3e0 con 0x7fa24c105a80 2024-09-13T17:26:32.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.006+0000 7fa2427fc640 1 -- 172.21.15.44:0/684998437 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24492 (secure 0 0 0) 0x7fa2440b9870 con 0x7fa24c105a80 2024-09-13T17:26:32.008 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:32.008 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":92,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:25:43.045876+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:32.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa2380a1110 msgr2=0x7fa2380a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa2380a1110 0x7fa2380a35d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa248005fd0 tx=0x7fa248005c70 comp rx=0 tx=0).stop 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 msgr2=0x7fa24c19d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c19d1a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa24402aeb0 tx=0x7fa244003c20 comp rx=0 tx=0).stop 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 shutdown_connections 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fa2380a1110 0x7fa2380a35d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa24c105a80 0x7fa24c19d1a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c103150 0x7fa24c196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 --2- 172.21.15.44:0/684998437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa24c100820 0x7fa24c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 >> 172.21.15.44:0/684998437 conn(0x7fa24c0fa440 msgr2=0x7fa24c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 shutdown_connections 2024-09-13T17:26:32.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.008+0000 7fa2536d7640 1 -- 172.21.15.44:0/684998437 wait complete. 2024-09-13T17:26:32.019 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:31 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3810045597' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T17:26:32.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:32.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:31 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3810045597' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T17:26:32.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 -- 172.21.15.44:0/4275777940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368101950 msgr2=0x7f43681081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 --2- 172.21.15.44:0/4275777940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368101950 0x7f43681081e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4360009ca0 tx=0x7f436002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 -- 172.21.15.44:0/4275777940 shutdown_connections 2024-09-13T17:26:32.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 --2- 172.21.15.44:0/4275777940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368100f90 0x7f4368101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 --2- 172.21.15.44:0/4275777940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368101950 0x7f43681081e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 --2- 172.21.15.44:0/4275777940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 0x7f43681001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 -- 172.21.15.44:0/4275777940 >> 172.21.15.44:0/4275777940 conn(0x7f43680fb520 msgr2=0x7f43680fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 -- 172.21.15.44:0/4275777940 shutdown_connections 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 -- 172.21.15.44:0/4275777940 wait complete. 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.126+0000 7f436dd25640 1 Processor -- start 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 -- start start 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 0x7f4368074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 0x7f43680728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368101950 0x7f4368072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368073540 con 0x7f4368100f90 2024-09-13T17:26:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680736b0 con 0x7f43680ffda0 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f436dd25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819fca0 con 0x7f4368101950 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 0x7f43680728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 0x7f43680728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34748/0 (socket says 172.21.15.44:34748) 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 -- 172.21.15.44:0/3477974209 learned_addr learned my addr 172.21.15.44:0/3477974209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4367fff640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368101950 0x7f4368072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f43677fe640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 0x7f4368074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 -- 172.21.15.44:0/3477974209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368101950 msgr2=0x7f4368072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368101950 0x7f4368072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 -- 172.21.15.44:0/3477974209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 msgr2=0x7f4368074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 0x7f4368074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 -- 172.21.15.44:0/3477974209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009900 con 0x7f4368100f90 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f43677fe640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 0x7f4368074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4367fff640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368101950 0x7f4368072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.127+0000 7f4366ffd640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 0x7f43680728a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f435c00d7a0 tx=0x7f435c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.128+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c009c30 con 0x7f4368100f90 2024-09-13T17:26:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.128+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f435c00deb0 con 0x7f4368100f90 2024-09-13T17:26:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.128+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436819ff50 con 0x7f4368100f90 2024-09-13T17:26:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.128+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c004d60 con 0x7f4368100f90 2024-09-13T17:26:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.128+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43681a03d0 con 0x7f4368100f90 2024-09-13T17:26:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.129+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368107280 con 0x7f4368100f90 2024-09-13T17:26:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.130+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f435c004ec0 con 0x7f4368100f90 2024-09-13T17:26:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.130+0000 7f4364ff9640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f43500a1110 0x7f43500a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.131+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f435c0cca30 con 0x7f4368100f90 2024-09-13T17:26:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.131+0000 7f43677fe640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f43500a1110 0x7f43500a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.131+0000 7f43677fe640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f43500a1110 0x7f43500a35d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4368100df0 tx=0x7f4358005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.133+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f435c095f00 con 0x7f4368100f90 2024-09-13T17:26:32.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.306+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f436802c410 con 0x7f4368100f90 2024-09-13T17:26:32.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.306+0000 7f4364ff9640 1 -- 172.21.15.44:0/3477974209 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f435c017070 con 0x7f4368100f90 2024-09-13T17:26:32.307 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f43500a1110 msgr2=0x7f43500a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f43500a1110 0x7f43500a35d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4368100df0 tx=0x7f4358005e70 comp rx=0 tx=0).stop 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 msgr2=0x7f43680728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 0x7f43680728a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f435c00d7a0 tx=0x7f435c00dc60 comp rx=0 tx=0).stop 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 shutdown_connections 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f43500a1110 0x7f43500a35d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4368101950 0x7f4368072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4368100f90 0x7f43680728a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 --2- 172.21.15.44:0/3477974209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43680ffda0 0x7f4368074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 >> 172.21.15.44:0/3477974209 conn(0x7f43680fb520 msgr2=0x7f4368104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 shutdown_connections 2024-09-13T17:26:32.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.308+0000 7f436dd25640 1 -- 172.21.15.44:0/3477974209 wait complete. 2024-09-13T17:26:32.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.424+0000 7f0c62308640 1 -- 172.21.15.44:0/3596091089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c1007e0 msgr2=0x7f0c5c102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.424+0000 7f0c62308640 1 --2- 172.21.15.44:0/3596091089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c1007e0 0x7f0c5c102bd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009900 tx=0x7f0c4c02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 -- 172.21.15.44:0/3596091089 shutdown_connections 2024-09-13T17:26:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 --2- 172.21.15.44:0/3596091089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c5c103110 0x7f0c5c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 --2- 172.21.15.44:0/3596091089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c5c105a40 0x7f0c5c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 --2- 172.21.15.44:0/3596091089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c1007e0 0x7f0c5c102bd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 -- 172.21.15.44:0/3596091089 >> 172.21.15.44:0/3596091089 conn(0x7f0c5c0fa3e0 msgr2=0x7f0c5c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 -- 172.21.15.44:0/3596091089 shutdown_connections 2024-09-13T17:26:32.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 -- 172.21.15.44:0/3596091089 wait complete. 2024-09-13T17:26:32.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 Processor -- start 2024-09-13T17:26:32.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 -- start start 2024-09-13T17:26:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c5c1007e0 0x7f0c5c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c5c103110 0x7f0c5c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.425+0000 7f0c62308640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 0x7f0c5c19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c62308640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19d8a0 con 0x7f0c5c105a40 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c62308640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19da10 con 0x7f0c5c103110 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c62308640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19dbd0 con 0x7f0c5c1007e0 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 0x7f0c5c19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 0x7f0c5c19d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34766/0 (socket says 172.21.15.44:34766) 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 -- 172.21.15.44:0/1411531677 learned_addr learned my addr 172.21.15.44:0/1411531677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c5b7fe640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c5c103110 0x7f0c5c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c5bfff640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c5c1007e0 0x7f0c5c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 -- 172.21.15.44:0/1411531677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c5c1007e0 msgr2=0x7f0c5c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c5c1007e0 0x7f0c5c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 -- 172.21.15.44:0/1411531677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c5c103110 msgr2=0x7f0c5c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c5c103110 0x7f0c5c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 -- 172.21.15.44:0/1411531677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c009560 con 0x7f0c5c105a40 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c608a5640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 0x7f0c5c19d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400dac0 tx=0x7f0c5400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400be30 con 0x7f0c5c105a40 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c54003e00 con 0x7f0c5c105a40 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c19deb0 con 0x7f0c5c105a40 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540106c0 con 0x7f0c5c105a40 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.426+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c19e280 con 0x7f0c5c105a40 2024-09-13T17:26:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.427+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c1034f0 con 0x7f0c5c105a40 2024-09-13T17:26:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.429+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0c54003970 con 0x7f0c5c105a40 2024-09-13T17:26:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.430+0000 7f0c597fa640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c440a1090 0x7f0c440a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.430+0000 7f0c5bfff640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c440a1090 0x7f0c440a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.430+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f0c540cc970 con 0x7f0c5c105a40 2024-09-13T17:26:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.430+0000 7f0c5bfff640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c440a1090 0x7f0c440a3550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006010 tx=0x7f0c4c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.433+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0c54095d80 con 0x7f0c5c105a40 2024-09-13T17:26:32.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.605+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c5c10b580 con 0x7f0c5c105a40 2024-09-13T17:26:32.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.606+0000 7f0c597fa640 1 -- 172.21.15.44:0/1411531677 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c540200a0 con 0x7f0c5c105a40 2024-09-13T17:26:32.607 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c440a1090 msgr2=0x7f0c440a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c440a1090 0x7f0c440a3550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006010 tx=0x7f0c4c000bc0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 msgr2=0x7f0c5c19d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 0x7f0c5c19d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400dac0 tx=0x7f0c5400df80 comp rx=0 tx=0).stop 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 shutdown_connections 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0c440a1090 0x7f0c440a3550 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c5c105a40 0x7f0c5c19d330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c5c103110 0x7f0c5c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 --2- 172.21.15.44:0/1411531677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c5c1007e0 0x7f0c5c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 >> 172.21.15.44:0/1411531677 conn(0x7f0c5c0fa3e0 msgr2=0x7f0c5c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 shutdown_connections 2024-09-13T17:26:32.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.608+0000 7f0c62308640 1 -- 172.21.15.44:0/1411531677 wait complete. 2024-09-13T17:26:32.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 -- 172.21.15.44:0/547164643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 msgr2=0x7f3460107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 --2- 172.21.15.44:0/547164643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f3460107ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3458009ca0 tx=0x7f345802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 -- 172.21.15.44:0/547164643 shutdown_connections 2024-09-13T17:26:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 --2- 172.21.15.44:0/547164643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f3460107ee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 --2- 172.21.15.44:0/547164643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3460103140 0x7f3460105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 --2- 172.21.15.44:0/547164643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 0x7f3460102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 -- 172.21.15.44:0/547164643 >> 172.21.15.44:0/547164643 conn(0x7f34600fa3f0 msgr2=0x7f34600fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 -- 172.21.15.44:0/547164643 shutdown_connections 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.724+0000 7f3467e6b640 1 -- 172.21.15.44:0/547164643 wait complete. 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 Processor -- start 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 -- start start 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 0x7f3460195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3460103140 0x7f34601962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f346019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346019d860 con 0x7f3460105a70 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346019d9d0 con 0x7f3460103140 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3467e6b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346019db90 con 0x7f3460100810 2024-09-13T17:26:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f346019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f346019d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34788/0 (socket says 172.21.15.44:34788) 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 -- 172.21.15.44:0/3365745612 learned_addr learned my addr 172.21.15.44:0/3365745612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3465406640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3460103140 0x7f34601962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3465c07640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 0x7f3460195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 -- 172.21.15.44:0/3365745612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 msgr2=0x7f3460195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 0x7f3460195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 -- 172.21.15.44:0/3365745612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3460103140 msgr2=0x7f34601962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3460103140 0x7f34601962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3466408640 1 -- 172.21.15.44:0/3365745612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009900 con 0x7f3460105a70 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.725+0000 7f3465c07640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 0x7f3460195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.726+0000 7f3466408640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f346019d320 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f345802aeb0 tx=0x7f3458003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.726+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580307c0 con 0x7f3460105a70 2024-09-13T17:26:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.726+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3458040070 con 0x7f3460105a70 2024-09-13T17:26:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.726+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346019de10 con 0x7f3460105a70 2024-09-13T17:26:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.726+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580044b0 con 0x7f3460105a70 2024-09-13T17:26:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.726+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346019e290 con 0x7f3460105a70 2024-09-13T17:26:32.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.727+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34600fbee0 con 0x7f3460105a70 2024-09-13T17:26:32.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.728+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3458042030 con 0x7f3460105a70 2024-09-13T17:26:32.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.729+0000 7f3452ffd640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f344c0a1090 0x7f344c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:32.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.729+0000 7f3465c07640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f344c0a1090 0x7f344c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:32.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.729+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f34580ee450 con 0x7f3460105a70 2024-09-13T17:26:32.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.729+0000 7f3465c07640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f344c0a1090 0x7f344c0a3550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3454005fd0 tx=0x7f3454005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:32.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.732+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f34580b7840 con 0x7f3460105a70 2024-09-13T17:26:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.901+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f346002c3e0 con 0x7f3460105a70 2024-09-13T17:26:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.902+0000 7f3452ffd640 1 -- 172.21.15.44:0/3365745612 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34580bae60 con 0x7f3460105a70 2024-09-13T17:26:32.903 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f344c0a1090 msgr2=0x7f344c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f344c0a1090 0x7f344c0a3550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3454005fd0 tx=0x7f3454005ea0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 msgr2=0x7f346019d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f346019d320 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f345802aeb0 tx=0x7f3458003c20 comp rx=0 tx=0).stop 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 shutdown_connections 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f344c0a1090 0x7f344c0a3550 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3460105a70 0x7f346019d320 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3460103140 0x7f34601962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 --2- 172.21.15.44:0/3365745612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3460100810 0x7f3460195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 >> 172.21.15.44:0/3365745612 conn(0x7f34600fa3f0 msgr2=0x7f3460102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 shutdown_connections 2024-09-13T17:26:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:32.904+0000 7f3467e6b640 1 -- 172.21.15.44:0/3365745612 wait complete. 2024-09-13T17:26:32.914 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:32 smithi044 ceph-mon[31175]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:32.914 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:32 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/684998437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:32.914 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:32 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3477974209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:32.914 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:32 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1411531677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:32.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:32 smithi070 ceph-mon[34502]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:32 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/684998437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:32 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3477974209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:32.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:32 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1411531677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 -- 172.21.15.44:0/1237234892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb8100840 msgr2=0x7fdeb81070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1237234892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb8100840 0x7fdeb81070d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009ca0 tx=0x7fdeb002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 -- 172.21.15.44:0/1237234892 shutdown_connections 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1237234892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80ffe80 0x7fdeb8100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1237234892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb8100840 0x7fdeb81070d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1237234892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb80fec90 0x7fdeb80ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 -- 172.21.15.44:0/1237234892 >> 172.21.15.44:0/1237234892 conn(0x7fdeb80fa3f0 msgr2=0x7fdeb80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 -- 172.21.15.44:0/1237234892 shutdown_connections 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 -- 172.21.15.44:0/1237234892 wait complete. 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.020+0000 7fdebea0d640 1 Processor -- start 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 -- start start 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 0x7fdeb80690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80ffe80 0x7fdeb819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb8100840 0x7fdeb819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819d780 con 0x7fdeb80fec90 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819d8f0 con 0x7fdeb8100840 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebea0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819dab0 con 0x7fdeb80ffe80 2024-09-13T17:26:33.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 0x7fdeb80690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 0x7fdeb80690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34802/0 (socket says 172.21.15.44:34802) 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 -- 172.21.15.44:0/1105544896 learned_addr learned my addr 172.21.15.44:0/1105544896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdebcfaa640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb8100840 0x7fdeb819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb77fe640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80ffe80 0x7fdeb819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 -- 172.21.15.44:0/1105544896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80ffe80 msgr2=0x7fdeb819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80ffe80 0x7fdeb819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 -- 172.21.15.44:0/1105544896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb8100840 msgr2=0x7fdeb819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb8100840 0x7fdeb819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 -- 172.21.15.44:0/1105544896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb0009900 con 0x7fdeb80fec90 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.021+0000 7fdeb7fff640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 0x7fdeb80690e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9c0 tx=0x7fdea800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.022+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8009700 con 0x7fdeb80fec90 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.022+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdea80044b0 con 0x7fdeb80fec90 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.022+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb819dd30 con 0x7fdeb80fec90 2024-09-13T17:26:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.022+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8003e40 con 0x7fdeb80fec90 2024-09-13T17:26:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.022+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb819e190 con 0x7fdeb80fec90 2024-09-13T17:26:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.023+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb8106110 con 0x7fdeb80fec90 2024-09-13T17:26:33.027 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:32 smithi100 ceph-mon[35055]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:33.027 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:32 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/684998437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:33.027 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:32 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3477974209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.027 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:32 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1411531677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.024+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdea8003fe0 con 0x7fdeb80fec90 2024-09-13T17:26:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.024+0000 7fdeb57fa640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdea00a1090 0x7fdea00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.025+0000 7fdeb77fe640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdea00a1090 0x7fdea00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.025+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fdea8014070 con 0x7fdeb80fec90 2024-09-13T17:26:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.025+0000 7fdeb77fe640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdea00a1090 0x7fdea00a3550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005fd0 tx=0x7fdeac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.028+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdea8096600 con 0x7fdeb80fec90 2024-09-13T17:26:33.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.201+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdeb810a400 con 0x7fdeb80fec90 2024-09-13T17:26:33.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.201+0000 7fdeb57fa640 1 -- 172.21.15.44:0/1105544896 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdea8099c20 con 0x7fdeb80fec90 2024-09-13T17:26:33.203 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdea00a1090 msgr2=0x7fdea00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdea00a1090 0x7fdea00a3550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005fd0 tx=0x7fdeac005850 comp rx=0 tx=0).stop 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 msgr2=0x7fdeb80690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 0x7fdeb80690e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9c0 tx=0x7fdea800ee80 comp rx=0 tx=0).stop 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 shutdown_connections 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdea00a1090 0x7fdea00a3550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80ffe80 0x7fdeb819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb8100840 0x7fdeb819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 --2- 172.21.15.44:0/1105544896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb80fec90 0x7fdeb80690e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 >> 172.21.15.44:0/1105544896 conn(0x7fdeb80fa3f0 msgr2=0x7fdeb81032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.203+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 shutdown_connections 2024-09-13T17:26:33.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.204+0000 7fdebea0d640 1 -- 172.21.15.44:0/1105544896 wait complete. 2024-09-13T17:26:33.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:33.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 -- 172.21.15.44:0/608949655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 msgr2=0x7f42580fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 --2- 172.21.15.44:0/608949655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f42580fe840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4254009900 tx=0x7f425402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:33.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 -- 172.21.15.44:0/608949655 shutdown_connections 2024-09-13T17:26:33.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 --2- 172.21.15.44:0/608949655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ff630 0x7f42580ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 --2- 172.21.15.44:0/608949655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42580ffff0 0x7f4258106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 --2- 172.21.15.44:0/608949655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f42580fe840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.316+0000 7f42602c9640 1 -- 172.21.15.44:0/608949655 >> 172.21.15.44:0/608949655 conn(0x7f42580f9bc0 msgr2=0x7f42580fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 -- 172.21.15.44:0/608949655 shutdown_connections 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 -- 172.21.15.44:0/608949655 wait complete. 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 Processor -- start 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 -- start start 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f4258195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42580ff630 0x7f4258196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ffff0 0x7f425819d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425819d840 con 0x7f42580fe440 2024-09-13T17:26:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425819d9b0 con 0x7f42580ff630 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f42602c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425819db70 con 0x7f42580ffff0 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f425e065640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f4258195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f425e065640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f4258195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34824/0 (socket says 172.21.15.44:34824) 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.317+0000 7f425e065640 1 -- 172.21.15.44:0/1622678766 learned_addr learned my addr 172.21.15.44:0/1622678766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e866640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ffff0 0x7f425819d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e065640 1 -- 172.21.15.44:0/1622678766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ffff0 msgr2=0x7f425819d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425d864640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42580ff630 0x7f4258196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e065640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ffff0 0x7f425819d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e065640 1 -- 172.21.15.44:0/1622678766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42580ff630 msgr2=0x7f4258196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e065640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42580ff630 0x7f4258196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e065640 1 -- 172.21.15.44:0/1622678766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4254009560 con 0x7f42580fe440 2024-09-13T17:26:33.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e866640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ffff0 0x7f425819d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f425e065640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f4258195d50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f425402a930 tx=0x7f4254005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254004c50 con 0x7f42580fe440 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4254004db0 con 0x7f42580fe440 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425819ddf0 con 0x7f42580fe440 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425403ca30 con 0x7f42580fe440 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.318+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425819e1e0 con 0x7f42580fe440 2024-09-13T17:26:33.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.319+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4258109c60 con 0x7f42580fe440 2024-09-13T17:26:33.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.320+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f42540035f0 con 0x7f42580fe440 2024-09-13T17:26:33.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.321+0000 7f424f7fe640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42440a1090 0x7f42440a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.321+0000 7f425d864640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42440a1090 0x7f42440a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.321+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f42540edfe0 con 0x7f42580fe440 2024-09-13T17:26:33.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.322+0000 7f425d864640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42440a1090 0x7f42440a3550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4258197260 tx=0x7f4248005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.325+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f42540b7430 con 0x7f42580fe440 2024-09-13T17:26:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.534+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f425819e3d0 con 0x7f42580fe440 2024-09-13T17:26:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.535+0000 7f424f7fe640 1 -- 172.21.15.44:0/1622678766 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f42540baa50 con 0x7f42580fe440 2024-09-13T17:26:33.536 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:33.536 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:47.147746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:26:33.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42440a1090 msgr2=0x7f42440a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42440a1090 0x7f42440a3550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4258197260 tx=0x7f4248005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:33.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 msgr2=0x7f4258195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f4258195d50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f425402a930 tx=0x7f4254005610 comp rx=0 tx=0).stop 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 shutdown_connections 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f42440a1090 0x7f42440a3550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ffff0 0x7f425819d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42580ff630 0x7f4258196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 --2- 172.21.15.44:0/1622678766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42580fe440 0x7f4258195d50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 >> 172.21.15.44:0/1622678766 conn(0x7f42580f9bc0 msgr2=0x7f42580fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 shutdown_connections 2024-09-13T17:26:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.537+0000 7f42602c9640 1 -- 172.21.15.44:0/1622678766 wait complete. 2024-09-13T17:26:33.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 -- 172.21.15.44:0/2831994657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 msgr2=0x7f09a0100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2831994657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0100340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0994009900 tx=0x7f099402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 -- 172.21.15.44:0/2831994657 shutdown_connections 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2831994657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 0x7f09a0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2831994657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0100340 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2831994657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a00fecd0 0x7f09a00ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.651+0000 7f09a5b95640 1 -- 172.21.15.44:0/2831994657 >> 172.21.15.44:0/2831994657 conn(0x7f09a00fa430 msgr2=0x7f09a00fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 -- 172.21.15.44:0/2831994657 shutdown_connections 2024-09-13T17:26:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 -- 172.21.15.44:0/2831994657 wait complete. 2024-09-13T17:26:33.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 Processor -- start 2024-09-13T17:26:33.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 -- start start 2024-09-13T17:26:33.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a00fecd0 0x7f09a0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 0x7f09a019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a019d820 con 0x7f09a00ffec0 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.652+0000 7f09a5b95640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a019d990 con 0x7f09a00fecd0 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f09a5b95640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a019db50 con 0x7f09a0100880 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34852/0 (socket says 172.21.15.44:34852) 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 -- 172.21.15.44:0/2603297871 learned_addr learned my addr 172.21.15.44:0/2603297871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099f7fe640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a00fecd0 0x7f09a0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099ffff640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 0x7f09a019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 -- 172.21.15.44:0/2603297871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 msgr2=0x7f09a019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 0x7f09a019d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 -- 172.21.15.44:0/2603297871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a00fecd0 msgr2=0x7f09a0195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a00fecd0 0x7f09a0195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 -- 172.21.15.44:0/2603297871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009560 con 0x7f09a00ffec0 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099ffff640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 0x7f09a019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099effd640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0196280 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f09940057c0 tx=0x7f09940052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994031d70 con 0x7f09a00ffec0 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09940054a0 con 0x7f09a00ffec0 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a019ddd0 con 0x7f09a00ffec0 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994003a10 con 0x7f09a00ffec0 2024-09-13T17:26:33.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.653+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a019e210 con 0x7f09a00ffec0 2024-09-13T17:26:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.654+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a0106220 con 0x7f09a00ffec0 2024-09-13T17:26:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.656+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0994003b70 con 0x7f09a00ffec0 2024-09-13T17:26:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.656+0000 7f099cff9640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f09880a1110 0x7f09880a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.656+0000 7f099f7fe640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f09880a1110 0x7f09880a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.656+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f09940ee120 con 0x7f09a00ffec0 2024-09-13T17:26:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.657+0000 7f099f7fe640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f09880a1110 0x7f09880a35d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09a0197210 tx=0x7f099000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.659+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f09940b74e0 con 0x7f09a00ffec0 2024-09-13T17:26:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.838+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09a010b5c0 con 0x7f09a00ffec0 2024-09-13T17:26:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.839+0000 7f099cff9640 1 -- 172.21.15.44:0/2603297871 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24492 (secure 0 0 0) 0x7f099404a090 con 0x7f09a00ffec0 2024-09-13T17:26:33.841 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:33.841 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":92,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:25:43.045876+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f09880a1110 msgr2=0x7f09880a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f09880a1110 0x7f09880a35d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09a0197210 tx=0x7f099000b390 comp rx=0 tx=0).stop 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 msgr2=0x7f09a0196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0196280 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f09940057c0 tx=0x7f09940052b0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 shutdown_connections 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f09880a1110 0x7f09880a35d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0100880 0x7f09a019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a00ffec0 0x7f09a0196280 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 --2- 172.21.15.44:0/2603297871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a00fecd0 0x7f09a0195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.841+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 >> 172.21.15.44:0/2603297871 conn(0x7f09a00fa430 msgr2=0x7f09a01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.842+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 shutdown_connections 2024-09-13T17:26:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.842+0000 7f09a5b95640 1 -- 172.21.15.44:0/2603297871 wait complete. 2024-09-13T17:26:33.852 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:33 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3365745612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.852 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:33 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1105544896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.852 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:33 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1622678766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:33.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:33.854 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:33.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:33 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3365745612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:33 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1105544896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:33.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:33 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1622678766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.958+0000 7fc44b896640 1 -- 172.21.15.44:0/769968563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 msgr2=0x7fc4441070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.958+0000 7fc44b896640 1 --2- 172.21.15.44:0/769968563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc4441070a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009ca0 tx=0x7fc43c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- 172.21.15.44:0/769968563 shutdown_connections 2024-09-13T17:26:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 --2- 172.21.15.44:0/769968563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc4441070a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 --2- 172.21.15.44:0/769968563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4440ffe50 0x7fc4441002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 --2- 172.21.15.44:0/769968563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440fec60 0x7fc4440ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- 172.21.15.44:0/769968563 >> 172.21.15.44:0/769968563 conn(0x7fc4440fa3e0 msgr2=0x7fc4440fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- 172.21.15.44:0/769968563 shutdown_connections 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- 172.21.15.44:0/769968563 wait complete. 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 Processor -- start 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- start start 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4440fec60 0x7fc444195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440ffe50 0x7fc4441961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc44419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419d740 con 0x7fc444100810 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419d8b0 con 0x7fc4440fec60 2024-09-13T17:26:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc44b896640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419da70 con 0x7fc4440ffe50 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.959+0000 7fc449e33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc44419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc44419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34868/0 (socket says 172.21.15.44:34868) 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 -- 172.21.15.44:0/3025423588 learned_addr learned my addr 172.21.15.44:0/3025423588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449632640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4440fec60 0x7fc444195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 -- 172.21.15.44:0/3025423588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440ffe50 msgr2=0x7fc4441961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc448e31640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440ffe50 0x7fc4441961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440ffe50 0x7fc4441961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 -- 172.21.15.44:0/3025423588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4440fec60 msgr2=0x7fc444195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4440fec60 0x7fc444195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 -- 172.21.15.44:0/3025423588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009900 con 0x7fc444100810 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc448e31640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440ffe50 0x7fc4441961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc449e33640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc44419d1d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02aeb0 tx=0x7fc43c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0307c0 con 0x7fc444100810 2024-09-13T17:26:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc43c040070 con 0x7fc444100810 2024-09-13T17:26:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44419dcf0 con 0x7fc444100810 2024-09-13T17:26:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0044e0 con 0x7fc444100810 2024-09-13T17:26:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.960+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44419e1a0 con 0x7fc444100810 2024-09-13T17:26:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.961+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44410a3d0 con 0x7fc444100810 2024-09-13T17:26:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.962+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc43c042030 con 0x7fc444100810 2024-09-13T17:26:33.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.963+0000 7fc43a7fc640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc4300a1110 0x7fc4300a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:33.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.963+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fc43c0ee320 con 0x7fc444100810 2024-09-13T17:26:33.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.963+0000 7fc449632640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc4300a1110 0x7fc4300a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:33.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.963+0000 7fc449632640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc4300a1110 0x7fc4300a35d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc4400058c0 tx=0x7fc44000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:33.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:33.967+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc43c0b7780 con 0x7fc444100810 2024-09-13T17:26:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:33 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3365745612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:33 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1105544896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:33 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1622678766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:34.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.139+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc44419e5e0 con 0x7fc444100810 2024-09-13T17:26:34.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.140+0000 7fc43a7fc640 1 -- 172.21.15.44:0/3025423588 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc43c0b96b0 con 0x7fc444100810 2024-09-13T17:26:34.141 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc4300a1110 msgr2=0x7fc4300a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc4300a1110 0x7fc4300a35d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc4400058c0 tx=0x7fc44000a2d0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 msgr2=0x7fc44419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc44419d1d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02aeb0 tx=0x7fc43c003c20 comp rx=0 tx=0).stop 2024-09-13T17:26:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 shutdown_connections 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fc4300a1110 0x7fc4300a35d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440ffe50 0x7fc4441961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc444100810 0x7fc44419d1d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 --2- 172.21.15.44:0/3025423588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4440fec60 0x7fc444195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 >> 172.21.15.44:0/3025423588 conn(0x7fc4440fa3e0 msgr2=0x7fc444067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 shutdown_connections 2024-09-13T17:26:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.142+0000 7fc44b896640 1 -- 172.21.15.44:0/3025423588 wait complete. 2024-09-13T17:26:34.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- 172.21.15.44:0/3722690262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0ff620 msgr2=0x7f892c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- 172.21.15.44:0/3722690262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0ff620 0x7f892c0ffaa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f891400aa10 tx=0x7f891402b720 comp rx=0 tx=0).stop 2024-09-13T17:26:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- 172.21.15.44:0/3722690262 shutdown_connections 2024-09-13T17:26:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- 172.21.15.44:0/3722690262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 0x7f892c106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- 172.21.15.44:0/3722690262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0ff620 0x7f892c0ffaa0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- 172.21.15.44:0/3722690262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0fe430 0x7f892c0fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- 172.21.15.44:0/3722690262 >> 172.21.15.44:0/3722690262 conn(0x7f892c0f9b70 msgr2=0x7f892c0fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- 172.21.15.44:0/3722690262 shutdown_connections 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- 172.21.15.44:0/3722690262 wait complete. 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 Processor -- start 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- start start 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 0x7f892c1023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0ff620 0x7f892c100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 0x7f892c100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c1016a0 con 0x7f892c0fe430 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c101810 con 0x7f892c0fffe0 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.259+0000 7f893323a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c19fd80 con 0x7f892c0ff620 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 0x7f892c1023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 0x7f892c1023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34882/0 (socket says 172.21.15.44:34882) 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 -- 172.21.15.44:0/1634530355 learned_addr learned my addr 172.21.15.44:0/1634530355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f89317d7640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 0x7f892c100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 -- 172.21.15.44:0/1634530355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0ff620 msgr2=0x7f892c100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f892bfff640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0ff620 0x7f892c100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0ff620 0x7f892c100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 -- 172.21.15.44:0/1634530355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 msgr2=0x7f892c100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 0x7f892c100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 -- 172.21.15.44:0/1634530355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891400a670 con 0x7f892c0fe430 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f89317d7640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 0x7f892c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f892bfff640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0ff620 0x7f892c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8930fd6640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 0x7f892c1023c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f891c00eb30 tx=0x7f891c00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c011470 con 0x7f892c0fe430 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f891c011a60 con 0x7f892c0fe430 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c19ffd0 con 0x7f892c0fe430 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c01a9c0 con 0x7f892c0fe430 2024-09-13T17:26:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.260+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c1a0450 con 0x7f892c0fe430 2024-09-13T17:26:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.262+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c109370 con 0x7f892c0fe430 2024-09-13T17:26:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.263+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f891c011bd0 con 0x7f892c0fe430 2024-09-13T17:26:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.263+0000 7f8929ffb640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89180a1090 0x7f89180a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.264+0000 7f892bfff640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89180a1090 0x7f89180a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.264+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f891c0cd280 con 0x7f892c0fe430 2024-09-13T17:26:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.264+0000 7f892bfff640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89180a1090 0x7f89180a3550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8914005ab0 tx=0x7f8914005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:34.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.268+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f891c0966d0 con 0x7f892c0fe430 2024-09-13T17:26:34.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.438+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f892c1a06d0 con 0x7f892c0fe430 2024-09-13T17:26:34.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.438+0000 7f8929ffb640 1 -- 172.21.15.44:0/1634530355 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f891c099cf0 con 0x7f892c0fe430 2024-09-13T17:26:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:34.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89180a1090 msgr2=0x7f89180a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89180a1090 0x7f89180a3550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8914005ab0 tx=0x7f8914005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 msgr2=0x7f892c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 0x7f892c1023c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f891c00eb30 tx=0x7f891c00c270 comp rx=0 tx=0).stop 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 shutdown_connections 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f89180a1090 0x7f89180a3550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c0ff620 0x7f892c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f892c0fffe0 0x7f892c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 --2- 172.21.15.44:0/1634530355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c0fe430 0x7f892c1023c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 >> 172.21.15.44:0/1634530355 conn(0x7f892c0f9b70 msgr2=0x7f892c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 shutdown_connections 2024-09-13T17:26:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.440+0000 7f893323a640 1 -- 172.21.15.44:0/1634530355 wait complete. 2024-09-13T17:26:34.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 -- 172.21.15.44:0/3768020579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07041007e0 msgr2=0x7f0704102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 --2- 172.21.15.44:0/3768020579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07041007e0 0x7f0704102bd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0700009900 tx=0x7f070002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 -- 172.21.15.44:0/3768020579 shutdown_connections 2024-09-13T17:26:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 --2- 172.21.15.44:0/3768020579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704105a40 0x7f0704107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 --2- 172.21.15.44:0/3768020579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0704103110 0x7f0704105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 --2- 172.21.15.44:0/3768020579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07041007e0 0x7f0704102bd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 -- 172.21.15.44:0/3768020579 >> 172.21.15.44:0/3768020579 conn(0x7f07040fa3e0 msgr2=0x7f07040fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 -- 172.21.15.44:0/3768020579 shutdown_connections 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 -- 172.21.15.44:0/3768020579 wait complete. 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 Processor -- start 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.557+0000 7f070bcd7640 1 -- start start 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041007e0 0x7f0704195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 0x7f07041961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0704105a40 0x7f070419d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070419d7c0 con 0x7f0704103110 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070419d930 con 0x7f0704105a40 2024-09-13T17:26:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070419daf0 con 0x7f07041007e0 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 0x7f07041961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 0x7f07041961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34906/0 (socket says 172.21.15.44:34906) 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 -- 172.21.15.44:0/2837152549 learned_addr learned my addr 172.21.15.44:0/2837152549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709a73640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041007e0 0x7f0704195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070a274640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0704105a40 0x7f070419d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 -- 172.21.15.44:0/2837152549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041007e0 msgr2=0x7f0704195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041007e0 0x7f0704195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 -- 172.21.15.44:0/2837152549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0704105a40 msgr2=0x7f070419d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0704105a40 0x7f070419d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 -- 172.21.15.44:0/2837152549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700009560 con 0x7f0704103110 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f0709272640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 0x7f07041961f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f06f4005fd0 tx=0x7f06f400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4009c30 con 0x7f0704103110 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06f40044b0 con 0x7f0704103110 2024-09-13T17:26:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070419ddd0 con 0x7f0704103110 2024-09-13T17:26:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4003da0 con 0x7f0704103110 2024-09-13T17:26:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.558+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070419e280 con 0x7f0704103110 2024-09-13T17:26:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.559+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07040fbed0 con 0x7f0704103110 2024-09-13T17:26:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.561+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f06f4004000 con 0x7f0704103110 2024-09-13T17:26:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.561+0000 7f06faffd640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06f00a1090 0x7f06f00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.561+0000 7f0709a73640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06f00a1090 0x7f06f00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.561+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f06f40cc9e0 con 0x7f0704103110 2024-09-13T17:26:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.562+0000 7f0709a73640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06f00a1090 0x7f06f00a3550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0700006010 tx=0x7f0700005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.565+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f06f4095e30 con 0x7f0704103110 2024-09-13T17:26:34.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.737+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f070402c480 con 0x7f0704103110 2024-09-13T17:26:34.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.737+0000 7f06faffd640 1 -- 172.21.15.44:0/2837152549 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06f40170a0 con 0x7f0704103110 2024-09-13T17:26:34.740 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06f00a1090 msgr2=0x7f06f00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06f00a1090 0x7f06f00a3550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0700006010 tx=0x7f0700005a40 comp rx=0 tx=0).stop 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 msgr2=0x7f07041961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 0x7f07041961f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f06f4005fd0 tx=0x7f06f400dde0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 shutdown_connections 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f06f00a1090 0x7f06f00a3550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0704105a40 0x7f070419d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0704103110 0x7f07041961f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 --2- 172.21.15.44:0/2837152549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041007e0 0x7f0704195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.739+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 >> 172.21.15.44:0/2837152549 conn(0x7f07040fa3e0 msgr2=0x7f07041012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.740+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 shutdown_connections 2024-09-13T17:26:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.740+0000 7f070bcd7640 1 -- 172.21.15.44:0/2837152549 wait complete. 2024-09-13T17:26:34.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:34 smithi044 ceph-mon[31175]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:34.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2603297871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:34.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3025423588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:34.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1634530355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:34.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/3308528565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280fec60 msgr2=0x7f2b280ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/3308528565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280fec60 0x7f2b280ff060 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009900 tx=0x7f2b1802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/3308528565 shutdown_connections 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/3308528565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b280ffe50 0x7f2b281002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/3308528565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b28100810 0x7f2b281070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/3308528565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280fec60 0x7f2b280ff060 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/3308528565 >> 172.21.15.44:0/3308528565 conn(0x7f2b280fa3e0 msgr2=0x7f2b280fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/3308528565 shutdown_connections 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/3308528565 wait complete. 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 Processor -- start 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.856+0000 7f2b2e9d5640 1 -- start start 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b280fec60 0x7f2b28195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280ffe50 0x7f2b28196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 0x7f2b2819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819d720 con 0x7f2b28100810 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819d890 con 0x7f2b280ffe50 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819da50 con 0x7f2b280fec60 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 0x7f2b2819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 0x7f2b2819d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34936/0 (socket says 172.21.15.44:34936) 2024-09-13T17:26:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 -- 172.21.15.44:0/2178012493 learned_addr learned my addr 172.21.15.44:0/2178012493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b27fff640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b280fec60 0x7f2b28195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b277fe640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280ffe50 0x7f2b28196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 -- 172.21.15.44:0/2178012493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b280fec60 msgr2=0x7f2b28195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b280fec60 0x7f2b28195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 -- 172.21.15.44:0/2178012493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280ffe50 msgr2=0x7f2b28196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280ffe50 0x7f2b28196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 -- 172.21.15.44:0/2178012493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18009560 con 0x7f2b28100810 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b277fe640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280ffe50 0x7f2b28196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2cf72640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 0x7f2b2819d1b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000cce0 tx=0x7f2b2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2000f450 con 0x7f2b28100810 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b2000fa40 con 0x7f2b28100810 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200189a0 con 0x7f2b28100810 2024-09-13T17:26:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2819dd30 con 0x7f2b28100810 2024-09-13T17:26:34.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.857+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2819e190 con 0x7f2b28100810 2024-09-13T17:26:34.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.859+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28106190 con 0x7f2b28100810 2024-09-13T17:26:34.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.860+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2b2000fbb0 con 0x7f2b28100810 2024-09-13T17:26:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.860+0000 7f2b257fa640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2b100a1110 0x7f2b100a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.860+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f2b200cd140 con 0x7f2b28100810 2024-09-13T17:26:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.860+0000 7f2b27fff640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2b100a1110 0x7f2b100a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.861+0000 7f2b27fff640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2b100a1110 0x7f2b100a35d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2b1802aa00 tx=0x7f2b18005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:34.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:34.865+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2b20096500 con 0x7f2b28100810 2024-09-13T17:26:34.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:34 smithi070 ceph-mon[34502]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:34.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2603297871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:34.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3025423588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:34.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1634530355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:34 smithi100 ceph-mon[35055]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-13T17:26:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2603297871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3025423588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1634530355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.035+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2b2802c480 con 0x7f2b28100810 2024-09-13T17:26:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.035+0000 7f2b257fa640 1 -- 172.21.15.44:0/2178012493 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2b20098430 con 0x7f2b28100810 2024-09-13T17:26:35.037 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2b100a1110 msgr2=0x7f2b100a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2b100a1110 0x7f2b100a35d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2b1802aa00 tx=0x7f2b18005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 msgr2=0x7f2b2819d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 0x7f2b2819d1b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000cce0 tx=0x7f2b2000a620 comp rx=0 tx=0).stop 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 shutdown_connections 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f2b100a1110 0x7f2b100a35d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b28100810 0x7f2b2819d1b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280ffe50 0x7f2b28196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 --2- 172.21.15.44:0/2178012493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b280fec60 0x7f2b28195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 >> 172.21.15.44:0/2178012493 conn(0x7f2b280fa3e0 msgr2=0x7f2b28103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 shutdown_connections 2024-09-13T17:26:35.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.038+0000 7f2b2e9d5640 1 -- 172.21.15.44:0/2178012493 wait complete. 2024-09-13T17:26:35.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/807098172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 msgr2=0x7fdf741070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/807098172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf741070e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009ca0 tx=0x7fdf6c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/807098172 shutdown_connections 2024-09-13T17:26:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/807098172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf740ffe90 0x7fdf74100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/807098172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf741070e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/807098172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740feca0 0x7fdf740ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/807098172 >> 172.21.15.44:0/807098172 conn(0x7fdf740fa440 msgr2=0x7fdf740fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/807098172 shutdown_connections 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.152+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/807098172 wait complete. 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 Processor -- start 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 -- start start 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf740feca0 0x7fdf74197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740ffe90 0x7fdf74198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf7419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7419f8c0 con 0x7fdf74100850 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7419fa30 con 0x7fdf740ffe90 2024-09-13T17:26:35.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7ab7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7419fbf0 con 0x7fdf740feca0 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf7419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf7419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34968/0 (socket says 172.21.15.44:34968) 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 -- 172.21.15.44:0/2338994996 learned_addr learned my addr 172.21.15.44:0/2338994996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf78919640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf740feca0 0x7fdf74197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf73fff640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740ffe90 0x7fdf74198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 -- 172.21.15.44:0/2338994996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf740feca0 msgr2=0x7fdf74197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf740feca0 0x7fdf74197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 -- 172.21.15.44:0/2338994996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740ffe90 msgr2=0x7fdf74198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740ffe90 0x7fdf74198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 -- 172.21.15.44:0/2338994996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c009900 con 0x7fdf74100850 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.153+0000 7fdf7911a640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf7419f380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c02aeb0 tx=0x7fdf6c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.154+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0307c0 con 0x7fdf74100850 2024-09-13T17:26:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.154+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf6c040070 con 0x7fdf74100850 2024-09-13T17:26:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.154+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7419fe70 con 0x7fdf74100850 2024-09-13T17:26:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.154+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0043c0 con 0x7fdf74100850 2024-09-13T17:26:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.154+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf741a0340 con 0x7fdf74100850 2024-09-13T17:26:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.155+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7410a3f0 con 0x7fdf74100850 2024-09-13T17:26:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.156+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdf6c042030 con 0x7fdf74100850 2024-09-13T17:26:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.157+0000 7fdf71ffb640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdf640a1110 0x7fdf640a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.157+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fdf6c0ee320 con 0x7fdf74100850 2024-09-13T17:26:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.157+0000 7fdf78919640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdf640a1110 0x7fdf640a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.158+0000 7fdf78919640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdf640a1110 0x7fdf640a35d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf740ffcf0 tx=0x7fdf6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.161+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdf6c0b7710 con 0x7fdf74100850 2024-09-13T17:26:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.367+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf7410a5d0 con 0x7fdf74100850 2024-09-13T17:26:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.367+0000 7fdf71ffb640 1 -- 172.21.15.44:0/2338994996 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fdf6c0bad30 con 0x7fdf74100850 2024-09-13T17:26:35.369 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:35.369 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:24:47.147746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 24 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdf640a1110 msgr2=0x7fdf640a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdf640a1110 0x7fdf640a35d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf740ffcf0 tx=0x7fdf6800b390 comp rx=0 tx=0).stop 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 msgr2=0x7fdf7419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf7419f380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c02aeb0 tx=0x7fdf6c003c20 comp rx=0 tx=0).stop 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 shutdown_connections 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fdf640a1110 0x7fdf640a35d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74100850 0x7fdf7419f380 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740ffe90 0x7fdf74198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 --2- 172.21.15.44:0/2338994996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf740feca0 0x7fdf74197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 >> 172.21.15.44:0/2338994996 conn(0x7fdf740fa440 msgr2=0x7fdf74067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 shutdown_connections 2024-09-13T17:26:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.369+0000 7fdf7ab7d640 1 -- 172.21.15.44:0/2338994996 wait complete. 2024-09-13T17:26:35.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 -- 172.21.15.44:0/891856970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 msgr2=0x7f0734102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 --2- 172.21.15.44:0/891856970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f0734102c40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0724009900 tx=0x7f072402a410 comp rx=0 tx=0).stop 2024-09-13T17:26:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 -- 172.21.15.44:0/891856970 shutdown_connections 2024-09-13T17:26:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 --2- 172.21.15.44:0/891856970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734103180 0x7f0734105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 --2- 172.21.15.44:0/891856970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734105ab0 0x7f0734107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 --2- 172.21.15.44:0/891856970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f0734102c40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.482+0000 7f0739ad7640 1 -- 172.21.15.44:0/891856970 >> 172.21.15.44:0/891856970 conn(0x7f07340fa430 msgr2=0x7f07340fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.483+0000 7f0739ad7640 1 -- 172.21.15.44:0/891856970 shutdown_connections 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.483+0000 7f0739ad7640 1 -- 172.21.15.44:0/891856970 wait complete. 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.483+0000 7f0739ad7640 1 Processor -- start 2024-09-13T17:26:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.483+0000 7f0739ad7640 1 -- start start 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.483+0000 7f0739ad7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f07341001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.483+0000 7f0739ad7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734103180 0x7f07340fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0739ad7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734105ab0 0x7f07340fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0739ad7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340ff4a0 con 0x7f0734100850 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0739ad7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340ff610 con 0x7f0734103180 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0739ad7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073419fe00 con 0x7f0734105ab0 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f07341001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f07341001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34992/0 (socket says 172.21.15.44:34992) 2024-09-13T17:26:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 -- 172.21.15.44:0/3463210084 learned_addr learned my addr 172.21.15.44:0/3463210084 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f072affd640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734103180 0x7f07340fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0733fff640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734105ab0 0x7f07340fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 -- 172.21.15.44:0/3463210084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734105ab0 msgr2=0x7f07340fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734105ab0 0x7f07340fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 -- 172.21.15.44:0/3463210084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734103180 msgr2=0x7f07340fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734103180 0x7f07340fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 -- 172.21.15.44:0/3463210084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0724009560 con 0x7f0734100850 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07337fe640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f07341001c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0724005ab0 tx=0x7f07240052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724031d70 con 0x7f0734100850 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07240054a0 con 0x7f0734100850 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724003a30 con 0x7f0734100850 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07341a0050 con 0x7f0734100850 2024-09-13T17:26:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.484+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07341a0500 con 0x7f0734100850 2024-09-13T17:26:35.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.485+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07340fbf20 con 0x7f0734100850 2024-09-13T17:26:35.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.487+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0724003bb0 con 0x7f0734100850 2024-09-13T17:26:35.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.487+0000 7f07317fa640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f07200a1110 0x7f07200a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.487+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f07240edf10 con 0x7f0734100850 2024-09-13T17:26:35.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.487+0000 7f072affd640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f07200a1110 0x7f07200a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.488+0000 7f072affd640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f07200a1110 0x7f07200a35d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07340fff30 tx=0x7f071c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.491+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f07240b7360 con 0x7f0734100850 2024-09-13T17:26:35.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.664+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07340fb990 con 0x7f0734100850 2024-09-13T17:26:35.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.666+0000 7f07317fa640 1 -- 172.21.15.44:0/3463210084 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24492 (secure 0 0 0) 0x7f07240ba980 con 0x7f0734100850 2024-09-13T17:26:35.667 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:35.667 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":92,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:25:43.045876+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:35.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f07200a1110 msgr2=0x7f07200a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f07200a1110 0x7f07200a35d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07340fff30 tx=0x7f071c00b410 comp rx=0 tx=0).stop 2024-09-13T17:26:35.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 msgr2=0x7f07341001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f07341001c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0724005ab0 tx=0x7f07240052b0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 shutdown_connections 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f07200a1110 0x7f07200a35d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734105ab0 0x7f07340fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734103180 0x7f07340fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 --2- 172.21.15.44:0/3463210084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0734100850 0x7f07341001c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 >> 172.21.15.44:0/3463210084 conn(0x7f07340fa430 msgr2=0x7f07341011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 shutdown_connections 2024-09-13T17:26:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.668+0000 7f0739ad7640 1 -- 172.21.15.44:0/3463210084 wait complete. 2024-09-13T17:26:35.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-13T17:24:21.722369+0000', 'modified': '2024-09-13T17:24:47.147746+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24517, 'mds_1': 24496, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6833/459553980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 459553980}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 459553980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6837/4202473697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 4202473697}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 4202473697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.70:6839/2731404816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6838', 'nonce': 2731404816}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 2731404816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6835/4065662345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 4065662345}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 4065662345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.100:6837/288521582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 288521582}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 288521582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.100:6839/3416533473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 3416533473}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 3416533473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:26:35.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-13T17:26:35.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-09-13T17:26:35.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 -- 172.21.15.44:0/4082035839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134103160 msgr2=0x7fd134105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 --2- 172.21.15.44:0/4082035839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134103160 0x7fd134105550 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009900 tx=0x7fd11c02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 -- 172.21.15.44:0/4082035839 shutdown_connections 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 --2- 172.21.15.44:0/4082035839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd134105a90 0x7fd134107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 --2- 172.21.15.44:0/4082035839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134103160 0x7fd134105550 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 --2- 172.21.15.44:0/4082035839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134100830 0x7fd134102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 -- 172.21.15.44:0/4082035839 >> 172.21.15.44:0/4082035839 conn(0x7fd1340fa430 msgr2=0x7fd1340fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 -- 172.21.15.44:0/4082035839 shutdown_connections 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.783+0000 7fd1392e9640 1 -- 172.21.15.44:0/4082035839 wait complete. 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 Processor -- start 2024-09-13T17:26:35.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 -- start start 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134100830 0x7fd1341001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd134103160 0x7fd1340fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 0x7fd1340fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340ff480 con 0x7fd134105a90 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340ff5f0 con 0x7fd134103160 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1392e9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13419fe00 con 0x7fd134100830 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1337fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 0x7fd1340fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1337fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 0x7fd1340fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35000/0 (socket says 172.21.15.44:35000) 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1337fe640 1 -- 172.21.15.44:0/11141048 learned_addr learned my addr 172.21.15.44:0/11141048 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.784+0000 7fd1327fc640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd134103160 0x7fd1340fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd132ffd640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134100830 0x7fd1341001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1337fe640 1 -- 172.21.15.44:0/11141048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134100830 msgr2=0x7fd1341001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1337fe640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134100830 0x7fd1341001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1337fe640 1 -- 172.21.15.44:0/11141048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd134103160 msgr2=0x7fd1340fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1337fe640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd134103160 0x7fd1340fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1337fe640 1 -- 172.21.15.44:0/11141048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009560 con 0x7fd134105a90 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1337fe640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 0x7fd1340fed20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00eb70 tx=0x7fd12c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd12bfff640 1 -- 172.21.15.44:0/11141048 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c004d20 con 0x7fd134105a90 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1341a00b0 con 0x7fd134105a90 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd12bfff640 1 -- 172.21.15.44:0/11141048 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd12c004e80 con 0x7fd134105a90 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd12bfff640 1 -- 172.21.15.44:0/11141048 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c010690 con 0x7fd134105a90 2024-09-13T17:26:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.785+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1341a0540 con 0x7fd134105a90 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.787+0000 7fd12bfff640 1 -- 172.21.15.44:0/11141048 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd12c005020 con 0x7fd134105a90 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.787+0000 7fd13c00a740 1 Processor -- start 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 -- start start 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bda5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bd007c0 con 0x55ff6bdaaed0 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bd0fb70 con 0x55ff6bda5e40 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd13c00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bd0d300 con 0x55ff6bda5500 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd1327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd1327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35004/0 (socket says 172.21.15.44:35004) 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd1327fc640 1 -- 172.21.15.44:0/198309215 learned_addr learned my addr 172.21.15.44:0/198309215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd1327fc640 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bda5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd12bfff640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1200a1090 0x7fd1200a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd132ffd640 1 -- 172.21.15.44:0/198309215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 msgr2=0x55ff6bda5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd132ffd640 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bda5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd132ffd640 1 -- 172.21.15.44:0/198309215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 msgr2=0x55ff6bdad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd132ffd640 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd1337fe640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1200a1090 0x7fd1200a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd132ffd640 1 -- 172.21.15.44:0/198309215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6bc954b0 con 0x55ff6bda5e40 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.788+0000 7fd1327fc640 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.789+0000 7fd12bfff640 1 -- 172.21.15.44:0/11141048 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fd12c014070 con 0x7fd134105a90 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.789+0000 7fd132ffd640 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaa900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd124009b30 tx=0x7fd12400e2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d9d212fd76c988 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.789+0000 7fd1337fe640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1200a1090 0x7fd1200a3550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd12c017000 tx=0x7fd12c0ba040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.789+0000 7fd1297fa640 1 -- 172.21.15.44:0/198309215 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400ed20 con 0x55ff6bda5e40 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.789+0000 7fd1297fa640 1 -- 172.21.15.44:0/198309215 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd12400ee80 con 0x55ff6bda5e40 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.789+0000 7fd1297fa640 1 -- 172.21.15.44:0/198309215 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124004e40 con 0x55ff6bda5e40 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- 172.21.15.44:0/198309215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 msgr2=0x55ff6bdaa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaa900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd124009b30 tx=0x7fd12400e2f0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- 172.21.15.44:0/198309215 shutdown_connections 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaa900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- 172.21.15.44:0/198309215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bda5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- 172.21.15.44:0/198309215 >> 172.21.15.44:0/198309215 conn(0x55ff6bda0b70 msgr2=0x55ff6bda0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- 172.21.15.44:0/198309215 shutdown_connections 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- 172.21.15.44:0/198309215 wait complete. 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- start start 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 Processor -- start 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 0x55ff6bdaf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bd08bf0 con 0x55ff6bda5500 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bcc7250 con 0x55ff6bda5e40 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.790+0000 7fd13c00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bcc6ef0 con 0x55ff6bdaaed0 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd13c00a740 10 client.0 _ll_register_callbacks cb 0x55ff6bcf9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:35.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 0x55ff6bdaf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 0x55ff6bdaf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35020/0 (socket says 172.21.15.44:35020) 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 -- 172.21.15.44:0/2072514404 learned_addr learned my addr 172.21.15.44:0/2072514404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 -- 172.21.15.44:0/2072514404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bdaaed0 msgr2=0x55ff6bdb6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 -- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 msgr2=0x55ff6bdaf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 -- 172.21.15.44:0/2072514404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd124009790 con 0x55ff6bda5500 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd1327fc640 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 0x55ff6bdaf0e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd11c02a930 tx=0x7fd11c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb41438e86b198 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd11a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd11a7fc640 1 -- 172.21.15.44:0/2072514404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c031880 con 0x55ff6bda5500 2024-09-13T17:26:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd11a7fc640 1 -- 172.21.15.44:0/2072514404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fd11c0319e0 con 0x55ff6bda5500 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff6b9fc3c0 con 0x55ff6bda5500 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd11a7fc640 1 -- 172.21.15.44:0/2072514404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd11c031ba0 con 0x55ff6bda5500 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd13c00a740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.791+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff6bd07270 con 0x55ff6bda5500 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd11a7fc640 1 -- 172.21.15.44:0/2072514404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff6b9fc3c0 con 0x55ff6bda5500 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd11a7fc640 1 -- 172.21.15.44:0/2072514404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd11c04aba0 con 0x55ff6bda5500 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd13c00a740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd13c00a740 10 client.15402 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd13c00a740 20 client.15402 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd13c00a740 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6bdbb580 0x55ff6bdbf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd13c00a740 4 client.15402 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff6ba0d6c0 con 0x55ff6bdbb580 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.792+0000 7fd132ffd640 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6bdbb580 0x55ff6bdbf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd11a7fc640 10 client.15402 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd132ffd640 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6bdbb580 0x55ff6bdbf700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd5dfa2a7431b4a server_cookie=ab2ed8f3f4924f50 in_seq=0 out_seq=0 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd11a7fc640 1 -- 172.21.15.44:0/2072514404 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ff6ba0d6c0 con 0x55ff6bdbb580 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd11a7fc640 10 client.15402 handle_command_reply: tid=0 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 client.15402 shutdown 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 msgr2=0x55ff6bdaf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 0x55ff6bdaf0e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd11c02a930 tx=0x7fd11c0044a0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 shutdown_connections 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6bdbb580 0x55ff6bdbf700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdaf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 --2- 172.21.15.44:0/2072514404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5500 0x55ff6bdaf0e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd11a7fc640 0 client.15402 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:35.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 >> 172.21.15.44:0/2072514404 conn(0x55ff6bda0b70 msgr2=0x55ff6bda2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.793+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 shutdown_connections 2024-09-13T17:26:35.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.794+0000 7fd13c00a740 1 -- 172.21.15.44:0/2072514404 wait complete. 2024-09-13T17:26:35.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.794+0000 7fd13c00a740 20 client.15402 trim_cache size 0 max 16384 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.808+0000 7fd13c00a740 1 Processor -- start 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.808+0000 7fd13c00a740 1 -- start start 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.808+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdaee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.808+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.808+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdaf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.808+0000 7fd13c00a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bc8c460 con 0x55ff6bda5e40 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd13c00a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bdb7a00 con 0x55ff6bdaaed0 2024-09-13T17:26:35.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd13c00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bdb7300 con 0x55ff6bda5500 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35030/0 (socket says 172.21.15.44:35030) 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 -- 172.21.15.44:0/3648639156 learned_addr learned my addr 172.21.15.44:0/3648639156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 -- 172.21.15.44:0/3648639156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 msgr2=0x55ff6bdaee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd1327fc640 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdaee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdaee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 -- 172.21.15.44:0/3648639156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bdaaed0 msgr2=0x55ff6bdaf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdaf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 -- 172.21.15.44:0/3648639156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6bca95f0 con 0x55ff6bda5e40 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd132ffd640 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb6580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd12400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c28fe38ddf1687c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd129ffb640 1 -- 172.21.15.44:0/3648639156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124021070 con 0x55ff6bda5e40 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd129ffb640 1 -- 172.21.15.44:0/3648639156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd124022040 con 0x55ff6bda5e40 2024-09-13T17:26:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.809+0000 7fd129ffb640 1 -- 172.21.15.44:0/3648639156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124027440 con 0x55ff6bda5e40 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- 172.21.15.44:0/3648639156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 msgr2=0x55ff6bdb6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb6580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd12400edb0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- 172.21.15.44:0/3648639156 shutdown_connections 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdaf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb6580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- 172.21.15.44:0/3648639156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdaee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- 172.21.15.44:0/3648639156 >> 172.21.15.44:0/3648639156 conn(0x55ff6bda0b70 msgr2=0x55ff6bdaa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- 172.21.15.44:0/3648639156 shutdown_connections 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- 172.21.15.44:0/3648639156 wait complete. 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- start start 2024-09-13T17:26:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 Processor -- start 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdb64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bdb3b40 con 0x55ff6bdaaed0 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6bdb3cb0 con 0x55ff6bda5e40 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.810+0000 7fd13c00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6be11440 con 0x55ff6bda5500 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd13c00a740 10 client.0 _ll_register_callbacks cb 0x7fd124009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35040/0 (socket says 172.21.15.44:35040) 2024-09-13T17:26:35.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 -- 172.21.15.44:0/43750295 learned_addr learned my addr 172.21.15.44:0/43750295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd132ffd640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdb64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 -- 172.21.15.44:0/43750295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 msgr2=0x55ff6bdb64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdb64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 -- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 msgr2=0x55ff6bdb3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 -- 172.21.15.44:0/43750295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff6bc954b0 con 0x55ff6bdaaed0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd1327fc640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb3580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd11c005ec0 tx=0x7fd11c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b928455f2126b739 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd11a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd11a7fc640 1 -- 172.21.15.44:0/43750295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c030780 con 0x55ff6bdaaed0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd11a7fc640 1 -- 172.21.15.44:0/43750295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fd11c000e40 con 0x55ff6bdaaed0 2024-09-13T17:26:35.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd12400ee80 con 0x55ff6bdaaed0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd11a7fc640 1 -- 172.21.15.44:0/43750295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd11c0308e0 con 0x55ff6bdaaed0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd13c00a740 10 client.15408 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.811+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff6bcced00 con 0x55ff6bdaaed0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd11a7fc640 1 -- 172.21.15.44:0/43750295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd12400ee80 con 0x55ff6bdaaed0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd11a7fc640 1 -- 172.21.15.44:0/43750295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd11c03caf0 con 0x55ff6bdaaed0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd13c00a740 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd13c00a740 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd13c00a740 20 client.15408 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd13c00a740 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6be160c0 0x55ff6be12a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd13c00a740 4 client.15408 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ff6ba60d10 con 0x55ff6be160c0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd132ffd640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6be160c0 0x55ff6be12a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd11a7fc640 10 client.15408 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.812+0000 7fd132ffd640 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6be160c0 0x55ff6be12a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30c31ac1ef1a47f server_cookie=ed8ed6874314ddd5 in_seq=0 out_seq=0 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd11a7fc640 1 -- 172.21.15.44:0/43750295 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55ff6ba60d10 con 0x55ff6be160c0 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd11a7fc640 10 client.15408 handle_command_reply: tid=0 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 client.15408 shutdown 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 msgr2=0x55ff6bdb3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb3580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd11c005ec0 tx=0x7fd11c03f040 comp rx=0 tx=0).stop 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 shutdown_connections 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ff6be160c0 0x55ff6be12a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff6bdaaed0 0x55ff6bdb3580 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff6bda5e40 0x55ff6bdb3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 --2- 172.21.15.44:0/43750295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff6bda5500 0x55ff6bdb64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd11a7fc640 0 client.15408 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:26:35.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 >> 172.21.15.44:0/43750295 conn(0x55ff6bda0b70 msgr2=0x55ff6bdc3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 shutdown_connections 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 1 -- 172.21.15.44:0/43750295 wait complete. 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd13c00a740 20 client.15408 trim_cache size 0 max 16384 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24449, 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:26:35.821 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:26:35.822 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 109.669381344, 2024-09-13T17:26:35.822 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 24, 2024-09-13T17:26:35.822 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 92, 2024-09-13T17:26:35.822 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 90, 2024-09-13T17:26:35.822 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 149.385862817 2024-09-13T17:26:35.822 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1200a1090 msgr2=0x7fd1200a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1200a1090 0x7fd1200a3550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd12c017000 tx=0x7fd12c0ba040 comp rx=0 tx=0).stop 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 msgr2=0x7fd1340fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 0x7fd1340fed20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00eb70 tx=0x7fd12c00c6a0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 shutdown_connections 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fd1200a1090 0x7fd1200a3550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd134105a90 0x7fd1340fed20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd134103160 0x7fd1340fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 --2- 172.21.15.44:0/11141048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134100830 0x7fd1341001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.813+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 >> 172.21.15.44:0/11141048 conn(0x7fd1340fa430 msgr2=0x7fd1341011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.814+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 shutdown_connections 2024-09-13T17:26:35.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.814+0000 7fd1392e9640 1 -- 172.21.15.44:0/11141048 wait complete. 2024-09-13T17:26:35.825 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T17:26:35.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 -- 172.21.15.44:0/1718225248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550105a90 msgr2=0x7f8550107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1718225248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550105a90 0x7f8550107f00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ca0 tx=0x7f854802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 -- 172.21.15.44:0/1718225248 shutdown_connections 2024-09-13T17:26:35.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1718225248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550105a90 0x7f8550107f00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1718225248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550103160 0x7f8550105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1718225248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8550100830 0x7f8550102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 -- 172.21.15.44:0/1718225248 >> 172.21.15.44:0/1718225248 conn(0x7f85500fa430 msgr2=0x7f85500fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 -- 172.21.15.44:0/1718225248 shutdown_connections 2024-09-13T17:26:35.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.927+0000 7f8554fd4640 1 -- 172.21.15.44:0/1718225248 wait complete. 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 Processor -- start 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 -- start start 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 0x7f85501001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8550103160 0x7f85500fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550105a90 0x7f85500fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500ff480 con 0x7f8550100830 2024-09-13T17:26:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500ff5f0 con 0x7f8550105a90 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f8554fd4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855019fd30 con 0x7f8550103160 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f854e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 0x7f85501001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f854e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 0x7f85501001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35046/0 (socket says 172.21.15.44:35046) 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f854e59c640 1 -- 172.21.15.44:0/1160084336 learned_addr learned my addr 172.21.15.44:0/1160084336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8550103160 0x7f85500fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.928+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550105a90 0x7f85500fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854e59c640 1 -- 172.21.15.44:0/1160084336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8550103160 msgr2=0x7f85500fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854e59c640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8550103160 0x7f85500fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854e59c640 1 -- 172.21.15.44:0/1160084336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550105a90 msgr2=0x7f85500fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854e59c640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550105a90 0x7f85500fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854e59c640 1 -- 172.21.15.44:0/1160084336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548009900 con 0x7f8550100830 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550105a90 0x7f85500fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f854e59c640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 0x7f85501001a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f854000c8c0 tx=0x7f854000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f85477fe640 1 -- 172.21.15.44:0/1160084336 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540004220 con 0x7f8550100830 2024-09-13T17:26:35.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855019ffe0 con 0x7f8550100830 2024-09-13T17:26:35.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f85477fe640 1 -- 172.21.15.44:0/1160084336 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8540007e40 con 0x7f8550100830 2024-09-13T17:26:35.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f85477fe640 1 -- 172.21.15.44:0/1160084336 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400047a0 con 0x7f8550100830 2024-09-13T17:26:35.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.929+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85501a0400 con 0x7f8550100830 2024-09-13T17:26:35.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.931+0000 7f85477fe640 1 -- 172.21.15.44:0/1160084336 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f85400079b0 con 0x7f8550100830 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.931+0000 7f8557dd0740 1 Processor -- start 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 -- start start 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ff500 0x561ab94ff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab9504900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab9507390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab945a7c0 con 0x561ab94ffe40 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab9469b70 con 0x561ab9504ed0 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f8557dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab9467300 con 0x561ab94ff500 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab9504900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab9504900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35052/0 (socket says 172.21.15.44:35052) 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854e59c640 1 -- 172.21.15.44:0/3380510460 learned_addr learned my addr 172.21.15.44:0/3380510460 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ff500 0x561ab94ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854dd9b640 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab9507390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f85477fe640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f853c0a1090 0x7f853c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854ed9d640 1 -- 172.21.15.44:0/3380510460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 msgr2=0x561ab9507390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab9507390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854ed9d640 1 -- 172.21.15.44:0/3380510460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 msgr2=0x561ab9504900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.932+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab9504900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f854ed9d640 1 -- 172.21.15.44:0/3380510460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ab93ef4b0 con 0x561ab94ff500 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f854e59c640 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab9504900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f85477fe640 1 -- 172.21.15.44:0/1160084336 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f85400cc960 con 0x7f8550100830 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f853c0a1090 0x7f853c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ff500 0x561ab94ff900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f854802aeb0 tx=0x7f854802f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e193f7e26ca206d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f853c0a1090 0x7f853c0a3550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x561ab9507bd0 tx=0x7f854804e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f8544ff9640 1 -- 172.21.15.44:0/3380510460 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480307c0 con 0x561ab94ff500 2024-09-13T17:26:35.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f8544ff9640 1 -- 172.21.15.44:0/3380510460 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8548004040 con 0x561ab94ff500 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.933+0000 7f8544ff9640 1 -- 172.21.15.44:0/3380510460 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854802f840 con 0x561ab94ff500 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 -- 172.21.15.44:0/3380510460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ff500 msgr2=0x561ab94ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ff500 0x561ab94ff900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f854802aeb0 tx=0x7f854802f460 comp rx=0 tx=0).stop 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 -- 172.21.15.44:0/3380510460 shutdown_connections 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab9507390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab9504900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3380510460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ff500 0x561ab94ff900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 -- 172.21.15.44:0/3380510460 >> 172.21.15.44:0/3380510460 conn(0x561ab94fab70 msgr2=0x561ab94faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 -- 172.21.15.44:0/3380510460 shutdown_connections 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 -- 172.21.15.44:0/3380510460 wait complete. 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 -- start start 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.934+0000 7f8557dd0740 1 Processor -- start 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab9509190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab95096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab9504ed0 0x561ab9510710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab9462bf0 con 0x561ab94ffe40 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab9421250 con 0x561ab94ff500 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab9420ef0 con 0x561ab9504ed0 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 10 client.0 _ll_register_callbacks cb 0x561ab9453740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab95096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab95096d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35056/0 (socket says 172.21.15.44:35056) 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 -- 172.21.15.44:0/1623340371 learned_addr learned my addr 172.21.15.44:0/1623340371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854e59c640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab9509190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab9504ed0 0x561ab9510710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 -- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab9504ed0 msgr2=0x561ab9510710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab9504ed0 0x561ab9510710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 -- 172.21.15.44:0/1623340371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 msgr2=0x561ab9509190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab9509190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 -- 172.21.15.44:0/1623340371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8550108ae0 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab9504ed0 0x561ab9510710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab95096d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8538010820 tx=0x7f8538010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603a1a198c2842d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8535ffb640 1 -- 172.21.15.44:0/1623340371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538015070 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8535ffb640 1 -- 172.21.15.44:0/1623340371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f853800fb10 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.935+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ab91563c0 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8535ffb640 1 -- 172.21.15.44:0/1623340371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f853800fcd0 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561ab9461270 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8535ffb640 1 -- 172.21.15.44:0/1623340371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561ab91563c0 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8535ffb640 1 -- 172.21.15.44:0/1623340371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f853800e440 con 0x561ab94ffe40 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:35.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 10 client.15420 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 20 client.15420 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9514c90 0x561ab9519930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 4 client.15420 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ab91676c0 con 0x561ab9514c90 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.936+0000 7f854e59c640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9514c90 0x561ab9519930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8535ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f854e59c640 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9514c90 0x561ab9519930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe0553caa76ef061 server_cookie=fdc68241f00d4386 in_seq=0 out_seq=0 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8535ffb640 1 -- 172.21.15.44:0/1623340371 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561ab91676c0 con 0x561ab9514c90 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8535ffb640 10 client.15420 handle_command_reply: tid=0 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 client.15420 shutdown 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 msgr2=0x561ab95096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab95096d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8538010820 tx=0x7f8538010ce0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 shutdown_connections 2024-09-13T17:26:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9514c90 0x561ab9519930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab9504ed0 0x561ab9510710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8535ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ffe40 0x561ab95096d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1623340371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab9509190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.937+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 >> 172.21.15.44:0/1623340371 conn(0x561ab94fab70 msgr2=0x561ab94fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.938+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 shutdown_connections 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.938+0000 7f8557dd0740 1 -- 172.21.15.44:0/1623340371 wait complete. 2024-09-13T17:26:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.938+0000 7f8557dd0740 20 client.15420 trim_cache size 0 max 16384 2024-09-13T17:26:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2837152549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2178012493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2338994996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3463210084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 Processor -- start 2024-09-13T17:26:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 -- start start 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab95088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab95106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 0x561ab9509650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab93e6460 con 0x561ab9504ed0 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab95013e0 con 0x561ab94ff500 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.952+0000 7f8557dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab94f6600 con 0x561ab94ffe40 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 0x561ab9509650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 0x561ab9509650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35068/0 (socket says 172.21.15.44:35068) 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 -- 172.21.15.44:0/3233265464 learned_addr learned my addr 172.21.15.44:0/3233265464 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854e59c640 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab95088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854dd9b640 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab95106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 -- 172.21.15.44:0/3233265464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 msgr2=0x561ab95106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab95106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 -- 172.21.15.44:0/3233265464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 msgr2=0x561ab95088e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab95088e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 -- 172.21.15.44:0/3233265464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ab94035f0 con 0x561ab9504ed0 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854e59c640 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab95088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f854ed9d640 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 0x561ab9509650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8548006010 tx=0x7f85480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a907cc4159c9085c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f85457fa640 1 -- 172.21.15.44:0/3233265464 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854802f820 con 0x561ab9504ed0 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f85457fa640 1 -- 172.21.15.44:0/3233265464 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8548057040 con 0x561ab9504ed0 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.953+0000 7f85457fa640 1 -- 172.21.15.44:0/3233265464 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480305c0 con 0x561ab9504ed0 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- 172.21.15.44:0/3233265464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 msgr2=0x561ab9509650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 0x561ab9509650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8548006010 tx=0x7f85480095c0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- 172.21.15.44:0/3233265464 shutdown_connections 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab95106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab9504ed0 0x561ab9509650 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- 172.21.15.44:0/3233265464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab94ff500 0x561ab95088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- 172.21.15.44:0/3233265464 >> 172.21.15.44:0/3233265464 conn(0x561ab94fab70 msgr2=0x561ab9508f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- 172.21.15.44:0/3233265464 shutdown_connections 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- 172.21.15.44:0/3233265464 wait complete. 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- start start 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 Processor -- start 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 0x561ab950b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab950ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab956b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab950c070 con 0x561ab94ff500 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab956bb30 con 0x561ab9504ed0 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab956bc70 con 0x561ab94ffe40 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.954+0000 7f8557dd0740 10 client.0 _ll_register_callbacks cb 0x7f8548009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 0x561ab950b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 0x561ab950b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35072/0 (socket says 172.21.15.44:35072) 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 -- 172.21.15.44:0/1241478454 learned_addr learned my addr 172.21.15.44:0/1241478454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854ed9d640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab956b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab950ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 -- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 msgr2=0x561ab950ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab950ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 -- 172.21.15.44:0/1241478454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 msgr2=0x561ab956b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab956b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 -- 172.21.15.44:0/1241478454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ab93ef4b0 con 0x561ab94ff500 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f854e59c640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 0x561ab950b4d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8540007fd0 tx=0x7f85400c9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0acca64dd91a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8535ffb640 1 -- 172.21.15.44:0/1241478454 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540004380 con 0x561ab94ff500 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8535ffb640 1 -- 172.21.15.44:0/1241478454 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f8540007c60 con 0x561ab94ff500 2024-09-13T17:26:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8548004040 con 0x561ab94ff500 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8535ffb640 1 -- 172.21.15.44:0/1241478454 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f854009a110 con 0x561ab94ff500 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8557dd0740 10 client.15426 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561ab9428d00 con 0x561ab94ff500 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.955+0000 7f8535ffb640 1 -- 172.21.15.44:0/1241478454 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f854007c020 con 0x561ab94ff500 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8535ffb640 1 -- 172.21.15.44:0/1241478454 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8540016030 con 0x561ab94ff500 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 10 client.15426 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 10 client.15426 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 20 client.15426 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9570aa0 0x561ab956d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 4 client.15426 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561ab91bad10 con 0x561ab9570aa0 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9570aa0 0x561ab956d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8535ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f854dd9b640 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9570aa0 0x561ab956d3d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=295a1ac78fc081e2 server_cookie=adaa02ccb29633f0 in_seq=0 out_seq=0 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8535ffb640 1 -- 172.21.15.44:0/1241478454 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x561ab91bad10 con 0x561ab9570aa0 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8535ffb640 10 client.15426 handle_command_reply: tid=0 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.956+0000 7f8557dd0740 1 client.15426 shutdown 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 msgr2=0x561ab950b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 0x561ab950b4d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8540007fd0 tx=0x7f85400c9f90 comp rx=0 tx=0).stop 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 shutdown_connections 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561ab9570aa0 0x561ab956d3d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab94ffe40 0x561ab950ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ab9504ed0 0x561ab956b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8535ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 --2- 172.21.15.44:0/1241478454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ab94ff500 0x561ab950b4d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 >> 172.21.15.44:0/1241478454 conn(0x561ab94fab70 msgr2=0x561ab951d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 shutdown_connections 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 1 -- 172.21.15.44:0/1241478454 wait complete. 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8557dd0740 20 client.15426 trim_cache size 0 max 16384 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24496, 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 110.370220628, 2024-09-13T17:26:35.965 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 24, 2024-09-13T17:26:35.966 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 92, 2024-09-13T17:26:35.966 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 90, 2024-09-13T17:26:35.966 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 152.092525795 2024-09-13T17:26:35.966 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f853c0a1090 msgr2=0x7f853c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f853c0a1090 0x7f853c0a3550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x561ab9507bd0 tx=0x7f854804e000 comp rx=0 tx=0).stop 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 msgr2=0x7f85501001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 0x7f85501001a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f854000c8c0 tx=0x7f854000cd80 comp rx=0 tx=0).stop 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 shutdown_connections 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f853c0a1090 0x7f853c0a3550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8550103160 0x7f85500fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8550105a90 0x7f85500fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 --2- 172.21.15.44:0/1160084336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8550100830 0x7f85501001a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 >> 172.21.15.44:0/1160084336 conn(0x7f85500fa430 msgr2=0x7f8550101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 shutdown_connections 2024-09-13T17:26:35.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:35.957+0000 7f8554fd4640 1 -- 172.21.15.44:0/1160084336 wait complete. 2024-09-13T17:26:35.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2837152549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2178012493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:35.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2338994996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:35.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3463210084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:35.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T17:26:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2837152549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2178012493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2338994996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3463210084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:36.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 -- 172.21.15.44:0/2842973697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548105af0 msgr2=0x7ff548107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2842973697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548105af0 0x7ff548107f00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff540009ca0 tx=0x7ff54002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 -- 172.21.15.44:0/2842973697 shutdown_connections 2024-09-13T17:26:36.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2842973697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548105af0 0x7ff548107f00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2842973697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5481031c0 0x7ff5481055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2842973697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff548100890 0x7ff548102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 -- 172.21.15.44:0/2842973697 >> 172.21.15.44:0/2842973697 conn(0x7ff5480fa430 msgr2=0x7ff5480fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 -- 172.21.15.44:0/2842973697 shutdown_connections 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.071+0000 7ff54efe2640 1 -- 172.21.15.44:0/2842973697 wait complete. 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 Processor -- start 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 -- start start 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 0x7ff548195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5481031c0 0x7ff5481962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff548105af0 0x7ff54819d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54819d800 con 0x7ff548100890 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54819d970 con 0x7ff548105af0 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54efe2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54819db30 con 0x7ff5481031c0 2024-09-13T17:26:36.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 0x7ff548195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 0x7ff548195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35082/0 (socket says 172.21.15.44:35082) 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 -- 172.21.15.44:0/2199995031 learned_addr learned my addr 172.21.15.44:0/2199995031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54d57f640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff548105af0 0x7ff54819d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 -- 172.21.15.44:0/2199995031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5481031c0 msgr2=0x7ff5481962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5481031c0 0x7ff5481962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 -- 172.21.15.44:0/2199995031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff548105af0 msgr2=0x7ff54819d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff548105af0 0x7ff54819d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.072+0000 7ff54cd7e640 1 -- 172.21.15.44:0/2199995031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540009900 con 0x7ff548100890 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.073+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 0x7ff548195d70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c970 tx=0x7ff53c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.073+0000 7ff545ffb640 1 -- 172.21.15.44:0/2199995031 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c007bb0 con 0x7ff548100890 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.073+0000 7ff545ffb640 1 -- 172.21.15.44:0/2199995031 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff53c007d10 con 0x7ff548100890 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.073+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54819de10 con 0x7ff548100890 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.073+0000 7ff545ffb640 1 -- 172.21.15.44:0/2199995031 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c004d70 con 0x7ff548100890 2024-09-13T17:26:36.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.073+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54819e1e0 con 0x7ff548100890 2024-09-13T17:26:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.075+0000 7ff545ffb640 1 -- 172.21.15.44:0/2199995031 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff53c005020 con 0x7ff548100890 2024-09-13T17:26:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.075+0000 7ff551ddc740 1 Processor -- start 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.075+0000 7ff551ddc740 1 -- start start 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.075+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a6fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a6fd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff551ddc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a6507c0 con 0x55793a6f5e40 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff551ddc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a65fb70 con 0x55793a6f5500 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff551ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a65d300 con 0x55793a6faed0 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a6fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff545ffb640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5380a1090 0x7ff5380a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a6fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35090/0 (socket says 172.21.15.44:35090) 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 -- 172.21.15.44:0/590358310 learned_addr learned my addr 172.21.15.44:0/590358310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff547fff640 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a6fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54d57f640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5380a1090 0x7ff5380a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff545ffb640 1 -- 172.21.15.44:0/2199995031 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7ff53c0ccad0 con 0x7ff548100890 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54d57f640 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 -- 172.21.15.44:0/590358310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 msgr2=0x55793a6fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a6fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 -- 172.21.15.44:0/590358310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 msgr2=0x55793a6f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 -- 172.21.15.44:0/590358310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793a5e54b0 con 0x55793a6f5e40 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a6fa900 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0998e0 tx=0x7ff53c099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac0d55e68ed65e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff54d57f640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5380a1090 0x7ff5380a3550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff540005cc0 tx=0x7ff540005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.076+0000 7ff532ffd640 1 -- 172.21.15.44:0/590358310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0046c0 con 0x55793a6f5e40 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff532ffd640 1 -- 172.21.15.44:0/590358310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff53c09a2e0 con 0x55793a6f5e40 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff532ffd640 1 -- 172.21.15.44:0/590358310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c096260 con 0x55793a6f5e40 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 -- 172.21.15.44:0/590358310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 msgr2=0x55793a6fa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a6fa900 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0998e0 tx=0x7ff53c099a70 comp rx=0 tx=0).stop 2024-09-13T17:26:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 -- 172.21.15.44:0/590358310 shutdown_connections 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a6fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a6fa900 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 --2- 172.21.15.44:0/590358310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 -- 172.21.15.44:0/590358310 >> 172.21.15.44:0/590358310 conn(0x55793a6f0b70 msgr2=0x55793a6f0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 -- 172.21.15.44:0/590358310 shutdown_connections 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 -- 172.21.15.44:0/590358310 wait complete. 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.077+0000 7ff551ddc740 1 -- start start 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 Processor -- start 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 0x55793a6ff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6faed0 0x55793a706640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a658bf0 con 0x55793a6f5500 2024-09-13T17:26:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a617250 con 0x55793a6faed0 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a616ef0 con 0x55793a6f5e40 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff551ddc740 10 client.0 _ll_register_callbacks cb 0x55793a649740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 0x55793a6ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 0x55793a6ff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35106/0 (socket says 172.21.15.44:35106) 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 -- 172.21.15.44:0/230020968 learned_addr learned my addr 172.21.15.44:0/230020968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54d57f640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6faed0 0x55793a706640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff547fff640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 -- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 msgr2=0x55793a6ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 -- 172.21.15.44:0/230020968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6faed0 msgr2=0x55793a706640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6faed0 0x55793a706640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff54cd7e640 1 -- 172.21.15.44:0/230020968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54819e1e0 con 0x55793a6f5500 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.078+0000 7ff547fff640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 0x55793a6ff110 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0058c0 tx=0x7ff53c09da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466f35a6c49b0479 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff527fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff527fff640 1 -- 172.21.15.44:0/230020968 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0b1070 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff527fff640 1 -- 172.21.15.44:0/230020968 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7ff53c0046c0 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55793a34c3c0 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff527fff640 1 -- 172.21.15.44:0/230020968 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53c0a1bd0 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 10 client.15438 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55793a657270 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff527fff640 1 -- 172.21.15.44:0/230020968 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55793a34c3c0 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff527fff640 1 -- 172.21.15.44:0/230020968 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff53c016030 con 0x55793a6f5500 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 10 client.15438 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 20 client.15438 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a70b170 0x55793a70f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 4 client.15438 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:36.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.079+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55793a35d6c0 con 0x55793a70b170 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.080+0000 7ff547fff640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a70b170 0x55793a70f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.080+0000 7ff527fff640 10 client.15438 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.080+0000 7ff547fff640 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a70b170 0x55793a70f2f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404790c7b3c738b3 server_cookie=1817e4c2b0f843bc in_seq=0 out_seq=0 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff527fff640 1 -- 172.21.15.44:0/230020968 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55793a35d6c0 con 0x55793a70b170 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff527fff640 10 client.15438 handle_command_reply: tid=0 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 client.15438 shutdown 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 msgr2=0x55793a6ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 0x55793a6ff110 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0058c0 tx=0x7ff53c09da80 comp rx=0 tx=0).stop 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 shutdown_connections 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a70b170 0x55793a70f2f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6faed0 0x55793a706640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff527fff640 0 client.15438 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:36.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 --2- 172.21.15.44:0/230020968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5500 0x55793a6ff110 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 >> 172.21.15.44:0/230020968 conn(0x55793a6f0b70 msgr2=0x55793a6f2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 shutdown_connections 2024-09-13T17:26:36.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 1 -- 172.21.15.44:0/230020968 wait complete. 2024-09-13T17:26:36.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.081+0000 7ff551ddc740 20 client.15438 trim_cache size 0 max 16384 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.096+0000 7ff551ddc740 1 Processor -- start 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.096+0000 7ff551ddc740 1 -- start start 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 0x55793a706560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff551ddc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a5dc460 con 0x55793a6faed0 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff551ddc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a34c3c0 con 0x55793a6f5500 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff551ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a61bd80 con 0x55793a6f5e40 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 0x55793a706560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 0x55793a706560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35110/0 (socket says 172.21.15.44:35110) 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 -- 172.21.15.44:0/1845175544 learned_addr learned my addr 172.21.15.44:0/1845175544 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff547fff640 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 -- 172.21.15.44:0/1845175544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 msgr2=0x55793a6ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 -- 172.21.15.44:0/1845175544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 msgr2=0x55793a6fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6fe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 -- 172.21.15.44:0/1845175544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793a5f95f0 con 0x55793a6faed0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff54d57f640 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 0x55793a706560 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5480696d0 tx=0x7ff54003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5c718e0cd52bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff5337fe640 1 -- 172.21.15.44:0/1845175544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540003af0 con 0x55793a6faed0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff5337fe640 1 -- 172.21.15.44:0/1845175544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff540030a50 con 0x55793a6faed0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.097+0000 7ff5337fe640 1 -- 172.21.15.44:0/1845175544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540004460 con 0x55793a6faed0 2024-09-13T17:26:36.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 -- 172.21.15.44:0/1845175544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 msgr2=0x55793a706560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 0x55793a706560 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5480696d0 tx=0x7ff54003f040 comp rx=0 tx=0).stop 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 -- 172.21.15.44:0/1845175544 shutdown_connections 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6f5e40 0x55793a6ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6faed0 0x55793a706560 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 --2- 172.21.15.44:0/1845175544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a6fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 -- 172.21.15.44:0/1845175544 >> 172.21.15.44:0/1845175544 conn(0x55793a6f0b70 msgr2=0x55793a6feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 -- 172.21.15.44:0/1845175544 shutdown_connections 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 -- 172.21.15.44:0/1845175544 wait complete. 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 -- start start 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.098+0000 7ff551ddc740 1 Processor -- start 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a703380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a7038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a760fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a7068e0 con 0x55793a6f5e40 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a706a50 con 0x55793a6f5500 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793a7614f0 con 0x55793a6faed0 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff551ddc740 10 client.0 _ll_register_callbacks cb 0x7ff53c0d1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a7038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a7038c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35116/0 (socket says 172.21.15.44:35116) 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 -- 172.21.15.44:0/4124828771 learned_addr learned my addr 172.21.15.44:0/4124828771 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff54d57f640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a760fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a703380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 -- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 msgr2=0x55793a760fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a760fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 -- 172.21.15.44:0/4124828771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 msgr2=0x55793a703380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:36.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a703380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 -- 172.21.15.44:0/4124828771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540042070 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff547fff640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a7038c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff5480690f0 tx=0x7ff53401af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e600cd4a654bf373 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.099+0000 7ff527fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff527fff640 1 -- 172.21.15.44:0/4124828771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534018050 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff527fff640 1 -- 172.21.15.44:0/4124828771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7ff534004230 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff53c09a2e0 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff527fff640 1 -- 172.21.15.44:0/4124828771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5340043f0 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 10 client.15444 fetch_fsmap learned FSMap version 24 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55793a61ed00 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff527fff640 1 -- 172.21.15.44:0/4124828771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff53c09a2e0 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff527fff640 1 -- 172.21.15.44:0/4124828771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff5340125c0 con 0x55793a6f5e40 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 10 client.15444 fetch_fsmap finished waiting for FSMap version 24 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T17:26:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 20 client.15444 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a7662f0 0x55793a762c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 4 client.15444 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.100+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55793a3b0d10 con 0x55793a7662f0 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a7662f0 0x55793a762c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff54cd7e640 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a7662f0 0x55793a762c60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f281f3589cfd68 server_cookie=63e941f93e312bb4 in_seq=0 out_seq=0 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff527fff640 10 client.15444 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff527fff640 1 -- 172.21.15.44:0/4124828771 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55793a3b0d10 con 0x55793a7662f0 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff527fff640 10 client.15444 handle_command_reply: tid=0 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 client.15444 shutdown 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 msgr2=0x55793a7038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a7038c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff5480690f0 tx=0x7ff53401af40 comp rx=0 tx=0).stop 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 shutdown_connections 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55793a7662f0 0x55793a762c60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55793a6faed0 0x55793a760fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55793a6f5e40 0x55793a7038c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 --2- 172.21.15.44:0/4124828771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55793a6f5500 0x55793a703380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff527fff640 0 client.15444 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.101+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 >> 172.21.15.44:0/4124828771 conn(0x55793a6f0b70 msgr2=0x55793a713310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 shutdown_connections 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff551ddc740 1 -- 172.21.15.44:0/4124828771 wait complete. 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff551ddc740 20 client.15444 trim_cache size 0 max 16384 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "ce020790-71f3-11ef-bceb-c7b262605968", 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T17:26:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24517, 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 134.376567034, 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 24, 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 92, 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 90, 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 147.271754867 2024-09-13T17:26:36.110 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5380a1090 msgr2=0x7ff5380a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5380a1090 0x7ff5380a3550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff540005cc0 tx=0x7ff540005c30 comp rx=0 tx=0).stop 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 msgr2=0x7ff548195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 0x7ff548195d70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c970 tx=0x7ff53c00ce30 comp rx=0 tx=0).stop 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 shutdown_connections 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7ff5380a1090 0x7ff5380a3550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5481031c0 0x7ff5481962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff548105af0 0x7ff54819d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 --2- 172.21.15.44:0/2199995031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff548100890 0x7ff548195d70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 >> 172.21.15.44:0/2199995031 conn(0x7ff5480fa430 msgr2=0x7ff5481012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 shutdown_connections 2024-09-13T17:26:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.102+0000 7ff54efe2640 1 -- 172.21.15.44:0/2199995031 wait complete. 2024-09-13T17:26:36.114 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T17:26:36.123 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T17:26:36.123 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T17:26:36.124 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T17:26:36.124 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-13T17:26:36.124 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T17:26:36.125 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:26:36.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-13T17:26:36.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 -- 172.21.15.44:0/2522638602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 msgr2=0x7f0f980ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/2522638602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f980ff060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f8802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 -- 172.21.15.44:0/2522638602 shutdown_connections 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/2522638602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f98100810 0x7f0f981070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/2522638602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f980ffe50 0x7f0f981002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/2522638602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f980ff060 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 -- 172.21.15.44:0/2522638602 >> 172.21.15.44:0/2522638602 conn(0x7f0f980fa3c0 msgr2=0x7f0f980fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 -- 172.21.15.44:0/2522638602 shutdown_connections 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.227+0000 7f0f9d35a640 1 -- 172.21.15.44:0/2522638602 wait complete. 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 Processor -- start 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 -- start start 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f98102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f980ffe50 0x7f0f98101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f98100810 0x7f0f98101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98101f00 con 0x7f0f980fec60 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98102070 con 0x7f0f980ffe50 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f9d35a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819fdc0 con 0x7f0f98100810 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f98102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f98102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35136/0 (socket says 172.21.15.44:35136) 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f96ffd640 1 -- 172.21.15.44:0/3389962560 learned_addr learned my addr 172.21.15.44:0/3389962560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f967fc640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f980ffe50 0x7f0f98101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f977fe640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f98100810 0x7f0f98101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f96ffd640 1 -- 172.21.15.44:0/3389962560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f98100810 msgr2=0x7f0f98101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.228+0000 7f0f96ffd640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f98100810 0x7f0f98101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f96ffd640 1 -- 172.21.15.44:0/3389962560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f980ffe50 msgr2=0x7f0f98101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f96ffd640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f980ffe50 0x7f0f98101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f96ffd640 1 -- 172.21.15.44:0/3389962560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009560 con 0x7f0f980fec60 2024-09-13T17:26:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f96ffd640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f98102ba0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005530 tx=0x7f0f88005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88004bd0 con 0x7f0f980fec60 2024-09-13T17:26:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f88004d30 con 0x7f0f980fec60 2024-09-13T17:26:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9819ffc0 con 0x7f0f980fec60 2024-09-13T17:26:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8803c8d0 con 0x7f0f980fec60 2024-09-13T17:26:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.229+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f981a03e0 con 0x7f0f980fec60 2024-09-13T17:26:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.230+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9810a420 con 0x7f0f980fec60 2024-09-13T17:26:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.231+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0f88003bf0 con 0x7f0f980fec60 2024-09-13T17:26:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.232+0000 7f0f8ffff640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0f840a1110 0x7f0f840a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.232+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7f0f880ede60 con 0x7f0f980fec60 2024-09-13T17:26:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.232+0000 7f0f967fc640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0f840a1110 0x7f0f840a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.232+0000 7f0f967fc640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0f840a1110 0x7f0f840a35d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f98102910 tx=0x7f0f80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:36.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.235+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0f880b72b0 con 0x7f0f980fec60 2024-09-13T17:26:36.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:36.455+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0f981a05d0 con 0x7f0f980fec60 2024-09-13T17:26:37.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.183+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 19) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f0f880b7680 con 0x7f0f980fec60 2024-09-13T17:26:37.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.184+0000 7f0f8ffff640 1 -- 172.21.15.44:0/3389962560 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0f880bad40 con 0x7f0f980fec60 2024-09-13T17:26:37.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0f840a1110 msgr2=0x7f0f840a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:37.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0f840a1110 0x7f0f840a35d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f98102910 tx=0x7f0f80005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:37.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 msgr2=0x7f0f98102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:37.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f98102ba0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005530 tx=0x7f0f88005560 comp rx=0 tx=0).stop 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 shutdown_connections 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7f0f840a1110 0x7f0f840a35d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f98100810 0x7f0f98101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f980ffe50 0x7f0f98101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 --2- 172.21.15.44:0/3389962560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f980fec60 0x7f0f98102ba0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 >> 172.21.15.44:0/3389962560 conn(0x7f0f980fa3c0 msgr2=0x7f0f980685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 shutdown_connections 2024-09-13T17:26:37.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:37.187+0000 7f0f9d35a640 1 -- 172.21.15.44:0/3389962560 wait complete. 2024-09-13T17:26:37.201 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:26:37.212 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:26:37.212 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-13T17:26:37.212 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-13T17:26:37.212 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:26:37.213 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:37.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:37.213 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T17:26:37.231 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T17:26:37.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:37.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T17:26:37.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:37.299 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-09-13T17:26:37.406 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:26:37.407 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:26:37.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-13T17:26:37.431 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ignoring --setuser ceph since I am not root 2024-09-13T17:26:37.431 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ignoring --setgroup ceph since I am not root 2024-09-13T17:26:37.432 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:37.334+0000 7fb71e188040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:26:37.432 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:37.398+0000 7fb71e188040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:26:37.432 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-mon[31175]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:37.432 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3389962560' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-13T17:26:37.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-mon[34502]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:37.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3389962560' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-13T17:26:37.447 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: ignoring --setuser ceph since I am not root 2024-09-13T17:26:37.447 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: ignoring --setgroup ceph since I am not root 2024-09-13T17:26:37.448 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-mgr[35908]: -- 172.21.15.70:0/1774625622 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55abd9aeed00 con 0x55abd9ac3400 2024-09-13T17:26:37.448 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:37.324+0000 7fd7313cc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:26:37.448 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:37.379+0000 7fd7313cc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:26:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-mon[35055]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-13T17:26:37.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3389962560' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-13T17:26:37.524 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: ignoring --setuser ceph since I am not root 2024-09-13T17:26:37.524 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: ignoring --setgroup ceph since I am not root 2024-09-13T17:26:37.524 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-mgr[36981]: -- 172.21.15.100:0/2713597445 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ffe8430d00 con 0x55ffe8404800 2024-09-13T17:26:37.524 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:37.324+0000 7fb6e90a7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:26:37.525 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:37.381+0000 7fb6e90a7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:26:37.626 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:26:37.747 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:37.562+0000 7fb71e188040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:26:37.747 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:37.643+0000 7fb71e188040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:26:37.879 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:37.527+0000 7fd7313cc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:26:37.880 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:37.593+0000 7fd7313cc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:26:37.880 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:37.734+0000 7fd7313cc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:26:37.880 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:37 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:37.877+0000 7fd7313cc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:26:37.884 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:37.529+0000 7fb6e90a7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:26:37.884 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:37.596+0000 7fb6e90a7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:26:37.884 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:37.739+0000 7fb6e90a7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:26:37.885 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:37.883+0000 7fb6e90a7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:26:38.047 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:37.806+0000 7fb71e188040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:26:38.047 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:37 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:37.955+0000 7fb71e188040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:26:38.348 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:38 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:38.283+0000 7fb71e188040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:26:38.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:38 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3389962560' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-13T17:26:38.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:38 smithi044 ceph-mon[31175]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-13T17:26:38.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:38 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3389962560' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-13T17:26:38.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:38 smithi070 ceph-mon[34502]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-13T17:26:38.429 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:38 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:38.184+0000 7fd7313cc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:26:38.429 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:38 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:38.245+0000 7fd7313cc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:26:38.429 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:38 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:38.426+0000 7fd7313cc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:26:38.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:38 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3389962560' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-13T17:26:38.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:38 smithi100 ceph-mon[35055]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-13T17:26:38.431 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:38.187+0000 7fb6e90a7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:26:38.431 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:38.247+0000 7fb6e90a7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:26:38.431 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:38.429+0000 7fb6e90a7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:26:38.606 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:38 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:38.346+0000 7fb71e188040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:26:38.606 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:38 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:38.540+0000 7fb71e188040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:26:39.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 -- 172.21.15.44:0/1217240709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 msgr2=0x7feec810c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:39.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 --2- 172.21.15.44:0/1217240709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec810c800 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feec0009ca0 tx=0x7feec002a950 comp rx=0 tx=0).stop 2024-09-13T17:26:39.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 -- 172.21.15.44:0/1217240709 shutdown_connections 2024-09-13T17:26:39.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 --2- 172.21.15.44:0/1217240709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec810c800 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 --2- 172.21.15.44:0/1217240709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8101850 0x7feec8109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 --2- 172.21.15.44:0/1217240709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8100f30 0x7feec8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 -- 172.21.15.44:0/1217240709 >> 172.21.15.44:0/1217240709 conn(0x7feec80fa9b0 msgr2=0x7feec80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 -- 172.21.15.44:0/1217240709 shutdown_connections 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 -- 172.21.15.44:0/1217240709 wait complete. 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 Processor -- start 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.396+0000 7feece30f640 1 -- start start 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feece30f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8100f30 0x7feec81961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feece30f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8101850 0x7feec81966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feece30f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feece30f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec819b0a0 con 0x7feec810a360 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feece30f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec819b210 con 0x7feec8100f30 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feece30f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec819b380 con 0x7feec8101850 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35178/0 (socket says 172.21.15.44:35178) 2024-09-13T17:26:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 -- 172.21.15.44:0/1243873542 learned_addr learned my addr 172.21.15.44:0/1243873542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feec77fe640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8101850 0x7feec81966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feec7fff640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8100f30 0x7feec81961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 -- 172.21.15.44:0/1243873542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8101850 msgr2=0x7feec81966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8101850 0x7feec81966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 -- 172.21.15.44:0/1243873542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8100f30 msgr2=0x7feec81961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8100f30 0x7feec81961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feecc8ac640 1 -- 172.21.15.44:0/1243873542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0009900 con 0x7feec810a360 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feec7fff640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8100f30 0x7feec81961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.397+0000 7feec77fe640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8101850 0x7feec81966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.398+0000 7feecc8ac640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec819aaa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feec0005e30 tx=0x7feec0003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.398+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec002fd50 con 0x7feec810a360 2024-09-13T17:26:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.398+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feec0003e60 con 0x7feec810a360 2024-09-13T17:26:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.398+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec819b600 con 0x7feec810a360 2024-09-13T17:26:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.398+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00045a0 con 0x7feec810a360 2024-09-13T17:26:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.398+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec819b920 con 0x7feec810a360 2024-09-13T17:26:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.399+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec80687e0 con 0x7feec810a360 2024-09-13T17:26:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.400+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 135890+0+0 (secure 0 0 0) 0x7feec002e050 con 0x7feec810a360 2024-09-13T17:26:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.401+0000 7feec57fa640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 0x7feeb00a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.401+0000 7feec7fff640 1 -- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 msgr2=0x7feeb00a3610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/322597156 2024-09-13T17:26:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.401+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7feec00ed810 con 0x7feec810a360 2024-09-13T17:26:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.401+0000 7feec7fff640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 0x7feeb00a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:26:39.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.405+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feec00b6d10 con 0x7feec810a360 2024-09-13T17:26:39.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.602+0000 7feec7fff640 1 -- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 msgr2=0x7feeb00a3610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/322597156 2024-09-13T17:26:39.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.602+0000 7feec7fff640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 0x7feeb00a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:26:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.617+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feec8101310 con 0x7feec810a360 2024-09-13T17:26:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.617+0000 7feec57fa640 1 -- 172.21.15.44:0/1243873542 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7feec00ba330 con 0x7feec810a360 2024-09-13T17:26:39.621 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:26:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 msgr2=0x7feeb00a3610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:26:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 0x7feeb00a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 msgr2=0x7feec819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec819aaa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feec0005e30 tx=0x7feec0003c70 comp rx=0 tx=0).stop 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 shutdown_connections 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7feeb00a1150 0x7feeb00a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8101850 0x7feec81966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec810a360 0x7feec819aaa0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 --2- 172.21.15.44:0/1243873542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feec8100f30 0x7feec81961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 >> 172.21.15.44:0/1243873542 conn(0x7feec80fa9b0 msgr2=0x7feec80fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 shutdown_connections 2024-09-13T17:26:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:39.619+0000 7feece30f640 1 -- 172.21.15.44:0/1243873542 wait complete. 2024-09-13T17:26:39.670 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:39 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:39.288+0000 7fd7313cc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:26:39.670 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:39 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:39.354+0000 7fd7313cc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:26:39.670 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:39.292+0000 7fb6e90a7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:26:39.670 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:39.358+0000 7fb6e90a7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:26:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:39 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1243873542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T17:26:39.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:39.491+0000 7fb71e188040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:26:39.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:39.562+0000 7fb71e188040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:26:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:39 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1243873542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T17:26:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:39 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1243873542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:26:40.291 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:26:40.292 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:26:40.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.292 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T17:26:40.319 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.063+0000 7fd7313cc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:26:40.320 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.176+0000 7fd7313cc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:26:40.320 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.065+0000 7fb6e90a7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:26:40.320 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.178+0000 7fb6e90a7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:26:40.348 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:26:40.348 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:26:40.348 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:26:40.349 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42169sec preferred_lft 42169sec 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T17:26:40.350 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:26:40.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.350 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:26:40.351 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:26:40.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:26:40.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-09-13T17:26:40.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-09-13T17:26:40.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:26:40.351 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:26:40.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.456 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:26:40.483 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T17:26:40.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.485 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T17:26:40.542 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T17:26:40.542 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:26:40.542 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-13T17:26:40.542 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-13T17:26:40.542 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:26:40.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.542 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:26:40.543 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:26:40.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T17:26:40.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T17:26:40.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T17:26:40.543 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:26:40.592 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.318+0000 7fd7313cc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:26:40.592 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.379+0000 7fd7313cc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:26:40.592 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.524+0000 7fd7313cc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:26:40.592 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.590+0000 7fd7313cc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:26:40.593 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.319+0000 7fb6e90a7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:26:40.593 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.379+0000 7fb6e90a7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:26:40.593 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.524+0000 7fb6e90a7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:26:40.593 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.592+0000 7fb6e90a7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:26:40.606 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.327+0000 7fb71e188040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:26:40.606 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.454+0000 7fb71e188040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:26:40.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.633 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T17:26:40.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.690 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-09-13T17:26:40.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.747 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:26:40.747 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:26:40.747 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:26:40.747 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T17:26:40.748 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:26:40.827 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T17:26:40.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.827 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:26:40.827 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:26:40.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T17:26:40.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:26:40.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T17:26:40.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T17:26:40.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T17:26:40.828 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:26:40.856 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.606+0000 7fb71e188040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:26:40.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.672+0000 7fb71e188040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:26:40.857 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.829+0000 7fb71e188040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:26:40.869 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.650+0000 7fd7313cc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:26:40.869 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.803+0000 7fd7313cc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:26:40.872 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.651+0000 7fb6e90a7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:26:40.873 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.806+0000 7fb6e90a7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:26:40.873 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.871+0000 7fb6e90a7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:26:40.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:40.938 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2024-09-13T17:26:40.938 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T17:26:40.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T17:26:40.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:26:40.938 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T17:26:40.978 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-13T17:26:40.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:26:40.978 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:41.034 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:26:41.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:26:41.034 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:41.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-09-13T17:26:41.133 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.868+0000 7fd7313cc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:26:41.133 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:40.933+0000 7fd7313cc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:26:41.133 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:41.014+0000 7fd7313cc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:26:41.133 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:41.073+0000 7fd7313cc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:26:41.138 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:40.937+0000 7fb6e90a7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:26:41.138 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:41.019+0000 7fb6e90a7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:26:41.138 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:41.077+0000 7fb6e90a7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:26:41.138 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:41.136+0000 7fb6e90a7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:26:41.139 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.906+0000 7fb71e188040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:26:41.139 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:40 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:40.972+0000 7fb71e188040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:26:41.139 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:41.138+0000 7fb71e188040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:26:41.157 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:26:41.157 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:26:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:26:41.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:26:41.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:41.224 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-09-13T17:26:41.280 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:26:41.280 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/b119cd80-e9f3-469f-ae00-550e1071ac52 --client_fs=cephfs) 2024-09-13T17:26:41.323 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-09-13T17:26:41.349 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:26:41.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:26:41.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:2024-09-13T17:26:41.408+0000 7f9a77e8d480 -1 init, newargv = 0x55580b39a340 newargc=15 2024-09-13T17:26:41.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[84318]: starting ceph client 2024-09-13T17:26:41.413 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:41.207+0000 7fb71e188040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:26:41.413 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:41.277+0000 7fb71e188040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:26:41.413 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:41.367+0000 7fb71e188040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:26:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:26:41.417 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:26:41.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:41.418 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-09-13T17:26:41.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[84318]: starting fuse 2024-09-13T17:26:41.440 INFO:teuthology.orchestra.run.smithi044.stdout:120 2024-09-13T17:26:41.440 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:26:41.441 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> 2024-09-13T17:26:41.442 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.0")) 2024-09-13T17:26:41.443 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-09-13T17:26:41.446 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:41.131+0000 7fd7313cc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:26:41.559 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.0.84318.asok 2024-09-13T17:26:41.743 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:41.437+0000 7fb71e188040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:26:41.743 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:41.500+0000 7fb71e188040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:26:41.744 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84318.asok 2024-09-13T17:26:41.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:41.745 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84318.asok status 2024-09-13T17:26:41.882 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "0", 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "84318", 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/volumes/qa/sv_0/b119cd80-e9f3-469f-ae00-550e1071ac52" 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-09-13T17:26:41.883 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 15480, 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 15480 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1920488869 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-09-13T17:26:41.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1920488869 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.15480 192.168.0.1:0/1920488869", 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/1920488869", 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 5, 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 24, 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 92, 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-09-13T17:26:41.885 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:41.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:41.891 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-09-13T17:26:41.891 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:41.908 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-09-13T17:26:41.908 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:41.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:26:41.908 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:41.980 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T17:26:41.994 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T17:26:41.995 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:26:41.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-13T17:26:42.043 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: Standby manager daemon y restarted 2024-09-13T17:26:42.043 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: Standby manager daemon y started 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: Standby manager daemon z restarted 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: Standby manager daemon z started 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T17:26:42.044 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:41 smithi044 ceph-mon[31175]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 -- 172.21.15.44:0/1174703317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 msgr2=0x7fcd8c100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 --2- 172.21.15.44:0/1174703317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c100130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009900 tx=0x7fcd7c02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 -- 172.21.15.44:0/1174703317 shutdown_connections 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 --2- 172.21.15.44:0/1174703317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd8c100f20 0x7fcd8c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 --2- 172.21.15.44:0/1174703317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1018e0 0x7fcd8c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 --2- 172.21.15.44:0/1174703317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c100130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 -- 172.21.15.44:0/1174703317 >> 172.21.15.44:0/1174703317 conn(0x7fcd8c0fb4d0 msgr2=0x7fcd8c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 -- 172.21.15.44:0/1174703317 shutdown_connections 2024-09-13T17:26:42.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.124+0000 7fcd91fec640 1 -- 172.21.15.44:0/1174703317 wait complete. 2024-09-13T17:26:42.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.125+0000 7fcd91fec640 1 Processor -- start 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.125+0000 7fcd91fec640 1 -- start start 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c100f20 0x7fcd8c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd8c1018e0 0x7fcd8c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19d750 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19d8c0 con 0x7fcd8c100f20 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19da80 con 0x7fcd8c1018e0 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40070/0 (socket says 172.21.15.44:40070) 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 -- 172.21.15.44:0/2176905706 learned_addr learned my addr 172.21.15.44:0/2176905706 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8bfff640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd8c1018e0 0x7fcd8c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:42.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd82ffd640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c100f20 0x7fcd8c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 -- 172.21.15.44:0/2176905706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd8c1018e0 msgr2=0x7fcd8c19d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd8c1018e0 0x7fcd8c19d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 -- 172.21.15.44:0/2176905706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c100f20 msgr2=0x7fcd8c196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c100f20 0x7fcd8c196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 -- 172.21.15.44:0/2176905706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84009840 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd8b7fe640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c195cc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005ab0 tx=0x7fcd7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c031d70 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd7c005490 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c009560 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.126+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0038e0 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.127+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c19e070 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.128+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c107300 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.129+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fcd7c003a40 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.130+0000 7fcd897fa640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 0x7fcd780a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:42.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.130+0000 7fcd82ffd640 1 -- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 msgr2=0x7fcd780a3620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/322597156 2024-09-13T17:26:42.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.130+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9057+0+0 (secure 0 0 0) 0x7fcd7c0edb50 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.130+0000 7fcd82ffd640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 0x7fcd780a3620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:26:42.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.134+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcd7c0b6eb0 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.197 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y[35904]: 2024-09-13T17:26:41.910+0000 7fd7313cc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:26:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: Standby manager daemon y restarted 2024-09-13T17:26:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: Standby manager daemon y started 2024-09-13T17:26:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T17:26:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:26:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: Standby manager daemon z restarted 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: Standby manager daemon z started 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T17:26:42.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:41 smithi070 ceph-mon[34502]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:26:42.274 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z[36941]: 2024-09-13T17:26:41.916+0000 7fb6e90a7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:26:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: Standby manager daemon y restarted 2024-09-13T17:26:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: Standby manager daemon y started 2024-09-13T17:26:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T17:26:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:26:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.70:0/1331558576' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: Standby manager daemon z restarted 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: Standby manager daemon z started 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T17:26:42.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:41 smithi100 ceph-mon[35055]: from='mgr.? 172.21.15.100:0/3012413019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T17:26:42.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.331+0000 7fcd82ffd640 1 -- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 msgr2=0x7fcd780a3620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/322597156 2024-09-13T17:26:42.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.331+0000 7fcd82ffd640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 0x7fcd780a3620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:26:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.346+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fcd8c10b480 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.356+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 21) v1 ==== 135558+0+0 (secure 0 0 0) 0x7fcd7c003cf0 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.357+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 msgr2=0x7fcd780a3620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:26:42.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.357+0000 7fcd897fa640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 0x7fcd780a3620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.342+0000 7fb71e188040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb686c17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb686c17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb686c17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb686c17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb686c17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb68941c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb68941c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb68941c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.607 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb68941c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.608 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:42.571+0000 7fb68941c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:42.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.983+0000 7fcd897fa640 1 -- 172.21.15.44:0/2176905706 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcd7c0abd20 con 0x7fcd8c0ffd30 2024-09-13T17:26:42.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 msgr2=0x7fcd8c195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:42.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c195cc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005ab0 tx=0x7fcd7c0042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 shutdown_connections 2024-09-13T17:26:42.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:6800/322597156,v1:172.21.15.44:6801/322597156] conn(0x7fcd780a1160 0x7fcd780a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd8c1018e0 0x7fcd8c19d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c100f20 0x7fcd8c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 --2- 172.21.15.44:0/2176905706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ffd30 0x7fcd8c195cc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 >> 172.21.15.44:0/2176905706 conn(0x7fcd8c0fb4d0 msgr2=0x7fcd8c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 shutdown_connections 2024-09-13T17:26:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:42.986+0000 7fcd91fec640 1 -- 172.21.15.44:0/2176905706 wait complete. 2024-09-13T17:26:43.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-13T17:26:43.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-13T17:26:43.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: Active manager daemon x restarted 2024-09-13T17:26:43.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: Activating manager daemon x 2024-09-13T17:26:43.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2176905706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-13T17:26:43.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T17:26:43.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: mgrmap e21: x(active, starting, since 0.0117061s), standbys: y, z 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:26:43.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:26:43.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: Manager daemon x is now available 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:26:43.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:42 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 -- 172.21.15.44:0/3588690236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000fefa0 msgr2=0x7f35000ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 --2- 172.21.15.44:0/3588690236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000fefa0 0x7f35000ff400 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009900 tx=0x7f34f402a480 comp rx=0 tx=0).stop 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 -- 172.21.15.44:0/3588690236 shutdown_connections 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 --2- 172.21.15.44:0/3588690236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000ff940 0x7f35000ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 --2- 172.21.15.44:0/3588690236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000fefa0 0x7f35000ff400 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 --2- 172.21.15.44:0/3588690236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fe660 0x7f35000fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 -- 172.21.15.44:0/3588690236 >> 172.21.15.44:0/3588690236 conn(0x7f35000fa3d0 msgr2=0x7f35000fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 -- 172.21.15.44:0/3588690236 shutdown_connections 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.157+0000 7f3506333640 1 -- 172.21.15.44:0/3588690236 wait complete. 2024-09-13T17:26:43.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 Processor -- start 2024-09-13T17:26:43.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 -- start start 2024-09-13T17:26:43.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 0x7f3500197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fefa0 0x7f3500198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000ff940 0x7f350019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350019f9b0 con 0x7f35000fefa0 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350019fb20 con 0x7f35000ff940 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f3506333640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350019fce0 con 0x7f35000fe660 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f34ff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fefa0 0x7f3500198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f34fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 0x7f3500197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f34ff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fefa0 0x7f3500198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40168/0 (socket says 172.21.15.44:40168) 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f34fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 0x7f3500197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:46396/0 (socket says 172.21.15.44:46396) 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.158+0000 7f34ff7fe640 1 -- 172.21.15.44:0/1592815627 learned_addr learned my addr 172.21.15.44:0/1592815627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000ff940 0x7f350019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 -- 172.21.15.44:0/1592815627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 msgr2=0x7f3500197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 0x7f3500197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 -- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fefa0 msgr2=0x7f3500198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fefa0 0x7f3500198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 -- 172.21.15.44:0/1592815627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8009840 con 0x7f35000ff940 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f34fffff640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 0x7f3500197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f35048d0640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000ff940 0x7f350019f470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ec50 tx=0x7f34f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8003b70 con 0x7f35000ff940 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34f800ce70 con 0x7f35000ff940 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f4009560 con 0x7f35000ff940 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8010690 con 0x7f35000ff940 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.159+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35001a0260 con 0x7f35000ff940 2024-09-13T17:26:43.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.160+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35000ff400 con 0x7f35000ff940 2024-09-13T17:26:43.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.162+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 135558+0+0 (secure 0 0 0) 0x7f34f8004b40 con 0x7f35000ff940 2024-09-13T17:26:43.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.162+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f34f8014070 con 0x7f35000ff940 2024-09-13T17:26:43.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.166+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f34f8096600 con 0x7f35000ff940 2024-09-13T17:26:43.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-13T17:26:43.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: Active manager daemon x restarted 2024-09-13T17:26:43.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: Activating manager daemon x 2024-09-13T17:26:43.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2176905706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-13T17:26:43.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: mgrmap e21: x(active, starting, since 0.0117061s), standbys: y, z 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:26:43.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:26:43.199 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: Manager daemon x is now available 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:26:43.200 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:42 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:26:43.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: Active manager daemon x restarted 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: Activating manager daemon x 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2176905706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: mgrmap e21: x(active, starting, since 0.0117061s), standbys: y, z 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T17:26:43.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T17:26:43.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T17:26:43.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: Manager daemon x is now available 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:26:43.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:42 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T17:26:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.361+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f34f80960b0 con 0x7f35000ff940 2024-09-13T17:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.361+0000 7f34fd7fa640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34e80a9330 0x7f34e80ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.366+0000 7f34fffff640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34e80a9330 0x7f34e80ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.368+0000 7f34fffff640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34e80a9330 0x7f34e80ab720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f34f0005d00 tx=0x7f34f000b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:43.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.401+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f350002c5c0 con 0x7f34e80a9330 2024-09-13T17:26:43.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.415+0000 7f34fd7fa640 1 -- 172.21.15.44:0/1592815627 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f350002c5c0 con 0x7f34e80a9330 2024-09-13T17:26:43.416 INFO:teuthology.orchestra.run.smithi044.stdout:{} 2024-09-13T17:26:43.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34e80a9330 msgr2=0x7f34e80ab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34e80a9330 0x7f34e80ab720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f34f0005d00 tx=0x7f34f000b410 comp rx=0 tx=0).stop 2024-09-13T17:26:43.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000ff940 msgr2=0x7f350019f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000ff940 0x7f350019f470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ec50 tx=0x7f34f800c6a0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 shutdown_connections 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34e80a9330 0x7f34e80ab720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35000ff940 0x7f350019f470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35000fefa0 0x7f3500198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 --2- 172.21.15.44:0/1592815627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35000fe660 0x7f3500197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 >> 172.21.15.44:0/1592815627 conn(0x7f35000fa3d0 msgr2=0x7f35000fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 shutdown_connections 2024-09-13T17:26:43.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.417+0000 7f3506333640 1 -- 172.21.15.44:0/1592815627 wait complete. 2024-09-13T17:26:43.429 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-13T17:26:43.451 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-13T17:26:43.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:43.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 -- 172.21.15.44:0/2926872005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 msgr2=0x7fd814107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/2926872005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd814107ef0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009900 tx=0x7fd80c02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:43.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 -- 172.21.15.44:0/2926872005 shutdown_connections 2024-09-13T17:26:43.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/2926872005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd814107ef0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/2926872005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814103150 0x7fd814105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/2926872005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814100820 0x7fd814102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 -- 172.21.15.44:0/2926872005 >> 172.21.15.44:0/2926872005 conn(0x7fd8140fa440 msgr2=0x7fd8140fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:43.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 -- 172.21.15.44:0/2926872005 shutdown_connections 2024-09-13T17:26:43.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.611+0000 7fd81bf8a640 1 -- 172.21.15.44:0/2926872005 wait complete. 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 Processor -- start 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 -- start start 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814100820 0x7fd814195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814103150 0x7fd814196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd81419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419d810 con 0x7fd814105a80 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419d980 con 0x7fd814100820 2024-09-13T17:26:43.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81bf8a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419db40 con 0x7fd814103150 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd81419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd81419d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40186/0 (socket says 172.21.15.44:40186) 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 -- 172.21.15.44:0/584035558 learned_addr learned my addr 172.21.15.44:0/584035558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd819d26640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814100820 0x7fd814195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd819525640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814103150 0x7fd814196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 -- 172.21.15.44:0/584035558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814103150 msgr2=0x7fd814196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814103150 0x7fd814196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 -- 172.21.15.44:0/584035558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814100820 msgr2=0x7fd814195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814100820 0x7fd814195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd81a527640 1 -- 172.21.15.44:0/584035558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c009560 con 0x7fd814105a80 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd819d26640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814100820 0x7fd814195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.612+0000 7fd819525640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814103150 0x7fd814196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:43.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.613+0000 7fd81a527640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd81419d2a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02aa00 tx=0x7fd80c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:43.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.613+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c031b60 con 0x7fd814105a80 2024-09-13T17:26:43.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.613+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd80c0044c0 con 0x7fd814105a80 2024-09-13T17:26:43.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.613+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81419ddc0 con 0x7fd814105a80 2024-09-13T17:26:43.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.613+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0039b0 con 0x7fd814105a80 2024-09-13T17:26:43.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.613+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81419e200 con 0x7fd814105a80 2024-09-13T17:26:43.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.614+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8140fbf30 con 0x7fd814105a80 2024-09-13T17:26:43.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.615+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fd80c031cc0 con 0x7fd814105a80 2024-09-13T17:26:43.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.616+0000 7fd812ffd640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7fc0a0f10 0x7fd7fc0a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.616+0000 7fd819d26640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7fc0a0f10 0x7fd7fc0a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.616+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd80c0edb50 con 0x7fd814105a80 2024-09-13T17:26:43.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.616+0000 7fd819d26640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7fc0a0f10 0x7fd7fc0a33d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd814197200 tx=0x7fd804009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:43.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.620+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd80c0b71b0 con 0x7fd814105a80 2024-09-13T17:26:43.709 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:43.406+0000 7fb683bd1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:43.710 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:43.406+0000 7fb683bd1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:43.710 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:43.406+0000 7fb683bd1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:43.710 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:43.406+0000 7fb683bd1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:43.710 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:43.406+0000 7fb683bd1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:26:43.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.794+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd81402c480 con 0x7fd814105a80 2024-09-13T17:26:43.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.796+0000 7fd812ffd640 1 -- 172.21.15.44:0/584035558 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24868 (secure 0 0 0) 0x7fd80c040090 con 0x7fd814105a80 2024-09-13T17:26:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:43.797 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":93,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:26:42.344870+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:43.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7fc0a0f10 msgr2=0x7fd7fc0a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7fc0a0f10 0x7fd7fc0a33d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd814197200 tx=0x7fd804009500 comp rx=0 tx=0).stop 2024-09-13T17:26:43.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 msgr2=0x7fd81419d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd81419d2a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02aa00 tx=0x7fd80c0040c0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 shutdown_connections 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7fc0a0f10 0x7fd7fc0a33d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd814103150 0x7fd814196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd814105a80 0x7fd81419d2a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 --2- 172.21.15.44:0/584035558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd814100820 0x7fd814195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 >> 172.21.15.44:0/584035558 conn(0x7fd8140fa440 msgr2=0x7fd814101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 shutdown_connections 2024-09-13T17:26:43.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.798+0000 7fd81bf8a640 1 -- 172.21.15.44:0/584035558 wait complete. 2024-09-13T17:26:43.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:43.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 -- 172.21.15.44:0/833350709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40ffe50 msgr2=0x7f1ad41002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 --2- 172.21.15.44:0/833350709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40ffe50 0x7f1ad41002d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009900 tx=0x7f1ac802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 -- 172.21.15.44:0/833350709 shutdown_connections 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 --2- 172.21.15.44:0/833350709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 0x7f1ad41070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 --2- 172.21.15.44:0/833350709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40ffe50 0x7f1ad41002d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 --2- 172.21.15.44:0/833350709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad40fec60 0x7f1ad40ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 -- 172.21.15.44:0/833350709 >> 172.21.15.44:0/833350709 conn(0x7f1ad40fa3e0 msgr2=0x7f1ad40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 -- 172.21.15.44:0/833350709 shutdown_connections 2024-09-13T17:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.918+0000 7f1ad8868640 1 -- 172.21.15.44:0/833350709 wait complete. 2024-09-13T17:26:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 Processor -- start 2024-09-13T17:26:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 -- start start 2024-09-13T17:26:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40fec60 0x7f1ad4068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad40ffe50 0x7f1ad419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 0x7f1ad419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419d810 con 0x7f1ad40fec60 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419d980 con 0x7f1ad4100810 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad8868640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419db40 con 0x7f1ad40ffe50 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40fec60 0x7f1ad4068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40fec60 0x7f1ad4068110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40204/0 (socket says 172.21.15.44:40204) 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad259c640 1 -- 172.21.15.44:0/3935906155 learned_addr learned my addr 172.21.15.44:0/3935906155 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad1d9b640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad40ffe50 0x7f1ad419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.919+0000 7f1ad2d9d640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 0x7f1ad419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad2d9d640 1 -- 172.21.15.44:0/3935906155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad40ffe50 msgr2=0x7f1ad419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad2d9d640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad40ffe50 0x7f1ad419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad2d9d640 1 -- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40fec60 msgr2=0x7f1ad4068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad2d9d640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40fec60 0x7f1ad4068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad2d9d640 1 -- 172.21.15.44:0/3935906155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8009560 con 0x7f1ad4100810 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad2d9d640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 0x7f1ad419d030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009fd0 tx=0x7f1acc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00ce20 con 0x7f1ad4100810 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1acc005020 con 0x7f1ad4100810 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad419ddc0 con 0x7f1ad4100810 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc010690 con 0x7f1ad4100810 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.920+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad419e1f0 con 0x7f1ad4100810 2024-09-13T17:26:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.921+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad41061e0 con 0x7f1ad4100810 2024-09-13T17:26:43.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.923+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f1acc005190 con 0x7f1ad4100810 2024-09-13T17:26:43.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.924+0000 7f1ac37fe640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1abc0a0e90 0x7f1abc0a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:43.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.924+0000 7f1ad259c640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1abc0a0e90 0x7f1abc0a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:43.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.924+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1acc014070 con 0x7f1ad4100810 2024-09-13T17:26:43.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.924+0000 7f1ad259c640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1abc0a0e90 0x7f1abc0a3350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40058c0 tx=0x7f1ac4005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:43.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:43.931+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1acc096650 con 0x7f1ad4100810 2024-09-13T17:26:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.131+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ad410a6c0 con 0x7f1ad4100810 2024-09-13T17:26:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.132+0000 7f1ac37fe640 1 -- 172.21.15.44:0/3935906155 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1acc099c70 con 0x7f1ad4100810 2024-09-13T17:26:44.133 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:44.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1abc0a0e90 msgr2=0x7f1abc0a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1abc0a0e90 0x7f1abc0a3350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40058c0 tx=0x7f1ac4005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 msgr2=0x7f1ad419d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 0x7f1ad419d030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009fd0 tx=0x7f1acc00ef10 comp rx=0 tx=0).stop 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 shutdown_connections 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1abc0a0e90 0x7f1abc0a3350 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad40ffe50 0x7f1ad419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad4100810 0x7f1ad419d030 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 --2- 172.21.15.44:0/3935906155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ad40fec60 0x7f1ad4068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 >> 172.21.15.44:0/3935906155 conn(0x7f1ad40fa3e0 msgr2=0x7f1ad41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 shutdown_connections 2024-09-13T17:26:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.134+0000 7f1ad8868640 1 -- 172.21.15.44:0/3935906155 wait complete. 2024-09-13T17:26:44.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:44.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:43 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2176905706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-13T17:26:44.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:43 smithi100 ceph-mon[35055]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T17:26:44.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:43 smithi100 ceph-mon[35055]: mgrmap e22: x(active, since 1.01758s), standbys: y, z 2024-09-13T17:26:44.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:43 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/584035558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:44.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 -- 172.21.15.44:0/6737724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc0fec90 msgr2=0x7f51bc0ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 --2- 172.21.15.44:0/6737724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc0fec90 0x7f51bc0ff090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009900 tx=0x7f51ac02a410 comp rx=0 tx=0).stop 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 -- 172.21.15.44:0/6737724 shutdown_connections 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 --2- 172.21.15.44:0/6737724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc100840 0x7f51bc1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 --2- 172.21.15.44:0/6737724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0ffe80 0x7f51bc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 --2- 172.21.15.44:0/6737724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc0fec90 0x7f51bc0ff090 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 -- 172.21.15.44:0/6737724 >> 172.21.15.44:0/6737724 conn(0x7f51bc0fa3f0 msgr2=0x7f51bc0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 -- 172.21.15.44:0/6737724 shutdown_connections 2024-09-13T17:26:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.279+0000 7f51c1619640 1 -- 172.21.15.44:0/6737724 wait complete. 2024-09-13T17:26:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.280+0000 7f51c1619640 1 Processor -- start 2024-09-13T17:26:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.280+0000 7f51c1619640 1 -- start start 2024-09-13T17:26:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.280+0000 7f51c1619640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 0x7f51bc068cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.280+0000 7f51c1619640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc0ffe80 0x7f51bc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51c1619640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 0x7f51bc19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51c1619640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc19d7c0 con 0x7f51bc100840 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51c1619640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc19d930 con 0x7f51bc0fec90 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51c1619640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc19daf0 con 0x7f51bc0ffe80 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 0x7f51bc19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 0x7f51bc068cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 0x7f51bc19d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40218/0 (socket says 172.21.15.44:40218) 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 0x7f51bc068cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:55798/0 (socket says 172.21.15.44:55798) 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 -- 172.21.15.44:0/3345410140 learned_addr learned my addr 172.21.15.44:0/3345410140 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51ba7fc640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc0ffe80 0x7f51bc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 -- 172.21.15.44:0/3345410140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc0ffe80 msgr2=0x7f51bc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc0ffe80 0x7f51bc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 -- 172.21.15.44:0/3345410140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 msgr2=0x7f51bc068cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 0x7f51bc068cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 -- 172.21.15.44:0/3345410140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0009490 con 0x7f51bc100840 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51baffd640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 0x7f51bc068cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51ba7fc640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc0ffe80 0x7f51bc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.281+0000 7f51bb7fe640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 0x7f51bc19d030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f51b400eb10 tx=0x7f51b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.282+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4004b20 con 0x7f51bc100840 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.282+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac009560 con 0x7f51bc100840 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.282+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51b4004c80 con 0x7f51bc100840 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.282+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4005a10 con 0x7f51bc100840 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.282+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc19e150 con 0x7f51bc100840 2024-09-13T17:26:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.283+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc1061c0 con 0x7f51bc100840 2024-09-13T17:26:44.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.285+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f51b40106b0 con 0x7f51bc100840 2024-09-13T17:26:44.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.286+0000 7f51a3fff640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51a40a0e90 0x7f51a40a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.286+0000 7f51baffd640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51a40a0e90 0x7f51a40a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.286+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f51b4014070 con 0x7f51bc100840 2024-09-13T17:26:44.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.286+0000 7f51baffd640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51a40a0e90 0x7f51a40a3350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0057c0 tx=0x7f51ac005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:44.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.292+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f51b4096520 con 0x7f51bc100840 2024-09-13T17:26:44.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2176905706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-13T17:26:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-mon[31175]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T17:26:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-mon[31175]: mgrmap e22: x(active, since 1.01758s), standbys: y, z 2024-09-13T17:26:44.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:43 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/584035558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:43 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2176905706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-13T17:26:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:43 smithi070 ceph-mon[34502]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T17:26:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:43 smithi070 ceph-mon[34502]: mgrmap e22: x(active, since 1.01758s), standbys: y, z 2024-09-13T17:26:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:43 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/584035558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:44.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.515+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51bc105c30 con 0x7f51bc100840 2024-09-13T17:26:44.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.515+0000 7f51a3fff640 1 -- 172.21.15.44:0/3345410140 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51b4099b40 con 0x7f51bc100840 2024-09-13T17:26:44.517 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:44.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51a40a0e90 msgr2=0x7f51a40a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51a40a0e90 0x7f51a40a3350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0057c0 tx=0x7f51ac005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 msgr2=0x7f51bc19d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 0x7f51bc19d030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f51b400eb10 tx=0x7f51b400c2d0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 shutdown_connections 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51a40a0e90 0x7f51a40a3350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51bc0ffe80 0x7f51bc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc100840 0x7f51bc19d030 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 --2- 172.21.15.44:0/3345410140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc0fec90 0x7f51bc068cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.517+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 >> 172.21.15.44:0/3345410140 conn(0x7f51bc0fa3f0 msgr2=0x7f51bc103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.518+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 shutdown_connections 2024-09-13T17:26:44.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.518+0000 7f51c1619640 1 -- 172.21.15.44:0/3345410140 wait complete. 2024-09-13T17:26:44.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 -- 172.21.15.44:0/1285499183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 msgr2=0x7f322c1055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 --2- 172.21.15.44:0/1285499183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c1055b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3218009900 tx=0x7f321802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 -- 172.21.15.44:0/1285499183 shutdown_connections 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 --2- 172.21.15.44:0/1285499183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c105af0 0x7f322c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 --2- 172.21.15.44:0/1285499183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c1055b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 --2- 172.21.15.44:0/1285499183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c100890 0x7f322c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 -- 172.21.15.44:0/1285499183 >> 172.21.15.44:0/1285499183 conn(0x7f322c0fa3d0 msgr2=0x7f322c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 -- 172.21.15.44:0/1285499183 shutdown_connections 2024-09-13T17:26:44.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.640+0000 7f3232f33640 1 -- 172.21.15.44:0/1285499183 wait complete. 2024-09-13T17:26:44.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.641+0000 7f3232f33640 1 Processor -- start 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.641+0000 7f3232f33640 1 -- start start 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3232f33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c100890 0x7f322c195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3232f33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3232f33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c105af0 0x7f322c19d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3232f33640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19d8d0 con 0x7f322c105af0 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3232f33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19da40 con 0x7f322c1031c0 2024-09-13T17:26:44.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3232f33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19dc00 con 0x7f322c100890 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f32314d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c105af0 0x7f322c19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:55842/0 (socket says 172.21.15.44:55842) 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 -- 172.21.15.44:0/3808975581 learned_addr learned my addr 172.21.15.44:0/3808975581 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f3230ccf640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c100890 0x7f322c195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 -- 172.21.15.44:0/3808975581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c100890 msgr2=0x7f322c195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c100890 0x7f322c195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 -- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c105af0 msgr2=0x7f322c19d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c105af0 0x7f322c19d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 -- 172.21.15.44:0/3808975581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3218009560 con 0x7f322c1031c0 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.642+0000 7f322bfff640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c196320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f321802a930 tx=0x7f3218005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.643+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218004c50 con 0x7f322c1031c0 2024-09-13T17:26:44.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.643+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c19de80 con 0x7f322c1031c0 2024-09-13T17:26:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.643+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3218004db0 con 0x7f322c1031c0 2024-09-13T17:26:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.643+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321803ca30 con 0x7f322c1031c0 2024-09-13T17:26:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.643+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c19e2a0 con 0x7f322c1031c0 2024-09-13T17:26:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.644+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c0fbec0 con 0x7f322c1031c0 2024-09-13T17:26:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.645+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f32180035f0 con 0x7f322c1031c0 2024-09-13T17:26:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.647+0000 7f3229ffb640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f321c0a0e90 0x7f321c0a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.647+0000 7f3230ccf640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f321c0a0e90 0x7f321c0a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.647+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f32180edb80 con 0x7f322c1031c0 2024-09-13T17:26:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.647+0000 7f3230ccf640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f321c0a0e90 0x7f321c0a3350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:44.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.650+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f32180b7230 con 0x7f322c1031c0 2024-09-13T17:26:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.823+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f322c02c3e0 con 0x7f322c1031c0 2024-09-13T17:26:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.823+0000 7f3229ffb640 1 -- 172.21.15.44:0/3808975581 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32180ba850 con 0x7f322c1031c0 2024-09-13T17:26:44.824 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f321c0a0e90 msgr2=0x7f321c0a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f321c0a0e90 0x7f321c0a3350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b390 comp rx=0 tx=0).stop 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 msgr2=0x7f322c196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c196320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f321802a930 tx=0x7f3218005610 comp rx=0 tx=0).stop 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 shutdown_connections 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f321c0a0e90 0x7f321c0a3350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c105af0 0x7f322c19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c1031c0 0x7f322c196320 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 --2- 172.21.15.44:0/3808975581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c100890 0x7f322c195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 >> 172.21.15.44:0/3808975581 conn(0x7f322c0fa3d0 msgr2=0x7f322c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:44.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 shutdown_connections 2024-09-13T17:26:44.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.826+0000 7f3232f33640 1 -- 172.21.15.44:0/3808975581 wait complete. 2024-09-13T17:26:44.838 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:44.748+0000 7fb66a69f640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:17:26:44] ENGINE Error in HTTPServer.serve 2024-09-13T17:26:44.838 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: Traceback (most recent call last): 2024-09-13T17:26:44.838 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:26:44.838 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._connections.run(self.expiration_interval) 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._run(expiration_interval) 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: s = self.context.wrap_socket( 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: return self.sslsocket_class._create( 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:26:44.839 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self.do_handshake() 2024-09-13T17:26:44.840 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:26:44.840 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: self._sslobj.do_handshake() 2024-09-13T17:26:44.840 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:26:44.840 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:26:44.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2103115559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81008a0 msgr2=0x7f74a8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2103115559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81008a0 0x7f74a8102c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7498009900 tx=0x7f749802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2103115559 shutdown_connections 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2103115559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a81031d0 0x7f74a81055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2103115559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a8105b00 0x7f74a8107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2103115559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81008a0 0x7f74a8102c90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2103115559 >> 172.21.15.44:0/2103115559 conn(0x7f74a80fa3e0 msgr2=0x7f74a80fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2103115559 shutdown_connections 2024-09-13T17:26:44.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.950+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2103115559 wait complete. 2024-09-13T17:26:44.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 Processor -- start 2024-09-13T17:26:44.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 -- start start 2024-09-13T17:26:44.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a81008a0 0x7f74a8100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 0x7f74a80fe850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a8105b00 0x7f74a80fed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80ff4f0 con 0x7f74a81031d0 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80ff660 con 0x7f74a81008a0 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74ad4c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a819fd30 con 0x7f74a8105b00 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74a67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 0x7f74a80fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74a67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 0x7f74a80fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40256/0 (socket says 172.21.15.44:40256) 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.951+0000 7f74a67fc640 1 -- 172.21.15.44:0/2867903043 learned_addr learned my addr 172.21.15.44:0/2867903043 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a6ffd640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a81008a0 0x7f74a8100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a77fe640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a8105b00 0x7f74a80fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a67fc640 1 -- 172.21.15.44:0/2867903043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a8105b00 msgr2=0x7f74a80fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a67fc640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a8105b00 0x7f74a80fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a67fc640 1 -- 172.21.15.44:0/2867903043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a81008a0 msgr2=0x7f74a8100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a67fc640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a81008a0 0x7f74a8100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a67fc640 1 -- 172.21.15.44:0/2867903043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009560 con 0x7f74a81031d0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74a67fc640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 0x7f74a80fe850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c9a0 tx=0x7f749c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c007bb0 con 0x7f74a81031d0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f749c007d10 con 0x7f74a81031d0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a819ffe0 con 0x7f74a81031d0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c018660 con 0x7f74a81031d0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.952+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a81a0410 con 0x7f74a81031d0 2024-09-13T17:26:44.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.954+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a80fbed0 con 0x7f74a81031d0 2024-09-13T17:26:44.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.954+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f749c00f450 con 0x7f74a81031d0 2024-09-13T17:26:44.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.955+0000 7f748ffff640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f74900a0e90 0x7f74900a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:44.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.955+0000 7f74a6ffd640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f74900a0e90 0x7f74900a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:44.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.956+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f749c0cca10 con 0x7f74a81031d0 2024-09-13T17:26:44.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.956+0000 7f74a6ffd640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f74900a0e90 0x7f74900a3350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f749802aa00 tx=0x7f7498005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:44.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:44.960+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f749c096010 con 0x7f74a81031d0 2024-09-13T17:26:45.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='client.25142 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3935906155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3345410140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3808975581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:44 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.150+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74a810b7f0 con 0x7f74a81031d0 2024-09-13T17:26:45.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.150+0000 7f748ffff640 1 -- 172.21.15.44:0/2867903043 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f749c099630 con 0x7f74a81031d0 2024-09-13T17:26:45.152 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f74900a0e90 msgr2=0x7f74900a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f74900a0e90 0x7f74900a3350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f749802aa00 tx=0x7f7498005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 msgr2=0x7f74a80fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 0x7f74a80fe850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c9a0 tx=0x7f749c00ce60 comp rx=0 tx=0).stop 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 shutdown_connections 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f74900a0e90 0x7f74900a3350 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a8105b00 0x7f74a80fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a81031d0 0x7f74a80fe850 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 --2- 172.21.15.44:0/2867903043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a81008a0 0x7f74a8100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.152+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 >> 172.21.15.44:0/2867903043 conn(0x7f74a80fa3e0 msgr2=0x7f74a8101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:45.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.153+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 shutdown_connections 2024-09-13T17:26:45.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.153+0000 7f74ad4c9640 1 -- 172.21.15.44:0/2867903043 wait complete. 2024-09-13T17:26:45.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 -- 172.21.15.44:0/964377919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd80feca0 msgr2=0x7fbfd80ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 --2- 172.21.15.44:0/964377919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd80feca0 0x7fbfd80ff0a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009900 tx=0x7fbfc802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 -- 172.21.15.44:0/964377919 shutdown_connections 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 --2- 172.21.15.44:0/964377919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd8100850 0x7fbfd81070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 --2- 172.21.15.44:0/964377919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80ffe90 0x7fbfd8100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 --2- 172.21.15.44:0/964377919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd80feca0 0x7fbfd80ff0a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 -- 172.21.15.44:0/964377919 >> 172.21.15.44:0/964377919 conn(0x7fbfd80fa440 msgr2=0x7fbfd80fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 -- 172.21.15.44:0/964377919 shutdown_connections 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.270+0000 7fbfdde96640 1 -- 172.21.15.44:0/964377919 wait complete. 2024-09-13T17:26:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 Processor -- start 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 -- start start 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 0x7fbfd8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd80ffe90 0x7fbfd8196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd8100850 0x7fbfd819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819d810 con 0x7fbfd8100850 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819d980 con 0x7fbfd80feca0 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfdde96640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819db40 con 0x7fbfd80ffe90 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 0x7fbfd8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd8100850 0x7fbfd819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 0x7fbfd8195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:55884/0 (socket says 172.21.15.44:55884) 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 -- 172.21.15.44:0/962427734 learned_addr learned my addr 172.21.15.44:0/962427734 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd6ffd640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd80ffe90 0x7fbfd8196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 -- 172.21.15.44:0/962427734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd80ffe90 msgr2=0x7fbfd8196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd80ffe90 0x7fbfd8196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 -- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd8100850 msgr2=0x7fbfd819d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd8100850 0x7fbfd819d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.271+0000 7fbfd77fe640 1 -- 172.21.15.44:0/962427734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8009560 con 0x7fbfd80feca0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.272+0000 7fbfd77fe640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 0x7fbfd8195d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8005ab0 tx=0x7fbfc80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.272+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8031d70 con 0x7fbfd80feca0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.272+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd819ddc0 con 0x7fbfd80feca0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.272+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfc8005490 con 0x7fbfd80feca0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.272+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8003990 con 0x7fbfd80feca0 2024-09-13T17:26:45.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.272+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd819e200 con 0x7fbfd80feca0 2024-09-13T17:26:45.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.273+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd810a410 con 0x7fbfd80feca0 2024-09-13T17:26:45.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:45.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='client.25142 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3935906155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3345410140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3808975581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.278 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.279 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:44 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.274+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fbfc8003af0 con 0x7fbfd80feca0 2024-09-13T17:26:45.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.275+0000 7fbfd4ff9640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc40a0e90 0x7fbfc40a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.275+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbfc80ed910 con 0x7fbfd80feca0 2024-09-13T17:26:45.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.275+0000 7fbfd6ffd640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc40a0e90 0x7fbfc40a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.275+0000 7fbfd6ffd640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc40a0e90 0x7fbfc40a3350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009bd0 tx=0x7fbfc0005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:45.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.278+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbfc80b6e80 con 0x7fbfd80feca0 2024-09-13T17:26:45.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.315+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7fbfc80b7250 con 0x7fbfd80feca0 2024-09-13T17:26:45.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:45.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='client.25142 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:45.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3935906155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3345410140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3808975581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:45.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:44 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.466+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfd819e570 con 0x7fbfd80feca0 2024-09-13T17:26:45.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.468+0000 7fbfd4ff9640 1 -- 172.21.15.44:0/962427734 <== mon.2 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24868 (secure 0 0 0) 0x7fbfc80acef0 con 0x7fbfd80feca0 2024-09-13T17:26:45.469 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:45.469 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":93,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:26:42.344870+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:45.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc40a0e90 msgr2=0x7fbfc40a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc40a0e90 0x7fbfc40a3350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009bd0 tx=0x7fbfc0005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 msgr2=0x7fbfd8195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 0x7fbfd8195d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8005ab0 tx=0x7fbfc80042f0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 shutdown_connections 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc40a0e90 0x7fbfc40a3350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd80ffe90 0x7fbfd8196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd8100850 0x7fbfd819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 --2- 172.21.15.44:0/962427734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd80feca0 0x7fbfd8195d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 >> 172.21.15.44:0/962427734 conn(0x7fbfd80fa440 msgr2=0x7fbfd8067650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 shutdown_connections 2024-09-13T17:26:45.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.470+0000 7fbfdde96640 1 -- 172.21.15.44:0/962427734 wait complete. 2024-09-13T17:26:45.484 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:45.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 -- 172.21.15.44:0/3999624193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d40ffd80 msgr2=0x7ff7d4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/3999624193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4100180 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009900 tx=0x7ff7d002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 -- 172.21.15.44:0/3999624193 shutdown_connections 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/3999624193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4101930 0x7ff7d41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/3999624193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d4100f70 0x7ff7d41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/3999624193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4100180 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 -- 172.21.15.44:0/3999624193 >> 172.21.15.44:0/3999624193 conn(0x7ff7d40fb4e0 msgr2=0x7ff7d40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 -- 172.21.15.44:0/3999624193 shutdown_connections 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.592+0000 7ff7dbbec640 1 -- 172.21.15.44:0/3999624193 wait complete. 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 Processor -- start 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 -- start start 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4100f70 0x7ff7d4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4101930 0x7ff7d419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419d820 con 0x7ff7d4101930 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419d990 con 0x7ff7d40ffd80 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7dbbec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419db50 con 0x7ff7d4100f70 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7da189640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4101930 0x7ff7d419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9187640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4100f70 0x7ff7d4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:55898/0 (socket says 172.21.15.44:55898) 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 -- 172.21.15.44:0/4270042474 learned_addr learned my addr 172.21.15.44:0/4270042474 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 -- 172.21.15.44:0/4270042474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4100f70 msgr2=0x7ff7d4196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4100f70 0x7ff7d4196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 -- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4101930 msgr2=0x7ff7d419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4101930 0x7ff7d419d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9988640 1 -- 172.21.15.44:0/4270042474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009840 con 0x7ff7d40ffd80 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7da189640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4101930 0x7ff7d419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.593+0000 7ff7d9187640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4100f70 0x7ff7d4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.594+0000 7ff7d9988640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4195d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7d002a930 tx=0x7ff7d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.594+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0004cc0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.594+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7d0003a80 con 0x7ff7d40ffd80 2024-09-13T17:26:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.594+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d003c900 con 0x7ff7d40ffd80 2024-09-13T17:26:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.594+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d0009560 con 0x7ff7d40ffd80 2024-09-13T17:26:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.594+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d419e0f0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.595+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d41072d0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.597+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7ff7d00035f0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.597+0000 7ff7caffd640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7c00a1310 0x7ff7c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.597+0000 7ff7d9187640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7c00a1310 0x7ff7c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.597+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff7d00ee1b0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.598+0000 7ff7d9187640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7c00a1310 0x7ff7c00a37d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400bfd0 tx=0x7ff7c400b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.601+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff7d00b70e0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.817+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7d402c3e0 con 0x7ff7d40ffd80 2024-09-13T17:26:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.817+0000 7ff7caffd640 1 -- 172.21.15.44:0/4270042474 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7d00b9010 con 0x7ff7d40ffd80 2024-09-13T17:26:45.819 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.819+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7c00a1310 msgr2=0x7ff7c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7c00a1310 0x7ff7c00a37d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400bfd0 tx=0x7ff7c400b390 comp rx=0 tx=0).stop 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 msgr2=0x7ff7d4195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4195d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7d002a930 tx=0x7ff7d00046c0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 shutdown_connections 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7c00a1310 0x7ff7c00a37d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d4100f70 0x7ff7d4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4101930 0x7ff7d419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 --2- 172.21.15.44:0/4270042474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d40ffd80 0x7ff7d4195d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 >> 172.21.15.44:0/4270042474 conn(0x7ff7d40fb4e0 msgr2=0x7ff7d41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 shutdown_connections 2024-09-13T17:26:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.820+0000 7ff7dbbec640 1 -- 172.21.15.44:0/4270042474 wait complete. 2024-09-13T17:26:45.836 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:45.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 -- 172.21.15.44:0/4011239345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0ff810 msgr2=0x7f3d2c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 --2- 172.21.15.44:0/4011239345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0ff810 0x7f3d2c103850 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009930 tx=0x7f3d1c02a420 comp rx=0 tx=0).stop 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 -- 172.21.15.44:0/4011239345 shutdown_connections 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 --2- 172.21.15.44:0/4011239345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 0x7f3d2c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 --2- 172.21.15.44:0/4011239345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0ff810 0x7f3d2c103850 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 --2- 172.21.15.44:0/4011239345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 -- 172.21.15.44:0/4011239345 >> 172.21.15.44:0/4011239345 conn(0x7f3d2c0fa910 msgr2=0x7f3d2c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 -- 172.21.15.44:0/4011239345 shutdown_connections 2024-09-13T17:26:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.948+0000 7f3d34718640 1 -- 172.21.15.44:0/4011239345 wait complete. 2024-09-13T17:26:45.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 Processor -- start 2024-09-13T17:26:45.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 -- start start 2024-09-13T17:26:45.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 0x7f3d2c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2c101370 0x7f3d2c1a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1a49a0 con 0x7f3d2c0fee60 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1a4b10 con 0x7f3d2c103fb0 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d34718640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1a4cd0 con 0x7f3d2c101370 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d324b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d324b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c1008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40316/0 (socket says 172.21.15.44:40316) 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d324b4640 1 -- 172.21.15.44:0/362728033 learned_addr learned my addr 172.21.15.44:0/362728033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.949+0000 7f3d31cb3640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 0x7f3d2c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d32cb5640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2c101370 0x7f3d2c1a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d324b4640 1 -- 172.21.15.44:0/362728033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2c101370 msgr2=0x7f3d2c1a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d324b4640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2c101370 0x7f3d2c1a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d324b4640 1 -- 172.21.15.44:0/362728033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 msgr2=0x7f3d2c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d324b4640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 0x7f3d2c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d324b4640 1 -- 172.21.15.44:0/362728033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24009840 con 0x7f3d2c0fee60 2024-09-13T17:26:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d31cb3640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 0x7f3d2c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d324b4640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c1008f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e9f0 tx=0x7f3d2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28009700 con 0x7f3d2c0fee60 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c009560 con 0x7f3d2c0fee60 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c1a52a0 con 0x7f3d2c0fee60 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d2800ccc0 con 0x7f3d2c0fee60 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.950+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28018680 con 0x7f3d2c0fee60 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.951+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c100330 con 0x7f3d2c0fee60 2024-09-13T17:26:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.953+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7f3d28010950 con 0x7f3d2c0fee60 2024-09-13T17:26:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.954+0000 7f3d237fe640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3d180a1310 0x7f3d180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.954+0000 7f3d31cb3640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3d180a1310 0x7f3d180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.954+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3d28014070 con 0x7f3d2c0fee60 2024-09-13T17:26:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.954+0000 7f3d31cb3640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3d180a1310 0x7f3d180a37d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c005ab0 tx=0x7f3d1c005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:45.960+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3d28052e20 con 0x7f3d2c0fee60 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: [13/Sep/2024:17:26:44] ENGINE Bus STARTING 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: [13/Sep/2024:17:26:44] ENGINE Serving on https://172.21.15.44:7150 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: [13/Sep/2024:17:26:44] ENGINE Error in HTTPServer.serve 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: Traceback (most recent call last): 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._connections.run(self.expiration_interval) 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._run(expiration_interval) 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:26:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: s = self.context.wrap_socket( 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: return self.sslsocket_class._create( 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: self.do_handshake() 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._sslobj.do_handshake() 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: [13/Sep/2024:17:26:44] ENGINE Serving on http://172.21.15.44:8765 2024-09-13T17:26:46.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: [13/Sep/2024:17:26:44] ENGINE Bus STARTED 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi070 to 176.0M 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Unable to set osd_memory_target on smithi070 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Unable to set osd_memory_target on smithi044 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Updating smithi044:/etc/ceph/ceph.conf 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2867903043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/962427734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4270042474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:46 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.163+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d2c0ff240 con 0x7f3d2c0fee60 2024-09-13T17:26:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.163+0000 7f3d237fe640 1 -- 172.21.15.44:0/362728033 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d28099e90 con 0x7f3d2c0fee60 2024-09-13T17:26:46.164 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3d180a1310 msgr2=0x7f3d180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3d180a1310 0x7f3d180a37d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c005ab0 tx=0x7f3d1c005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 msgr2=0x7f3d2c1008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c1008f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e9f0 tx=0x7f3d2800edf0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 shutdown_connections 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3d180a1310 0x7f3d180a37d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2c101370 0x7f3d2c1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d2c103fb0 0x7f3d2c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 --2- 172.21.15.44:0/362728033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d2c0fee60 0x7f3d2c1008f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.165+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 >> 172.21.15.44:0/362728033 conn(0x7f3d2c0fa910 msgr2=0x7f3d2c0686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.166+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 shutdown_connections 2024-09-13T17:26:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.166+0000 7f3d34718640 1 -- 172.21.15.44:0/362728033 wait complete. 2024-09-13T17:26:46.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 -- 172.21.15.44:0/2489189304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fee60 msgr2=0x7fda100ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 --2- 172.21.15.44:0/2489189304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fee60 0x7fda100ff240 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fda00009930 tx=0x7fda0002a420 comp rx=0 tx=0).stop 2024-09-13T17:26:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 -- 172.21.15.44:0/2489189304 shutdown_connections 2024-09-13T17:26:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 --2- 172.21.15.44:0/2489189304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda100ff810 0x7fda10103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 --2- 172.21.15.44:0/2489189304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda10103fb0 0x7fda1010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 --2- 172.21.15.44:0/2489189304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fee60 0x7fda100ff240 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 -- 172.21.15.44:0/2489189304 >> 172.21.15.44:0/2489189304 conn(0x7fda10075c60 msgr2=0x7fda100fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 -- 172.21.15.44:0/2489189304 shutdown_connections 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.285+0000 7fda16ee5640 1 -- 172.21.15.44:0/2489189304 wait complete. 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 Processor -- start 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 -- start start 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda100fee60 0x7fda100fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda10103fb0 0x7fda100fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 0x7fda101a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101a49a0 con 0x7fda100fbb20 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101a4b10 con 0x7fda10103fb0 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda16ee5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101a4cd0 con 0x7fda100fee60 2024-09-13T17:26:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 0x7fda101a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 0x7fda101a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40342/0 (socket says 172.21.15.44:40342) 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 -- 172.21.15.44:0/3740255021 learned_addr learned my addr 172.21.15.44:0/3740255021 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda0ffff640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda10103fb0 0x7fda100fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda14c81640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda100fee60 0x7fda100fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 -- 172.21.15.44:0/3740255021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda100fee60 msgr2=0x7fda100fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda100fee60 0x7fda100fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 -- 172.21.15.44:0/3740255021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda10103fb0 msgr2=0x7fda100fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda10103fb0 0x7fda100fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.286+0000 7fda15482640 1 -- 172.21.15.44:0/3740255021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00009560 con 0x7fda100fbb20 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.287+0000 7fda15482640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 0x7fda101a43d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fda0800da40 tx=0x7fda0800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.287+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0800bdd0 con 0x7fda100fbb20 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.287+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda08009890 con 0x7fda100fbb20 2024-09-13T17:26:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.287+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda101a4fb0 con 0x7fda100fbb20 2024-09-13T17:26:46.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.287+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08010460 con 0x7fda100fbb20 2024-09-13T17:26:46.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.287+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda101a5440 con 0x7fda100fbb20 2024-09-13T17:26:46.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.288+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda10100330 con 0x7fda100fbb20 2024-09-13T17:26:46.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.289+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7fda08003970 con 0x7fda100fbb20 2024-09-13T17:26:46.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.290+0000 7fda0dffb640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9f80a12c0 0x7fd9f80a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.290+0000 7fda14c81640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9f80a12c0 0x7fd9f80a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.290+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fda080ccd80 con 0x7fda100fbb20 2024-09-13T17:26:46.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.290+0000 7fda14c81640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9f80a12c0 0x7fd9f80a3780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fda00006010 tx=0x7fda00005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:46.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.294+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fda08095ce0 con 0x7fda100fbb20 2024-09-13T17:26:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: [13/Sep/2024:17:26:44] ENGINE Bus STARTING 2024-09-13T17:26:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: [13/Sep/2024:17:26:44] ENGINE Serving on https://172.21.15.44:7150 2024-09-13T17:26:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: [13/Sep/2024:17:26:44] ENGINE Error in HTTPServer.serve 2024-09-13T17:26:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout: Traceback (most recent call last): 2024-09-13T17:26:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: self._run(expiration_interval) 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: s = self.context.wrap_socket( 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: return self.sslsocket_class._create( 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:26:46.448 INFO:journalctl@ceph.mon.b.smithi070.stdout: self.do_handshake() 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout: self._sslobj.do_handshake() 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: [13/Sep/2024:17:26:44] ENGINE Serving on http://172.21.15.44:8765 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: [13/Sep/2024:17:26:44] ENGINE Bus STARTED 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi070 to 176.0M 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Unable to set osd_memory_target on smithi070 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Unable to set osd_memory_target on smithi044 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Updating smithi044:/etc/ceph/ceph.conf 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2867903043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:46.450 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-13T17:26:46.451 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/962427734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:46.451 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4270042474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:46.451 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.451 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.451 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:46 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.464+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda100ff240 con 0x7fda100fbb20 2024-09-13T17:26:46.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.464+0000 7fda0dffb640 1 -- 172.21.15.44:0/3740255021 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda08099300 con 0x7fda100fbb20 2024-09-13T17:26:46.466 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:46.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9f80a12c0 msgr2=0x7fd9f80a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9f80a12c0 0x7fd9f80a3780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fda00006010 tx=0x7fda00005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 msgr2=0x7fda101a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 0x7fda101a43d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fda0800da40 tx=0x7fda0800df00 comp rx=0 tx=0).stop 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 shutdown_connections 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9f80a12c0 0x7fd9f80a3780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda100fbb20 0x7fda101a43d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda10103fb0 0x7fda100fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 --2- 172.21.15.44:0/3740255021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda100fee60 0x7fda100fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 >> 172.21.15.44:0/3740255021 conn(0x7fda10075c60 msgr2=0x7fda10067800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:46.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 shutdown_connections 2024-09-13T17:26:46.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.466+0000 7fda16ee5640 1 -- 172.21.15.44:0/3740255021 wait complete. 2024-09-13T17:26:46.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:46.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: [13/Sep/2024:17:26:44] ENGINE Bus STARTING 2024-09-13T17:26:46.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: [13/Sep/2024:17:26:44] ENGINE Serving on https://172.21.15.44:7150 2024-09-13T17:26:46.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: [13/Sep/2024:17:26:44] ENGINE Error in HTTPServer.serve 2024-09-13T17:26:46.524 INFO:journalctl@ceph.mon.c.smithi100.stdout: Traceback (most recent call last): 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: self._connections.run(self.expiration_interval) 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: self._run(expiration_interval) 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: s = self.context.wrap_socket( 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: return self.sslsocket_class._create( 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T17:26:46.525 INFO:journalctl@ceph.mon.c.smithi100.stdout: self.do_handshake() 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout: self._sslobj.do_handshake() 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: [13/Sep/2024:17:26:44] ENGINE Serving on http://172.21.15.44:8765 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: [13/Sep/2024:17:26:44] ENGINE Bus STARTED 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi070 to 176.0M 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Unable to set osd_memory_target on smithi070 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Unable to set osd_memory_target on smithi044 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Updating smithi044:/etc/ceph/ceph.conf 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-13T17:26:46.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2867903043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:46.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-13T17:26:46.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/962427734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:46.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4270042474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:46.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:46 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- 172.21.15.44:0/3335477299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e80fff90 msgr2=0x7f19e8100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- 172.21.15.44:0/3335477299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e80fff90 0x7f19e8100370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009930 tx=0x7f19d802a420 comp rx=0 tx=0).stop 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- 172.21.15.44:0/3335477299 shutdown_connections 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- 172.21.15.44:0/3335477299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e81050e0 0x7f19e810cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- 172.21.15.44:0/3335477299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e8100940 0x7f19e8104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- 172.21.15.44:0/3335477299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e80fff90 0x7f19e8100370 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- 172.21.15.44:0/3335477299 >> 172.21.15.44:0/3335477299 conn(0x7f19e80fba20 msgr2=0x7f19e80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- 172.21.15.44:0/3335477299 shutdown_connections 2024-09-13T17:26:46.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- 172.21.15.44:0/3335477299 wait complete. 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 Processor -- start 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- start start 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e80fff90 0x7f19e80727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 0x7f19e8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8073230 0x7f19e81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81a49e0 con 0x7f19e81050e0 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81a4b50 con 0x7f19e80fff90 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.583+0000 7f19eece5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81a4d10 con 0x7f19e8073230 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 0x7f19e8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 0x7f19e8072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40362/0 (socket says 172.21.15.44:40362) 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 -- 172.21.15.44:0/1875731705 learned_addr learned my addr 172.21.15.44:0/1875731705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19eca81640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e80fff90 0x7f19e80727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19ed282640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8073230 0x7f19e81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 -- 172.21.15.44:0/1875731705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8073230 msgr2=0x7f19e81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8073230 0x7f19e81a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 -- 172.21.15.44:0/1875731705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e80fff90 msgr2=0x7f19e80727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e80fff90 0x7f19e80727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 -- 172.21.15.44:0/1875731705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d8009560 con 0x7f19e81050e0 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19eca81640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e80fff90 0x7f19e80727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e7fff640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 0x7f19e8072cf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f19d000e9c0 tx=0x7f19d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0009700 con 0x7f19e81050e0 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19d000cd80 con 0x7f19e81050e0 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e81a4ff0 con 0x7f19e81050e0 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.584+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0010430 con 0x7f19e81050e0 2024-09-13T17:26:46.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.585+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e81a54c0 con 0x7f19e81050e0 2024-09-13T17:26:46.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.586+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8101460 con 0x7f19e81050e0 2024-09-13T17:26:46.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.587+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7f19d0004eb0 con 0x7f19e81050e0 2024-09-13T17:26:46.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.587+0000 7f19e5ffb640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f19d40a1340 0x7f19d40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.588+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f19d0014070 con 0x7f19e81050e0 2024-09-13T17:26:46.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.588+0000 7f19eca81640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f19d40a1340 0x7f19d40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.588+0000 7f19eca81640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f19d40a1340 0x7f19d40a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19d80057c0 tx=0x7f19d8000bc0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:46.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.592+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f19d0096580 con 0x7f19e81050e0 2024-09-13T17:26:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.765+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19e8108290 con 0x7f19e81050e0 2024-09-13T17:26:46.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.766+0000 7f19e5ffb640 1 -- 172.21.15.44:0/1875731705 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19d0099ba0 con 0x7f19e81050e0 2024-09-13T17:26:46.767 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f19d40a1340 msgr2=0x7f19d40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f19d40a1340 0x7f19d40a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19d80057c0 tx=0x7f19d8000bc0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 msgr2=0x7f19e8072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 0x7f19e8072cf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f19d000e9c0 tx=0x7f19d000ee80 comp rx=0 tx=0).stop 2024-09-13T17:26:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 shutdown_connections 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f19d40a1340 0x7f19d40a3800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8073230 0x7f19e81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19e81050e0 0x7f19e8072cf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 --2- 172.21.15.44:0/1875731705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e80fff90 0x7f19e80727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 >> 172.21.15.44:0/1875731705 conn(0x7f19e80fba20 msgr2=0x7f19e80fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 shutdown_connections 2024-09-13T17:26:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.768+0000 7f19eece5640 1 -- 172.21.15.44:0/1875731705 wait complete. 2024-09-13T17:26:46.780 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-13T17:26:46.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:46.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 -- 172.21.15.44:0/777095280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 msgr2=0x7fd73c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 --2- 172.21.15.44:0/777095280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c1012d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009900 tx=0x7fd72c02a480 comp rx=0 tx=0).stop 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 -- 172.21.15.44:0/777095280 shutdown_connections 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 --2- 172.21.15.44:0/777095280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd73c101810 0x7fd73c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 --2- 172.21.15.44:0/777095280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c10a310 0x7fd73c10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 --2- 172.21.15.44:0/777095280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c1012d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 -- 172.21.15.44:0/777095280 >> 172.21.15.44:0/777095280 conn(0x7fd73c0fa900 msgr2=0x7fd73c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 -- 172.21.15.44:0/777095280 shutdown_connections 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.882+0000 7fd742595640 1 -- 172.21.15.44:0/777095280 wait complete. 2024-09-13T17:26:46.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 Processor -- start 2024-09-13T17:26:46.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 -- start start 2024-09-13T17:26:46.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c101810 0x7fd73c196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd73c10a310 0x7fd73c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c19b130 con 0x7fd73c100ef0 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c19b2a0 con 0x7fd73c101810 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd742595640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c19b460 con 0x7fd73c10a310 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40386/0 (socket says 172.21.15.44:40386) 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd73bfff640 1 -- 172.21.15.44:0/3480678730 learned_addr learned my addr 172.21.15.44:0/3480678730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd73b7fe640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c101810 0x7fd73c196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.883+0000 7fd740b32640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd73c10a310 0x7fd73c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd73bfff640 1 -- 172.21.15.44:0/3480678730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd73c10a310 msgr2=0x7fd73c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd73bfff640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd73c10a310 0x7fd73c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd73bfff640 1 -- 172.21.15.44:0/3480678730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c101810 msgr2=0x7fd73c196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd73bfff640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c101810 0x7fd73c196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd73bfff640 1 -- 172.21.15.44:0/3480678730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c009560 con 0x7fd73c100ef0 2024-09-13T17:26:46.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd73bfff640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c196200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0057c0 tx=0x7fd72c032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c005000 con 0x7fd73c100ef0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd72c0046c0 con 0x7fd73c100ef0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c19b6e0 con 0x7fd73c100ef0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c003850 con 0x7fd73c100ef0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.884+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c1a2fc0 con 0x7fd73c100ef0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.885+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c068ed0 con 0x7fd73c100ef0 2024-09-13T17:26:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.886+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7fd72c005160 con 0x7fd73c100ef0 2024-09-13T17:26:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.887+0000 7fd7397fa640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7240a1310 0x7fd7240a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.887+0000 7fd73b7fe640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7240a1310 0x7fd7240a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.887+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd72c0ee2b0 con 0x7fd73c100ef0 2024-09-13T17:26:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.887+0000 7fd73b7fe640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7240a1310 0x7fd7240a37d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd73c1977a0 tx=0x7fd730005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:46.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:46.890+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd72c0b7270 con 0x7fd73c100ef0 2024-09-13T17:26:47.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.101+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd73c1053d0 con 0x7fd73c100ef0 2024-09-13T17:26:47.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.101+0000 7fd7397fa640 1 -- 172.21.15.44:0/3480678730 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fd72c04a090 con 0x7fd73c100ef0 2024-09-13T17:26:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:47.103 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:47.103 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7240a1310 msgr2=0x7fd7240a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7240a1310 0x7fd7240a37d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd73c1977a0 tx=0x7fd730005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 msgr2=0x7fd73c196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c196200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0057c0 tx=0x7fd72c032be0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 shutdown_connections 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7240a1310 0x7fd7240a37d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd73c10a310 0x7fd73c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c101810 0x7fd73c196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 --2- 172.21.15.44:0/3480678730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd73c100ef0 0x7fd73c196200 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 >> 172.21.15.44:0/3480678730 conn(0x7fd73c0fa900 msgr2=0x7fd73c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:47.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 shutdown_connections 2024-09-13T17:26:47.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.104+0000 7fd742595640 1 -- 172.21.15.44:0/3480678730 wait complete. 2024-09-13T17:26:47.116 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-13T17:26:47.116 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-13T17:26:47.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:26:47.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 -- 172.21.15.44:0/1718951380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb4100940 msgr2=0x7fdeb4104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/1718951380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb4100940 0x7fdeb4104980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009930 tx=0x7fdea402a420 comp rx=0 tx=0).stop 2024-09-13T17:26:47.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 -- 172.21.15.44:0/1718951380 shutdown_connections 2024-09-13T17:26:47.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/1718951380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb41050e0 0x7fdeb410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/1718951380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb4100940 0x7fdeb4104980 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/1718951380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb40fff90 0x7fdeb4100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 -- 172.21.15.44:0/1718951380 >> 172.21.15.44:0/1718951380 conn(0x7fdeb40fba20 msgr2=0x7fdeb40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 -- 172.21.15.44:0/1718951380 shutdown_connections 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.218+0000 7fdebc2d0640 1 -- 172.21.15.44:0/1718951380 wait complete. 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 Processor -- start 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 -- start start 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb40fff90 0x7fdeb4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 0x7fdeb41967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb419ab60 0x7fdeb419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419b550 con 0x7fdeb41050e0 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419b6c0 con 0x7fdeb40fff90 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdebc2d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419b830 con 0x7fdeb419ab60 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 0x7fdeb41967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 0x7fdeb41967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40398/0 (socket says 172.21.15.44:40398) 2024-09-13T17:26:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 -- 172.21.15.44:0/2218908405 learned_addr learned my addr 172.21.15.44:0/2218908405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeba06c640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb40fff90 0x7fdeb4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeba86d640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb419ab60 0x7fdeb419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 -- 172.21.15.44:0/2218908405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb419ab60 msgr2=0x7fdeb419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb419ab60 0x7fdeb419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 -- 172.21.15.44:0/2218908405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb40fff90 msgr2=0x7fdeb4196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb40fff90 0x7fdeb4196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.219+0000 7fdeb986b640 1 -- 172.21.15.44:0/2218908405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4009560 con 0x7fdeb41050e0 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.220+0000 7fdeb986b640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 0x7fdeb41967c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005ab0 tx=0x7fdea40329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.220+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4032df0 con 0x7fdeb41050e0 2024-09-13T17:26:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.220+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdea4005490 con 0x7fdeb41050e0 2024-09-13T17:26:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.220+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb41a2e10 con 0x7fdeb41050e0 2024-09-13T17:26:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.220+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40035f0 con 0x7fdeb41050e0 2024-09-13T17:26:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.220+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb41a31d0 con 0x7fdeb41050e0 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.221+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb4101460 con 0x7fdeb41050e0 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.222+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 136044+0+0 (secure 0 0 0) 0x7fdea40047c0 con 0x7fdeb41050e0 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.222+0000 7fdeab7fe640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdea00a1290 0x7fdea00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.223+0000 7fdeba06c640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdea00a1290 0x7fdea00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.223+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fdea40edd60 con 0x7fdeb41050e0 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.223+0000 7fdeba06c640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdea00a1290 0x7fdea00a3750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009bd0 tx=0x7fdeb0005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.226+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdea40b6cc0 con 0x7fdeb41050e0 2024-09-13T17:26:47.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.243+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fdea40b7090 con 0x7fdeb41050e0 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/362728033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3740255021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:47 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1875731705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.399+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdeb4108340 con 0x7fdea00a1290 2024-09-13T17:26:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.411+0000 7fdeab7fe640 1 -- 172.21.15.44:0/2218908405 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426558 (secure 0 0 0) 0x7fdeb4108340 con 0x7fdea00a1290 2024-09-13T17:26:47.417 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:47.420 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdea00a1290 msgr2=0x7fdea00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdea00a1290 0x7fdea00a3750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009bd0 tx=0x7fdeb0005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 msgr2=0x7fdeb41967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 0x7fdeb41967c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005ab0 tx=0x7fdea40329f0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 shutdown_connections 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdea00a1290 0x7fdea00a3750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb419ab60 0x7fdeb419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb41050e0 0x7fdeb41967c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 --2- 172.21.15.44:0/2218908405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb40fff90 0x7fdeb4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 >> 172.21.15.44:0/2218908405 conn(0x7fdeb40fba20 msgr2=0x7fdeb40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 shutdown_connections 2024-09-13T17:26:47.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.416+0000 7fdebc2d0640 1 -- 172.21.15.44:0/2218908405 wait complete. 2024-09-13T17:26:47.427 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-13T17:26:46.362976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188973,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":968,"num_read_kb":647,"num_write":465,"num_write_kb":4379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386016,"kb_used_data":12632,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124555808,"statfs":{"total":1151940427776,"available":1151545147392,"internally_reserved":0,"allocated":12935168,"data_stored":9841364,"data_compressed":502488,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989616"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.554882+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:26:25.554882+0000","last_peered":"2024-09-13T17:26:25.554882+0000","last_clean":"2024-09-13T17:26:25.554882+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:26:25.554882+0000","last_undegraded":"2024-09-13T17:26:25.554882+0000","last_fullsized":"2024-09-13T17:26:25.554882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:22.937512+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:26:22.937512+0000","last_peered":"2024-09-13T17:26:22.937512+0000","last_clean":"2024-09-13T17:26:22.937512+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:26:22.937512+0000","last_undegraded":"2024-09-13T17:26:22.937512+0000","last_fullsized":"2024-09-13T17:26:22.937512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056659+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:25:43.056659+0000","last_peered":"2024-09-13T17:25:43.056659+0000","last_clean":"2024-09-13T17:25:43.056659+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:25:43.056659+0000","last_undegraded":"2024-09-13T17:25:43.056659+0000","last_fullsized":"2024-09-13T17:25:43.056659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:22.937537+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:26:22.937537+0000","last_peered":"2024-09-13T17:26:22.937537+0000","last_clean":"2024-09-13T17:26:22.937537+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:26:22.937537+0000","last_undegraded":"2024-09-13T17:26:22.937537+0000","last_fullsized":"2024-09-13T17:26:22.937537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917611+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:26:30.917611+0000","last_peered":"2024-09-13T17:26:30.917611+0000","last_clean":"2024-09-13T17:26:30.917611+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:26:30.917611+0000","last_undegraded":"2024-09-13T17:26:30.917611+0000","last_fullsized":"2024-09-13T17:26:30.917611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.055957+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:25:43.055957+0000","last_peered":"2024-09-13T17:25:43.055957+0000","last_clean":"2024-09-13T17:25:43.055957+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:25:43.055957+0000","last_undegraded":"2024-09-13T17:25:43.055957+0000","last_fullsized":"2024-09-13T17:25:43.055957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:44.231710+0000","last_change":"2024-09-13T17:25:43.062483+0000","last_active":"2024-09-13T17:25:44.231710+0000","last_peered":"2024-09-13T17:25:44.231710+0000","last_clean":"2024-09-13T17:25:44.231710+0000","last_became_active":"2024-09-13T17:25:43.058145+0000","last_became_peered":"2024-09-13T17:25:43.058145+0000","last_unstale":"2024-09-13T17:25:44.231710+0000","last_undegraded":"2024-09-13T17:25:44.231710+0000","last_fullsized":"2024-09-13T17:25:44.231710+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:10.848768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.576853+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:26:11.576853+0000","last_peered":"2024-09-13T17:26:11.576853+0000","last_clean":"2024-09-13T17:26:11.576853+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:26:11.576853+0000","last_undegraded":"2024-09-13T17:26:11.576853+0000","last_fullsized":"2024-09-13T17:26:11.576853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056991+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:43.056991+0000","last_peered":"2024-09-13T17:25:43.056991+0000","last_clean":"2024-09-13T17:25:43.056991+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:25:43.056991+0000","last_undegraded":"2024-09-13T17:25:43.056991+0000","last_fullsized":"2024-09-13T17:25:43.056991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.051509+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:25:43.051509+0000","last_peered":"2024-09-13T17:25:43.051509+0000","last_clean":"2024-09-13T17:25:43.051509+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:25:43.051509+0000","last_undegraded":"2024-09-13T17:25:43.051509+0000","last_fullsized":"2024-09-13T17:25:43.051509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.569770+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:26:11.569770+0000","last_peered":"2024-09-13T17:26:11.569770+0000","last_clean":"2024-09-13T17:26:11.569770+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:26:11.569770+0000","last_undegraded":"2024-09-13T17:26:11.569770+0000","last_fullsized":"2024-09-13T17:26:11.569770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.055992+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:25:43.055992+0000","last_peered":"2024-09-13T17:25:43.055992+0000","last_clean":"2024-09-13T17:25:43.055992+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:25:43.055992+0000","last_undegraded":"2024-09-13T17:25:43.055992+0000","last_fullsized":"2024-09-13T17:25:43.055992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:36.185390+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:26:36.185390+0000","last_peered":"2024-09-13T17:26:36.185390+0000","last_clean":"2024-09-13T17:26:36.185390+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:26:36.185390+0000","last_undegraded":"2024-09-13T17:26:36.185390+0000","last_fullsized":"2024-09-13T17:26:36.185390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423394+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:26:42.423394+0000","last_peered":"2024-09-13T17:26:42.423394+0000","last_clean":"2024-09-13T17:26:42.423394+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:26:42.423394+0000","last_undegraded":"2024-09-13T17:26:42.423394+0000","last_fullsized":"2024-09-13T17:26:42.423394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351560+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:26:42.351560+0000","last_peered":"2024-09-13T17:26:42.351560+0000","last_clean":"2024-09-13T17:26:42.351560+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:26:42.351560+0000","last_undegraded":"2024-09-13T17:26:42.351560+0000","last_fullsized":"2024-09-13T17:26:42.351560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423943+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:26:42.423943+0000","last_peered":"2024-09-13T17:26:42.423943+0000","last_clean":"2024-09-13T17:26:42.423943+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:26:42.423943+0000","last_undegraded":"2024-09-13T17:26:42.423943+0000","last_fullsized":"2024-09-13T17:26:42.423943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817187+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:26:30.817187+0000","last_peered":"2024-09-13T17:26:30.817187+0000","last_clean":"2024-09-13T17:26:30.817187+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:26:30.817187+0000","last_undegraded":"2024-09-13T17:26:30.817187+0000","last_fullsized":"2024-09-13T17:26:30.817187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351304+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:26:42.351304+0000","last_peered":"2024-09-13T17:26:42.351304+0000","last_clean":"2024-09-13T17:26:42.351304+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:26:42.351304+0000","last_undegraded":"2024-09-13T17:26:42.351304+0000","last_fullsized":"2024-09-13T17:26:42.351304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237712+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:26:34.237712+0000","last_peered":"2024-09-13T17:26:34.237712+0000","last_clean":"2024-09-13T17:26:34.237712+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:26:34.237712+0000","last_undegraded":"2024-09-13T17:26:34.237712+0000","last_fullsized":"2024-09-13T17:26:34.237712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938386+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:26:32.938386+0000","last_peered":"2024-09-13T17:26:32.938386+0000","last_clean":"2024-09-13T17:26:32.938386+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:26:32.938386+0000","last_undegraded":"2024-09-13T17:26:32.938386+0000","last_fullsized":"2024-09-13T17:26:32.938386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.421368+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:26:42.421368+0000","last_peered":"2024-09-13T17:26:42.421368+0000","last_clean":"2024-09-13T17:26:42.421368+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:26:42.421368+0000","last_undegraded":"2024-09-13T17:26:42.421368+0000","last_fullsized":"2024-09-13T17:26:42.421368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.149653+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:26:34.149653+0000","last_peered":"2024-09-13T17:26:34.149653+0000","last_clean":"2024-09-13T17:26:34.149653+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:26:34.149653+0000","last_undegraded":"2024-09-13T17:26:34.149653+0000","last_fullsized":"2024-09-13T17:26:34.149653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.951691+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:26:33.951691+0000","last_peered":"2024-09-13T17:26:33.951691+0000","last_clean":"2024-09-13T17:26:33.951691+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:26:33.951691+0000","last_undegraded":"2024-09-13T17:26:33.951691+0000","last_fullsized":"2024-09-13T17:26:33.951691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351389+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:26:42.351389+0000","last_peered":"2024-09-13T17:26:42.351389+0000","last_clean":"2024-09-13T17:26:42.351389+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:26:42.351389+0000","last_undegraded":"2024-09-13T17:26:42.351389+0000","last_fullsized":"2024-09-13T17:26:42.351389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.918018+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:26:35.918018+0000","last_peered":"2024-09-13T17:26:35.918018+0000","last_clean":"2024-09-13T17:26:35.918018+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:26:35.918018+0000","last_undegraded":"2024-09-13T17:26:35.918018+0000","last_fullsized":"2024-09-13T17:26:35.918018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:31.909648+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:26:31.909648+0000","last_peered":"2024-09-13T17:26:31.909648+0000","last_clean":"2024-09-13T17:26:31.909648+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:26:31.909648+0000","last_undegraded":"2024-09-13T17:26:31.909648+0000","last_fullsized":"2024-09-13T17:26:31.909648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817100+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:26:30.817100+0000","last_peered":"2024-09-13T17:26:30.817100+0000","last_clean":"2024-09-13T17:26:30.817100+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:26:30.817100+0000","last_undegraded":"2024-09-13T17:26:30.817100+0000","last_fullsized":"2024-09-13T17:26:30.817100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938437+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:26:32.938437+0000","last_peered":"2024-09-13T17:26:32.938437+0000","last_clean":"2024-09-13T17:26:32.938437+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:26:32.938437+0000","last_undegraded":"2024-09-13T17:26:32.938437+0000","last_fullsized":"2024-09-13T17:26:32.938437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'41","reported_seq":86,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.537171+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:26:42.537171+0000","last_peered":"2024-09-13T17:26:42.537171+0000","last_clean":"2024-09-13T17:26:42.537171+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:26:42.537171+0000","last_undegraded":"2024-09-13T17:26:42.537171+0000","last_fullsized":"2024-09-13T17:26:42.537171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":73,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:27.937937+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:26:27.937937+0000","last_peered":"2024-09-13T17:26:27.937937+0000","last_clean":"2024-09-13T17:26:27.937937+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:26:27.937937+0000","last_undegraded":"2024-09-13T17:26:27.937937+0000","last_fullsized":"2024-09-13T17:26:27.937937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:28.129039+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:26:28.129039+0000","last_peered":"2024-09-13T17:26:28.129039+0000","last_clean":"2024-09-13T17:26:28.129039+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:26:28.129039+0000","last_undegraded":"2024-09-13T17:26:28.129039+0000","last_fullsized":"2024-09-13T17:26:28.129039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.053691+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:25:43.053691+0000","last_peered":"2024-09-13T17:25:43.053691+0000","last_clean":"2024-09-13T17:25:43.053691+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:25:43.053691+0000","last_undegraded":"2024-09-13T17:25:43.053691+0000","last_fullsized":"2024-09-13T17:25:43.053691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237734+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:26:34.237734+0000","last_peered":"2024-09-13T17:26:34.237734+0000","last_clean":"2024-09-13T17:26:34.237734+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:26:34.237734+0000","last_undegraded":"2024-09-13T17:26:34.237734+0000","last_fullsized":"2024-09-13T17:26:34.237734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.981541+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:26:29.981541+0000","last_peered":"2024-09-13T17:26:29.981541+0000","last_clean":"2024-09-13T17:26:29.981541+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:26:29.981541+0000","last_undegraded":"2024-09-13T17:26:29.981541+0000","last_fullsized":"2024-09-13T17:26:29.981541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.554849+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:26:25.554849+0000","last_peered":"2024-09-13T17:26:25.554849+0000","last_clean":"2024-09-13T17:26:25.554849+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:26:25.554849+0000","last_undegraded":"2024-09-13T17:26:25.554849+0000","last_fullsized":"2024-09-13T17:26:25.554849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.149669+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:26:34.149669+0000","last_peered":"2024-09-13T17:26:34.149669+0000","last_clean":"2024-09-13T17:26:34.149669+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:26:34.149669+0000","last_undegraded":"2024-09-13T17:26:34.149669+0000","last_fullsized":"2024-09-13T17:26:34.149669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.981522+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:26:29.981522+0000","last_peered":"2024-09-13T17:26:29.981522+0000","last_clean":"2024-09-13T17:26:29.981522+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:26:29.981522+0000","last_undegraded":"2024-09-13T17:26:29.981522+0000","last_fullsized":"2024-09-13T17:26:29.981522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:26.184106+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:26:26.184106+0000","last_peered":"2024-09-13T17:26:26.184106+0000","last_clean":"2024-09-13T17:26:26.184106+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:26:26.184106+0000","last_undegraded":"2024-09-13T17:26:26.184106+0000","last_fullsized":"2024-09-13T17:26:26.184106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.129425+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:26:33.129425+0000","last_peered":"2024-09-13T17:26:33.129425+0000","last_clean":"2024-09-13T17:26:33.129425+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:26:33.129425+0000","last_undegraded":"2024-09-13T17:26:33.129425+0000","last_fullsized":"2024-09-13T17:26:33.129425+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:24.730697+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:26:24.730697+0000","last_peered":"2024-09-13T17:26:24.730697+0000","last_clean":"2024-09-13T17:26:24.730697+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:26:24.730697+0000","last_undegraded":"2024-09-13T17:26:24.730697+0000","last_fullsized":"2024-09-13T17:26:24.730697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350284+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:26:42.350284+0000","last_peered":"2024-09-13T17:26:42.350284+0000","last_clean":"2024-09-13T17:26:42.350284+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:26:42.350284+0000","last_undegraded":"2024-09-13T17:26:42.350284+0000","last_fullsized":"2024-09-13T17:26:42.350284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350448+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:26:42.350448+0000","last_peered":"2024-09-13T17:26:42.350448+0000","last_clean":"2024-09-13T17:26:42.350448+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:26:42.350448+0000","last_undegraded":"2024-09-13T17:26:42.350448+0000","last_fullsized":"2024-09-13T17:26:42.350448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:28.128970+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:26:28.128970+0000","last_peered":"2024-09-13T17:26:28.128970+0000","last_clean":"2024-09-13T17:26:28.128970+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:26:28.128970+0000","last_undegraded":"2024-09-13T17:26:28.128970+0000","last_fullsized":"2024-09-13T17:26:28.128970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:21.183486+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:26:21.183486+0000","last_peered":"2024-09-13T17:26:21.183486+0000","last_clean":"2024-09-13T17:26:21.183486+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:26:21.183486+0000","last_undegraded":"2024-09-13T17:26:21.183486+0000","last_fullsized":"2024-09-13T17:26:21.183486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351357+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:26:42.351357+0000","last_peered":"2024-09-13T17:26:42.351357+0000","last_clean":"2024-09-13T17:26:42.351357+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:26:42.351357+0000","last_undegraded":"2024-09-13T17:26:42.351357+0000","last_fullsized":"2024-09-13T17:26:42.351357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:27.937986+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:26:27.937986+0000","last_peered":"2024-09-13T17:26:27.937986+0000","last_clean":"2024-09-13T17:26:27.937986+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:26:27.937986+0000","last_undegraded":"2024-09-13T17:26:27.937986+0000","last_fullsized":"2024-09-13T17:26:27.937986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:26.909107+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:26:26.909107+0000","last_peered":"2024-09-13T17:26:26.909107+0000","last_clean":"2024-09-13T17:26:26.909107+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:26:26.909107+0000","last_undegraded":"2024-09-13T17:26:26.909107+0000","last_fullsized":"2024-09-13T17:26:26.909107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:01.181134+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:26:01.181134+0000","last_peered":"2024-09-13T17:26:01.181134+0000","last_clean":"2024-09-13T17:26:01.181134+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:26:01.181134+0000","last_undegraded":"2024-09-13T17:26:01.181134+0000","last_fullsized":"2024-09-13T17:26:01.181134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:31.184745+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:26:31.184745+0000","last_peered":"2024-09-13T17:26:31.184745+0000","last_clean":"2024-09-13T17:26:31.184745+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:26:31.184745+0000","last_undegraded":"2024-09-13T17:26:31.184745+0000","last_fullsized":"2024-09-13T17:26:31.184745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501870+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:26:42.501870+0000","last_peered":"2024-09-13T17:26:42.501870+0000","last_clean":"2024-09-13T17:26:42.501870+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:26:42.501870+0000","last_undegraded":"2024-09-13T17:26:42.501870+0000","last_fullsized":"2024-09-13T17:26:42.501870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.817900+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:26:35.817900+0000","last_peered":"2024-09-13T17:26:35.817900+0000","last_clean":"2024-09-13T17:26:35.817900+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:26:35.817900+0000","last_undegraded":"2024-09-13T17:26:35.817900+0000","last_fullsized":"2024-09-13T17:26:35.817900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500602+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:26:42.500602+0000","last_peered":"2024-09-13T17:26:42.500602+0000","last_clean":"2024-09-13T17:26:42.500602+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:26:42.500602+0000","last_undegraded":"2024-09-13T17:26:42.500602+0000","last_fullsized":"2024-09-13T17:26:42.500602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501617+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:26:42.501617+0000","last_peered":"2024-09-13T17:26:42.501617+0000","last_clean":"2024-09-13T17:26:42.501617+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:26:42.501617+0000","last_undegraded":"2024-09-13T17:26:42.501617+0000","last_fullsized":"2024-09-13T17:26:42.501617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.540986+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:26:42.540986+0000","last_peered":"2024-09-13T17:26:42.540986+0000","last_clean":"2024-09-13T17:26:42.540986+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:26:42.540986+0000","last_undegraded":"2024-09-13T17:26:42.540986+0000","last_fullsized":"2024-09-13T17:26:42.540986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.503498+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:26:42.503498+0000","last_peered":"2024-09-13T17:26:42.503498+0000","last_clean":"2024-09-13T17:26:42.503498+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:26:42.503498+0000","last_undegraded":"2024-09-13T17:26:42.503498+0000","last_fullsized":"2024-09-13T17:26:42.503498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501867+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:26:42.501867+0000","last_peered":"2024-09-13T17:26:42.501867+0000","last_clean":"2024-09-13T17:26:42.501867+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:26:42.501867+0000","last_undegraded":"2024-09-13T17:26:42.501867+0000","last_fullsized":"2024-09-13T17:26:42.501867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350561+0000","last_change":"2024-09-13T17:25:43.395166+0000","last_active":"2024-09-13T17:26:42.350561+0000","last_peered":"2024-09-13T17:26:42.350561+0000","last_clean":"2024-09-13T17:26:42.350561+0000","last_became_active":"2024-09-13T17:25:43.392868+0000","last_became_peered":"2024-09-13T17:25:43.392868+0000","last_unstale":"2024-09-13T17:26:42.350561+0000","last_undegraded":"2024-09-13T17:26:42.350561+0000","last_fullsized":"2024-09-13T17:26:42.350561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:24.860809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350994+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:26:42.350994+0000","last_peered":"2024-09-13T17:26:42.350994+0000","last_clean":"2024-09-13T17:26:42.350994+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:26:42.350994+0000","last_undegraded":"2024-09-13T17:26:42.350994+0000","last_fullsized":"2024-09-13T17:26:42.350994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.502163+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:26:42.502163+0000","last_peered":"2024-09-13T17:26:42.502163+0000","last_clean":"2024-09-13T17:26:42.502163+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:26:42.502163+0000","last_undegraded":"2024-09-13T17:26:42.502163+0000","last_fullsized":"2024-09-13T17:26:42.502163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423700+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:26:42.423700+0000","last_peered":"2024-09-13T17:26:42.423700+0000","last_clean":"2024-09-13T17:26:42.423700+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:26:42.423700+0000","last_undegraded":"2024-09-13T17:26:42.423700+0000","last_fullsized":"2024-09-13T17:26:42.423700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351063+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:26:42.351063+0000","last_peered":"2024-09-13T17:26:42.351063+0000","last_clean":"2024-09-13T17:26:42.351063+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:26:42.351063+0000","last_undegraded":"2024-09-13T17:26:42.351063+0000","last_fullsized":"2024-09-13T17:26:42.351063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351181+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:26:42.351181+0000","last_peered":"2024-09-13T17:26:42.351181+0000","last_clean":"2024-09-13T17:26:42.351181+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:26:42.351181+0000","last_undegraded":"2024-09-13T17:26:42.351181+0000","last_fullsized":"2024-09-13T17:26:42.351181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423263+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:26:42.423263+0000","last_peered":"2024-09-13T17:26:42.423263+0000","last_clean":"2024-09-13T17:26:42.423263+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:26:42.423263+0000","last_undegraded":"2024-09-13T17:26:42.423263+0000","last_fullsized":"2024-09-13T17:26:42.423263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.054174+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:25:43.054174+0000","last_peered":"2024-09-13T17:25:43.054174+0000","last_clean":"2024-09-13T17:25:43.054174+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:25:43.054174+0000","last_undegraded":"2024-09-13T17:25:43.054174+0000","last_fullsized":"2024-09-13T17:25:43.054174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501336+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:26:42.501336+0000","last_peered":"2024-09-13T17:26:42.501336+0000","last_clean":"2024-09-13T17:26:42.501336+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:26:42.501336+0000","last_undegraded":"2024-09-13T17:26:42.501336+0000","last_fullsized":"2024-09-13T17:26:42.501336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:43.351080+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:26:43.351080+0000","last_peered":"2024-09-13T17:26:43.351080+0000","last_clean":"2024-09-13T17:26:43.351080+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:26:43.351080+0000","last_undegraded":"2024-09-13T17:26:43.351080+0000","last_fullsized":"2024-09-13T17:26:43.351080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1959,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.051664+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:25:43.051664+0000","last_peered":"2024-09-13T17:25:43.051664+0000","last_clean":"2024-09-13T17:25:43.051664+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:25:43.051664+0000","last_undegraded":"2024-09-13T17:25:43.051664+0000","last_fullsized":"2024-09-13T17:25:43.051664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.567360+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:26:11.567360+0000","last_peered":"2024-09-13T17:26:11.567360+0000","last_clean":"2024-09-13T17:26:11.567360+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:26:11.567360+0000","last_undegraded":"2024-09-13T17:26:11.567360+0000","last_fullsized":"2024-09-13T17:26:11.567360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351446+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:26:42.351446+0000","last_peered":"2024-09-13T17:26:42.351446+0000","last_clean":"2024-09-13T17:26:42.351446+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:26:42.351446+0000","last_undegraded":"2024-09-13T17:26:42.351446+0000","last_fullsized":"2024-09-13T17:26:42.351446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.053590+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:43.053590+0000","last_peered":"2024-09-13T17:25:43.053590+0000","last_clean":"2024-09-13T17:25:43.053590+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:25:43.053590+0000","last_undegraded":"2024-09-13T17:25:43.053590+0000","last_fullsized":"2024-09-13T17:25:43.053590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350518+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:26:42.350518+0000","last_peered":"2024-09-13T17:26:42.350518+0000","last_clean":"2024-09-13T17:26:42.350518+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:26:42.350518+0000","last_undegraded":"2024-09-13T17:26:42.350518+0000","last_fullsized":"2024-09-13T17:26:42.350518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501769+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:26:42.501769+0000","last_peered":"2024-09-13T17:26:42.501769+0000","last_clean":"2024-09-13T17:26:42.501769+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:26:42.501769+0000","last_undegraded":"2024-09-13T17:26:42.501769+0000","last_fullsized":"2024-09-13T17:26:42.501769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.237179+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:25:43.237179+0000","last_peered":"2024-09-13T17:25:43.237179+0000","last_clean":"2024-09-13T17:25:43.237179+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:25:43.237179+0000","last_undegraded":"2024-09-13T17:25:43.237179+0000","last_fullsized":"2024-09-13T17:25:43.237179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.560671+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:26:11.560671+0000","last_peered":"2024-09-13T17:26:11.560671+0000","last_clean":"2024-09-13T17:26:11.560671+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:26:11.560671+0000","last_undegraded":"2024-09-13T17:26:11.560671+0000","last_fullsized":"2024-09-13T17:26:11.560671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'40","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:43.319859+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:26:43.319859+0000","last_peered":"2024-09-13T17:26:43.319859+0000","last_clean":"2024-09-13T17:26:43.319859+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:26:43.319859+0000","last_undegraded":"2024-09-13T17:26:43.319859+0000","last_fullsized":"2024-09-13T17:26:43.319859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68128,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":191,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.049487+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:25:43.049487+0000","last_peered":"2024-09-13T17:25:43.049487+0000","last_clean":"2024-09-13T17:25:43.049487+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:25:43.049487+0000","last_undegraded":"2024-09-13T17:25:43.049487+0000","last_fullsized":"2024-09-13T17:25:43.049487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.577017+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:26:11.577017+0000","last_peered":"2024-09-13T17:26:11.577017+0000","last_clean":"2024-09-13T17:26:11.577017+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:26:11.577017+0000","last_undegraded":"2024-09-13T17:26:11.577017+0000","last_fullsized":"2024-09-13T17:26:11.577017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.237369+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:25:43.237369+0000","last_peered":"2024-09-13T17:25:43.237369+0000","last_clean":"2024-09-13T17:25:43.237369+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:25:43.237369+0000","last_undegraded":"2024-09-13T17:25:43.237369+0000","last_fullsized":"2024-09-13T17:25:43.237369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056840+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:25:43.056840+0000","last_peered":"2024-09-13T17:25:43.056840+0000","last_clean":"2024-09-13T17:25:43.056840+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:25:43.056840+0000","last_undegraded":"2024-09-13T17:25:43.056840+0000","last_fullsized":"2024-09-13T17:25:43.056840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.568212+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:26:11.568212+0000","last_peered":"2024-09-13T17:26:11.568212+0000","last_clean":"2024-09-13T17:26:11.568212+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:26:11.568212+0000","last_undegraded":"2024-09-13T17:26:11.568212+0000","last_fullsized":"2024-09-13T17:26:11.568212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501742+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:26:42.501742+0000","last_peered":"2024-09-13T17:26:42.501742+0000","last_clean":"2024-09-13T17:26:42.501742+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:26:42.501742+0000","last_undegraded":"2024-09-13T17:26:42.501742+0000","last_fullsized":"2024-09-13T17:26:42.501742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.420590+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:26:42.420590+0000","last_peered":"2024-09-13T17:26:42.420590+0000","last_clean":"2024-09-13T17:26:42.420590+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:26:42.420590+0000","last_undegraded":"2024-09-13T17:26:42.420590+0000","last_fullsized":"2024-09-13T17:26:42.420590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572870+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:26:11.572870+0000","last_peered":"2024-09-13T17:26:11.572870+0000","last_clean":"2024-09-13T17:26:11.572870+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:26:11.572870+0000","last_undegraded":"2024-09-13T17:26:11.572870+0000","last_fullsized":"2024-09-13T17:26:11.572870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":143,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.432692+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:26:42.432692+0000","last_peered":"2024-09-13T17:26:42.432692+0000","last_clean":"2024-09-13T17:26:42.432692+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:26:42.432692+0000","last_undegraded":"2024-09-13T17:26:42.432692+0000","last_fullsized":"2024-09-13T17:26:42.432692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056274+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:25:43.056274+0000","last_peered":"2024-09-13T17:25:43.056274+0000","last_clean":"2024-09-13T17:25:43.056274+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:25:43.056274+0000","last_undegraded":"2024-09-13T17:25:43.056274+0000","last_fullsized":"2024-09-13T17:25:43.056274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.917114+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:26:25.917114+0000","last_peered":"2024-09-13T17:26:25.917114+0000","last_clean":"2024-09-13T17:26:25.917114+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:26:25.917114+0000","last_undegraded":"2024-09-13T17:26:25.917114+0000","last_fullsized":"2024-09-13T17:26:25.917114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.564988+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:26:11.564988+0000","last_peered":"2024-09-13T17:26:11.564988+0000","last_clean":"2024-09-13T17:26:11.564988+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:26:11.564988+0000","last_undegraded":"2024-09-13T17:26:11.564988+0000","last_fullsized":"2024-09-13T17:26:11.564988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.236896+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:25:43.236896+0000","last_peered":"2024-09-13T17:25:43.236896+0000","last_clean":"2024-09-13T17:25:43.236896+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:25:43.236896+0000","last_undegraded":"2024-09-13T17:25:43.236896+0000","last_fullsized":"2024-09-13T17:25:43.236896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237862+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:26:34.237862+0000","last_peered":"2024-09-13T17:26:34.237862+0000","last_clean":"2024-09-13T17:26:34.237862+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:26:34.237862+0000","last_undegraded":"2024-09-13T17:26:34.237862+0000","last_fullsized":"2024-09-13T17:26:34.237862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.564967+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:26:11.564967+0000","last_peered":"2024-09-13T17:26:11.564967+0000","last_clean":"2024-09-13T17:26:11.564967+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:26:11.564967+0000","last_undegraded":"2024-09-13T17:26:11.564967+0000","last_fullsized":"2024-09-13T17:26:11.564967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.054105+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:25:43.054105+0000","last_peered":"2024-09-13T17:25:43.054105+0000","last_clean":"2024-09-13T17:25:43.054105+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:25:43.054105+0000","last_undegraded":"2024-09-13T17:25:43.054105+0000","last_fullsized":"2024-09-13T17:25:43.054105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.050335+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:25:43.050335+0000","last_peered":"2024-09-13T17:25:43.050335+0000","last_clean":"2024-09-13T17:25:43.050335+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:25:43.050335+0000","last_undegraded":"2024-09-13T17:25:43.050335+0000","last_fullsized":"2024-09-13T17:25:43.050335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572729+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:26:11.572729+0000","last_peered":"2024-09-13T17:26:11.572729+0000","last_clean":"2024-09-13T17:26:11.572729+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:26:11.572729+0000","last_undegraded":"2024-09-13T17:26:11.572729+0000","last_fullsized":"2024-09-13T17:26:11.572729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056930+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:25:43.056930+0000","last_peered":"2024-09-13T17:25:43.056930+0000","last_clean":"2024-09-13T17:25:43.056930+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:25:43.056930+0000","last_undegraded":"2024-09-13T17:25:43.056930+0000","last_fullsized":"2024-09-13T17:25:43.056930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351902+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:26:42.351902+0000","last_peered":"2024-09-13T17:26:42.351902+0000","last_clean":"2024-09-13T17:26:42.351902+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:26:42.351902+0000","last_undegraded":"2024-09-13T17:26:42.351902+0000","last_fullsized":"2024-09-13T17:26:42.351902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.557262+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:26:11.557262+0000","last_peered":"2024-09-13T17:26:11.557262+0000","last_clean":"2024-09-13T17:26:11.557262+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:26:11.557262+0000","last_undegraded":"2024-09-13T17:26:11.557262+0000","last_fullsized":"2024-09-13T17:26:11.557262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.057073+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:25:43.057073+0000","last_peered":"2024-09-13T17:25:43.057073+0000","last_clean":"2024-09-13T17:25:43.057073+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:25:43.057073+0000","last_undegraded":"2024-09-13T17:25:43.057073+0000","last_fullsized":"2024-09-13T17:25:43.057073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.236979+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:26:29.236979+0000","last_peered":"2024-09-13T17:26:29.236979+0000","last_clean":"2024-09-13T17:26:29.236979+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:26:29.236979+0000","last_undegraded":"2024-09-13T17:26:29.236979+0000","last_fullsized":"2024-09-13T17:26:29.236979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817145+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:26:30.817145+0000","last_peered":"2024-09-13T17:26:30.817145+0000","last_clean":"2024-09-13T17:26:30.817145+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:26:30.817145+0000","last_undegraded":"2024-09-13T17:26:30.817145+0000","last_fullsized":"2024-09-13T17:26:30.817145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817167+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:26:30.817167+0000","last_peered":"2024-09-13T17:26:30.817167+0000","last_clean":"2024-09-13T17:26:30.817167+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:26:30.817167+0000","last_undegraded":"2024-09-13T17:26:30.817167+0000","last_fullsized":"2024-09-13T17:26:30.817167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917538+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:26:30.917538+0000","last_peered":"2024-09-13T17:26:30.917538+0000","last_clean":"2024-09-13T17:26:30.917538+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:26:30.917538+0000","last_undegraded":"2024-09-13T17:26:30.917538+0000","last_fullsized":"2024-09-13T17:26:30.917538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.129438+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:26:33.129438+0000","last_peered":"2024-09-13T17:26:33.129438+0000","last_clean":"2024-09-13T17:26:33.129438+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:26:33.129438+0000","last_undegraded":"2024-09-13T17:26:33.129438+0000","last_fullsized":"2024-09-13T17:26:33.129438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237831+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:26:34.237831+0000","last_peered":"2024-09-13T17:26:34.237831+0000","last_clean":"2024-09-13T17:26:34.237831+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:26:34.237831+0000","last_undegraded":"2024-09-13T17:26:34.237831+0000","last_fullsized":"2024-09-13T17:26:34.237831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.817949+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:26:35.817949+0000","last_peered":"2024-09-13T17:26:35.817949+0000","last_clean":"2024-09-13T17:26:35.817949+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:26:35.817949+0000","last_undegraded":"2024-09-13T17:26:35.817949+0000","last_fullsized":"2024-09-13T17:26:35.817949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938425+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:26:32.938425+0000","last_peered":"2024-09-13T17:26:32.938425+0000","last_clean":"2024-09-13T17:26:32.938425+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:26:32.938425+0000","last_undegraded":"2024-09-13T17:26:32.938425+0000","last_fullsized":"2024-09-13T17:26:32.938425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.731999+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:26:34.731999+0000","last_peered":"2024-09-13T17:26:34.731999+0000","last_clean":"2024-09-13T17:26:34.731999+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:26:34.731999+0000","last_undegraded":"2024-09-13T17:26:34.731999+0000","last_fullsized":"2024-09-13T17:26:34.731999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.424680+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:26:42.424680+0000","last_peered":"2024-09-13T17:26:42.424680+0000","last_clean":"2024-09-13T17:26:42.424680+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:26:42.424680+0000","last_undegraded":"2024-09-13T17:26:42.424680+0000","last_fullsized":"2024-09-13T17:26:42.424680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500772+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:26:42.500772+0000","last_peered":"2024-09-13T17:26:42.500772+0000","last_clean":"2024-09-13T17:26:42.500772+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:26:42.500772+0000","last_undegraded":"2024-09-13T17:26:42.500772+0000","last_fullsized":"2024-09-13T17:26:42.500772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350799+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:26:42.350799+0000","last_peered":"2024-09-13T17:26:42.350799+0000","last_clean":"2024-09-13T17:26:42.350799+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:26:42.350799+0000","last_undegraded":"2024-09-13T17:26:42.350799+0000","last_fullsized":"2024-09-13T17:26:42.350799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501815+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:26:42.501815+0000","last_peered":"2024-09-13T17:26:42.501815+0000","last_clean":"2024-09-13T17:26:42.501815+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:26:42.501815+0000","last_undegraded":"2024-09-13T17:26:42.501815+0000","last_fullsized":"2024-09-13T17:26:42.501815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501512+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:26:42.501512+0000","last_peered":"2024-09-13T17:26:42.501512+0000","last_clean":"2024-09-13T17:26:42.501512+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:26:42.501512+0000","last_undegraded":"2024-09-13T17:26:42.501512+0000","last_fullsized":"2024-09-13T17:26:42.501512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.421423+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:26:42.421423+0000","last_peered":"2024-09-13T17:26:42.421423+0000","last_clean":"2024-09-13T17:26:42.421423+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:26:42.421423+0000","last_undegraded":"2024-09-13T17:26:42.421423+0000","last_fullsized":"2024-09-13T17:26:42.421423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.502291+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:26:42.502291+0000","last_peered":"2024-09-13T17:26:42.502291+0000","last_clean":"2024-09-13T17:26:42.502291+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:26:42.502291+0000","last_undegraded":"2024-09-13T17:26:42.502291+0000","last_fullsized":"2024-09-13T17:26:42.502291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500771+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:26:42.500771+0000","last_peered":"2024-09-13T17:26:42.500771+0000","last_clean":"2024-09-13T17:26:42.500771+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:26:42.500771+0000","last_undegraded":"2024-09-13T17:26:42.500771+0000","last_fullsized":"2024-09-13T17:26:42.500771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.566896+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:26:11.566896+0000","last_peered":"2024-09-13T17:26:11.566896+0000","last_clean":"2024-09-13T17:26:11.566896+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:26:11.566896+0000","last_undegraded":"2024-09-13T17:26:11.566896+0000","last_fullsized":"2024-09-13T17:26:11.566896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.569146+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:26:11.569146+0000","last_peered":"2024-09-13T17:26:11.569146+0000","last_clean":"2024-09-13T17:26:11.569146+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:26:11.569146+0000","last_undegraded":"2024-09-13T17:26:11.569146+0000","last_fullsized":"2024-09-13T17:26:11.569146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.562194+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:26:11.562194+0000","last_peered":"2024-09-13T17:26:11.562194+0000","last_clean":"2024-09-13T17:26:11.562194+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:26:11.562194+0000","last_undegraded":"2024-09-13T17:26:11.562194+0000","last_fullsized":"2024-09-13T17:26:11.562194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.543425+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:26:11.543425+0000","last_peered":"2024-09-13T17:26:11.543425+0000","last_clean":"2024-09-13T17:26:11.543425+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:26:11.543425+0000","last_undegraded":"2024-09-13T17:26:11.543425+0000","last_fullsized":"2024-09-13T17:26:11.543425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.575431+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:26:11.575431+0000","last_peered":"2024-09-13T17:26:11.575431+0000","last_clean":"2024-09-13T17:26:11.575431+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:26:11.575431+0000","last_undegraded":"2024-09-13T17:26:11.575431+0000","last_fullsized":"2024-09-13T17:26:11.575431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.561159+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:26:11.561159+0000","last_peered":"2024-09-13T17:26:11.561159+0000","last_clean":"2024-09-13T17:26:11.561159+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:26:11.561159+0000","last_undegraded":"2024-09-13T17:26:11.561159+0000","last_fullsized":"2024-09-13T17:26:11.561159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.576704+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:26:11.576704+0000","last_peered":"2024-09-13T17:26:11.576704+0000","last_clean":"2024-09-13T17:26:11.576704+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:26:11.576704+0000","last_undegraded":"2024-09-13T17:26:11.576704+0000","last_fullsized":"2024-09-13T17:26:11.576704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572750+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:26:11.572750+0000","last_peered":"2024-09-13T17:26:11.572750+0000","last_clean":"2024-09-13T17:26:11.572750+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:26:11.572750+0000","last_undegraded":"2024-09-13T17:26:11.572750+0000","last_fullsized":"2024-09-13T17:26:11.572750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.573518+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:26:11.573518+0000","last_peered":"2024-09-13T17:26:11.573518+0000","last_clean":"2024-09-13T17:26:11.573518+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:26:11.573518+0000","last_undegraded":"2024-09-13T17:26:11.573518+0000","last_fullsized":"2024-09-13T17:26:11.573518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.556715+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:26:11.556715+0000","last_peered":"2024-09-13T17:26:11.556715+0000","last_clean":"2024-09-13T17:26:11.556715+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:26:11.556715+0000","last_undegraded":"2024-09-13T17:26:11.556715+0000","last_fullsized":"2024-09-13T17:26:11.556715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.565559+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:26:11.565559+0000","last_peered":"2024-09-13T17:26:11.565559+0000","last_clean":"2024-09-13T17:26:11.565559+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:26:11.565559+0000","last_undegraded":"2024-09-13T17:26:11.565559+0000","last_fullsized":"2024-09-13T17:26:11.565559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.571323+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:26:11.571323+0000","last_peered":"2024-09-13T17:26:11.571323+0000","last_clean":"2024-09-13T17:26:11.571323+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:26:11.571323+0000","last_undegraded":"2024-09-13T17:26:11.571323+0000","last_fullsized":"2024-09-13T17:26:11.571323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917553+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:26:30.917553+0000","last_peered":"2024-09-13T17:26:30.917553+0000","last_clean":"2024-09-13T17:26:30.917553+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:26:30.917553+0000","last_undegraded":"2024-09-13T17:26:30.917553+0000","last_fullsized":"2024-09-13T17:26:30.917553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917595+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:26:30.917595+0000","last_peered":"2024-09-13T17:26:30.917595+0000","last_clean":"2024-09-13T17:26:30.917595+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:26:30.917595+0000","last_undegraded":"2024-09-13T17:26:30.917595+0000","last_fullsized":"2024-09-13T17:26:30.917595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.237162+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:26:29.237162+0000","last_peered":"2024-09-13T17:26:29.237162+0000","last_clean":"2024-09-13T17:26:29.237162+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:26:29.237162+0000","last_undegraded":"2024-09-13T17:26:29.237162+0000","last_fullsized":"2024-09-13T17:26:29.237162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598371,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":832,"num_read_kb":528,"num_write":282,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1795113,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108906,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416424,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":524981,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70699999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.57899999999999996,"5min":0.745,"15min":0.745},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.30399999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.81699999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.371},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61499999999999999},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82899999999999996,"5min":1.05,"15min":1.05},"last":0.35799999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85199999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86299999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.53700000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.97099999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.50800000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1196032,"data_stored":1317091,"data_compressed":68931,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56699999999999995,"15min":0.57399999999999995},"min":{"1min":0.35499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.435},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.438,"15min":0.438},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75600000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.58799999999999997},"min":{"1min":0.38200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74399999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.376}]},{"osd":5,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.503}]},{"osd":6,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":1.073,"15min":1.073},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.91300000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.496}]},{"osd":9,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70399999999999996,"15min":0.72199999999999998},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88600000000000001,"5min":1.147,"15min":1.147},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.52700000000000002}]},{"osd":11,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82399999999999995,"5min":1.175,"15min":1.175},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91700000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.54600000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":918411,"data_compressed":45619,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45500000000000002,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64200000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.49299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68200000000000005,"5min":0.999,"15min":0.999},"last":0.24399999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.852,"5min":13.852,"15min":13.852},"last":1.04}]},{"osd":2,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":0.88300000000000001,"15min":0.77400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":17.055,"5min":17.055,"15min":17.055},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61399999999999999,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57699999999999996,"15min":0.623},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.77700000000000002,"15min":0.71699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.074999999999999,"5min":17.074999999999999,"15min":17.074999999999999},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67200000000000004,"15min":0.70899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88700000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.42199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.753,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":1.054,"15min":1.054},"last":1.1519999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86499999999999999,"5min":1.083,"15min":1.083},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.71799999999999997,"15min":0.75600000000000001},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93700000000000006,"5min":1.04,"15min":1.04},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.006,"5min":1.288,"15min":1.288},"last":0.69899999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":0.83099999999999996,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":17.100999999999999,"5min":17.100999999999999,"15min":17.100999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.78800000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103223,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":917978,"data_compressed":45619,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.83899999999999997,"15min":0.97499999999999998},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.5679999999999996,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":0.68500000000000005,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":16.945,"5min":16.945,"15min":16.945},"last":0.372}]},{"osd":1,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":0.77400000000000002,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.544,"5min":22.544,"15min":22.544},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.59299999999999997,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.56399999999999995,"15min":0.46800000000000003},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":15.054,"5min":15.054,"15min":15.054},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.125,"5min":0.56699999999999995,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":14.135999999999999,"5min":14.135999999999999,"15min":14.135999999999999},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":0.73099999999999998,"15min":0.61499999999999999},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":19.047000000000001,"5min":19.047000000000001,"15min":19.047000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.778,"5min":3.778,"15min":3.778},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.61899999999999999,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.3220000000000001,"5min":7.3220000000000001,"15min":7.3220000000000001},"last":0.496},{"interface":"front","average":{"1min":1.077,"5min":0.68700000000000006,"15min":0.62},"min":{"1min":0.50800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":9.7200000000000006,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.8170000000000002,"5min":3.8170000000000002,"15min":3.8170000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":0.68300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":16.911000000000001,"5min":16.911000000000001,"15min":16.911000000000001},"last":0.436}]},{"osd":6,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.7280000000000002,"5min":3.7280000000000002,"15min":3.7280000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.151,"5min":0.754,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.5979999999999999,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.71999999999999997,"15min":0.66000000000000003},"min":{"1min":0.51500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7279999999999998,"5min":6.7279999999999998,"15min":6.7279999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":0.80400000000000005,"15min":0.68799999999999994},"min":{"1min":0.52800000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":16.847000000000001,"5min":16.847000000000001,"15min":16.847000000000001},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 17:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.73099999999999998,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.44,"15min":0.44},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.58599999999999997}]},{"osd":10,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.57499999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.57299999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.82099999999999995,"15min":0.80300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.9239999999999999,"5min":3.9239999999999999,"15min":3.9239999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.101,"5min":0.81599999999999995,"15min":0.76800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.9370000000000003,"5min":7.9370000000000003,"15min":7.9370000000000003},"last":0.59599999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":1089536,"data_stored":1122450,"data_compressed":62775,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71699999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.39800000000000002,"15min":0.379},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.313}]},{"osd":2,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.439}]},{"osd":4,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45400000000000001,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":0.75,"15min":0.75},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.34399999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80600000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.746,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83999999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41699999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.627},"min":{"1min":0.53200000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47799999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52100000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84999999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.502}]},{"osd":11,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.67100000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59099999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":520371,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51800000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.314},{"interface":"front","average":{"1min":0.372,"5min":0.39400000000000002,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.45900000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.751,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76800000000000002,"5min":1.008,"15min":1.008},"last":0.441}]},{"osd":2,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73299999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.49199999999999999},"min":{"1min":0.36399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67300000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.66600000000000004,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.623},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":1.018,"15min":1.018},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91900000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.52200000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.046,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53100000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90200000000000002,"5min":1.298,"15min":1.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.139,"15min":1.139},"last":0.65900000000000003}]},{"osd":11,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.61099999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.93799999999999994,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.77300000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.745}]}]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1118208,"data_stored":1248267,"data_compressed":68931,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.502,"15min":0.47699999999999998},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.39800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76200000000000001,"5min":1.415,"15min":1.415},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.35999999999999999},"min":{"1min":0.28699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48999999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.36899999999999999},"min":{"1min":0.28199999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.496,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54200000000000004,"15min":0.48699999999999999},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":1.522,"15min":1.522},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.42499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90000000000000002,"5min":1.3,"15min":1.3},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.85499999999999998,"5min":1.244,"15min":1.244},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.87,"5min":1.272,"15min":1.272},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77700000000000002,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":4.327,"15min":4.327},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56000000000000005,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":1.212,"15min":1.212},"last":0.44}]},{"osd":9,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51500000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.67900000000000005},"min":{"1min":0.496,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83499999999999996,"5min":1.337,"15min":1.337},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.63600000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":526392,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.34300000000000003},"min":{"1min":0.32300000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.497,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.29699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55700000000000005,"15min":0.5},"min":{"1min":0.48999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83599999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.441},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.53200000000000003,"15min":0.502},"min":{"1min":0.45800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35899999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.50800000000000001},"min":{"1min":0.47999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78700000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57299999999999995,"15min":0.51700000000000002},"min":{"1min":0.53100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.76300000000000001,"15min":0.63},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81100000000000005,"5min":14.458,"15min":14.458},"last":0.39900000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.309,"15min":0.309},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.45600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73499999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45700000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.55000000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.432},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.495,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55200000000000005}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.58699999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.501}]},{"osd":10,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.54200000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.878,"5min":1.02,"15min":1.02},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41399999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.65400000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.60999999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":1077248,"data_stored":1045381,"data_compressed":51775,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.35999999999999999},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.19500000000000001},"max":{"1min":0.51300000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42499999999999999,"15min":0.373},"min":{"1min":0.28899999999999998,"5min":0.253,"15min":0.246},"max":{"1min":0.54000000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91600000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.64000000000000001},"min":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":4.3979999999999997},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60299999999999998,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.6040000000000001,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:26:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.115,"15min":1.115},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.65000000000000002},"min":{"1min":0.5,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.64100000000000001},"min":{"1min":0.497,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82699999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98899999999999999,"5min":1.069,"15min":1.069},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.498}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73599999999999999,"5min":1.05,"15min":1.05},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.78200000000000003,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.81000000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92800000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.85499999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":1114112,"data_stored":589874,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52900000000000003,"15min":0.46300000000000002},"min":{"1min":0.38800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.313},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.40799999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68799999999999994,"5min":0.753,"15min":0.753},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54700000000000004,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.48399999999999999},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.34699999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.44700000000000001,"15min":0.41499999999999998},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.375},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.495,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.55100000000000005,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56299999999999994,"15min":0.53200000000000003},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.625,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63700000000000001}]},{"osd":11,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.628},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54900000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":1093632,"data_stored":588409,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.313},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84499999999999997,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.57699999999999996,"15min":0.54100000000000004},"min":{"1min":0.48699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":1.536,"15min":1.536},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.50800000000000001},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.89400000000000002,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.39300000000000002},"min":{"1min":0.34000000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.40899999999999997},"min":{"1min":0.33000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73999999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81499999999999995,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.504},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.57699999999999996},"min":{"1min":0.57399999999999995,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86199999999999999,"5min":1.613,"15min":1.613},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Sep 13 17:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83699999999999997,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.86199999999999999,"5min":2.504,"15min":2.504},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.53500000000000003},"min":{"1min":0.45300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94199999999999995,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.55300000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.60399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96799999999999997,"5min":1.458,"15min":1.458},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.61699999999999999},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.67500000000000004}]},{"osd":9,"last update":"Fri Sep 13 17:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.58899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98099999999999998,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.60399999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.58799999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":1.655,"15min":1.655},"last":0.58399999999999996}]},{"osd":11,"last update":"Fri Sep 13 17:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96499999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.69099999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.008,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.69599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":521759,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.45800000000000002},"min":{"1min":0.40400000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.51000000000000001},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.73599999999999999,"15min":0.56699999999999995},"min":{"1min":0.42999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75900000000000001,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.28499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78000000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65500000000000003,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.75}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.70699999999999996,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":17.844999999999999,"5min":17.844999999999999,"15min":17.844999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42599999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.78500000000000003,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78900000000000003,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":1.766,"15min":1.766},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75700000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89400000000000002,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84899999999999998,"5min":1.472,"15min":1.472},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.54700000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89700000000000002,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86299999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85999999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72599999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.91900000000000004,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84499999999999997,"5min":1.048,"15min":1.048},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133522,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202677,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:26:47.483 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.484 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/362728033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3740255021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.485 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:47 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1875731705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: Updating smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/362728033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:47.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:47.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:47.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3740255021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:47 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1875731705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 -- 172.21.15.44:0/1861646355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c1050e0 msgr2=0x7fb12c10cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 --2- 172.21.15.44:0/1861646355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c1050e0 0x7fb12c10cc30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb124009cd0 tx=0x7fb12402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 -- 172.21.15.44:0/1861646355 shutdown_connections 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 --2- 172.21.15.44:0/1861646355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c1050e0 0x7fb12c10cc30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 --2- 172.21.15.44:0/1861646355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c100940 0x7fb12c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 --2- 172.21.15.44:0/1861646355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c0fff90 0x7fb12c100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 -- 172.21.15.44:0/1861646355 >> 172.21.15.44:0/1861646355 conn(0x7fb12c0fba20 msgr2=0x7fb12c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 -- 172.21.15.44:0/1861646355 shutdown_connections 2024-09-13T17:26:47.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.587+0000 7fb132419640 1 -- 172.21.15.44:0/1861646355 wait complete. 2024-09-13T17:26:47.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 Processor -- start 2024-09-13T17:26:47.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 -- start start 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 0x7fb12c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c1050e0 0x7fb12c196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c19ab80 0x7fb12c19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c19b570 con 0x7fb12c0fff90 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c19b6e0 con 0x7fb12c19ab80 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb132419640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c19b850 con 0x7fb12c1050e0 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 0x7fb12c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 0x7fb12c196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40422/0 (socket says 172.21.15.44:40422) 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.588+0000 7fb12bfff640 1 -- 172.21.15.44:0/33939480 learned_addr learned my addr 172.21.15.44:0/33939480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12b7fe640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c1050e0 0x7fb12c196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb1309b6640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c19ab80 0x7fb12c19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12bfff640 1 -- 172.21.15.44:0/33939480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c1050e0 msgr2=0x7fb12c196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12bfff640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c1050e0 0x7fb12c196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12bfff640 1 -- 172.21.15.44:0/33939480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c19ab80 msgr2=0x7fb12c19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12bfff640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c19ab80 0x7fb12c19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12bfff640 1 -- 172.21.15.44:0/33939480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124009900 con 0x7fb12c0fff90 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb1309b6640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c19ab80 0x7fb12c19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb12bfff640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 0x7fb12c196250 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d950 tx=0x7fb11c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c009c30 con 0x7fb12c0fff90 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb11c009d90 con 0x7fb12c0fff90 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c1a2e30 con 0x7fb12c0fff90 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00f620 con 0x7fb12c0fff90 2024-09-13T17:26:47.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.589+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c1a3280 con 0x7fb12c0fff90 2024-09-13T17:26:47.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.590+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c101460 con 0x7fb12c0fff90 2024-09-13T17:26:47.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.591+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fb11c0109f0 con 0x7fb12c0fff90 2024-09-13T17:26:47.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.592+0000 7fb1297fa640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1140a1320 0x7fb1140a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.592+0000 7fb12b7fe640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1140a1320 0x7fb1140a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.592+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb11c0ccf50 con 0x7fb12c0fff90 2024-09-13T17:26:47.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.593+0000 7fb12b7fe640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1140a1320 0x7fb1140a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb12c1977f0 tx=0x7fb12000b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:47.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.596+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb11c095dd0 con 0x7fb12c0fff90 2024-09-13T17:26:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.765+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb12c108360 con 0x7fb1140a1320 2024-09-13T17:26:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.776+0000 7fb1297fa640 1 -- 172.21.15.44:0/33939480 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426558 (secure 0 0 0) 0x7fb12c108360 con 0x7fb1140a1320 2024-09-13T17:26:47.782 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:26:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1140a1320 msgr2=0x7fb1140a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1140a1320 0x7fb1140a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb12c1977f0 tx=0x7fb12000b410 comp rx=0 tx=0).stop 2024-09-13T17:26:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 msgr2=0x7fb12c196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 0x7fb12c196250 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d950 tx=0x7fb11c00de10 comp rx=0 tx=0).stop 2024-09-13T17:26:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 shutdown_connections 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1140a1320 0x7fb1140a37e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c1050e0 0x7fb12c196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12c19ab80 0x7fb12c19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 --2- 172.21.15.44:0/33939480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb12c0fff90 0x7fb12c196250 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 >> 172.21.15.44:0/33939480 conn(0x7fb12c0fba20 msgr2=0x7fb12c0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 shutdown_connections 2024-09-13T17:26:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.780+0000 7fb132419640 1 -- 172.21.15.44:0/33939480 wait complete. 2024-09-13T17:26:47.792 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-13T17:26:46.362976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188973,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":968,"num_read_kb":647,"num_write":465,"num_write_kb":4379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386016,"kb_used_data":12632,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124555808,"statfs":{"total":1151940427776,"available":1151545147392,"internally_reserved":0,"allocated":12935168,"data_stored":9841364,"data_compressed":502488,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989616"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.554882+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:26:25.554882+0000","last_peered":"2024-09-13T17:26:25.554882+0000","last_clean":"2024-09-13T17:26:25.554882+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:26:25.554882+0000","last_undegraded":"2024-09-13T17:26:25.554882+0000","last_fullsized":"2024-09-13T17:26:25.554882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:22.937512+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:26:22.937512+0000","last_peered":"2024-09-13T17:26:22.937512+0000","last_clean":"2024-09-13T17:26:22.937512+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:26:22.937512+0000","last_undegraded":"2024-09-13T17:26:22.937512+0000","last_fullsized":"2024-09-13T17:26:22.937512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056659+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:25:43.056659+0000","last_peered":"2024-09-13T17:25:43.056659+0000","last_clean":"2024-09-13T17:25:43.056659+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:25:43.056659+0000","last_undegraded":"2024-09-13T17:25:43.056659+0000","last_fullsized":"2024-09-13T17:25:43.056659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:22.937537+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:26:22.937537+0000","last_peered":"2024-09-13T17:26:22.937537+0000","last_clean":"2024-09-13T17:26:22.937537+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:26:22.937537+0000","last_undegraded":"2024-09-13T17:26:22.937537+0000","last_fullsized":"2024-09-13T17:26:22.937537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917611+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:26:30.917611+0000","last_peered":"2024-09-13T17:26:30.917611+0000","last_clean":"2024-09-13T17:26:30.917611+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:26:30.917611+0000","last_undegraded":"2024-09-13T17:26:30.917611+0000","last_fullsized":"2024-09-13T17:26:30.917611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.055957+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:25:43.055957+0000","last_peered":"2024-09-13T17:25:43.055957+0000","last_clean":"2024-09-13T17:25:43.055957+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:25:43.055957+0000","last_undegraded":"2024-09-13T17:25:43.055957+0000","last_fullsized":"2024-09-13T17:25:43.055957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:44.231710+0000","last_change":"2024-09-13T17:25:43.062483+0000","last_active":"2024-09-13T17:25:44.231710+0000","last_peered":"2024-09-13T17:25:44.231710+0000","last_clean":"2024-09-13T17:25:44.231710+0000","last_became_active":"2024-09-13T17:25:43.058145+0000","last_became_peered":"2024-09-13T17:25:43.058145+0000","last_unstale":"2024-09-13T17:25:44.231710+0000","last_undegraded":"2024-09-13T17:25:44.231710+0000","last_fullsized":"2024-09-13T17:25:44.231710+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:10.848768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.576853+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:26:11.576853+0000","last_peered":"2024-09-13T17:26:11.576853+0000","last_clean":"2024-09-13T17:26:11.576853+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:26:11.576853+0000","last_undegraded":"2024-09-13T17:26:11.576853+0000","last_fullsized":"2024-09-13T17:26:11.576853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056991+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:43.056991+0000","last_peered":"2024-09-13T17:25:43.056991+0000","last_clean":"2024-09-13T17:25:43.056991+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:25:43.056991+0000","last_undegraded":"2024-09-13T17:25:43.056991+0000","last_fullsized":"2024-09-13T17:25:43.056991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.051509+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:25:43.051509+0000","last_peered":"2024-09-13T17:25:43.051509+0000","last_clean":"2024-09-13T17:25:43.051509+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:25:43.051509+0000","last_undegraded":"2024-09-13T17:25:43.051509+0000","last_fullsized":"2024-09-13T17:25:43.051509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.569770+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:26:11.569770+0000","last_peered":"2024-09-13T17:26:11.569770+0000","last_clean":"2024-09-13T17:26:11.569770+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:26:11.569770+0000","last_undegraded":"2024-09-13T17:26:11.569770+0000","last_fullsized":"2024-09-13T17:26:11.569770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.055992+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:25:43.055992+0000","last_peered":"2024-09-13T17:25:43.055992+0000","last_clean":"2024-09-13T17:25:43.055992+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:25:43.055992+0000","last_undegraded":"2024-09-13T17:25:43.055992+0000","last_fullsized":"2024-09-13T17:25:43.055992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:36.185390+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:26:36.185390+0000","last_peered":"2024-09-13T17:26:36.185390+0000","last_clean":"2024-09-13T17:26:36.185390+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:26:36.185390+0000","last_undegraded":"2024-09-13T17:26:36.185390+0000","last_fullsized":"2024-09-13T17:26:36.185390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423394+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:26:42.423394+0000","last_peered":"2024-09-13T17:26:42.423394+0000","last_clean":"2024-09-13T17:26:42.423394+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:26:42.423394+0000","last_undegraded":"2024-09-13T17:26:42.423394+0000","last_fullsized":"2024-09-13T17:26:42.423394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351560+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:26:42.351560+0000","last_peered":"2024-09-13T17:26:42.351560+0000","last_clean":"2024-09-13T17:26:42.351560+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:26:42.351560+0000","last_undegraded":"2024-09-13T17:26:42.351560+0000","last_fullsized":"2024-09-13T17:26:42.351560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423943+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:26:42.423943+0000","last_peered":"2024-09-13T17:26:42.423943+0000","last_clean":"2024-09-13T17:26:42.423943+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:26:42.423943+0000","last_undegraded":"2024-09-13T17:26:42.423943+0000","last_fullsized":"2024-09-13T17:26:42.423943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817187+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:26:30.817187+0000","last_peered":"2024-09-13T17:26:30.817187+0000","last_clean":"2024-09-13T17:26:30.817187+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:26:30.817187+0000","last_undegraded":"2024-09-13T17:26:30.817187+0000","last_fullsized":"2024-09-13T17:26:30.817187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351304+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:26:42.351304+0000","last_peered":"2024-09-13T17:26:42.351304+0000","last_clean":"2024-09-13T17:26:42.351304+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:26:42.351304+0000","last_undegraded":"2024-09-13T17:26:42.351304+0000","last_fullsized":"2024-09-13T17:26:42.351304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237712+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:26:34.237712+0000","last_peered":"2024-09-13T17:26:34.237712+0000","last_clean":"2024-09-13T17:26:34.237712+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:26:34.237712+0000","last_undegraded":"2024-09-13T17:26:34.237712+0000","last_fullsized":"2024-09-13T17:26:34.237712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938386+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:26:32.938386+0000","last_peered":"2024-09-13T17:26:32.938386+0000","last_clean":"2024-09-13T17:26:32.938386+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:26:32.938386+0000","last_undegraded":"2024-09-13T17:26:32.938386+0000","last_fullsized":"2024-09-13T17:26:32.938386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.421368+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:26:42.421368+0000","last_peered":"2024-09-13T17:26:42.421368+0000","last_clean":"2024-09-13T17:26:42.421368+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:26:42.421368+0000","last_undegraded":"2024-09-13T17:26:42.421368+0000","last_fullsized":"2024-09-13T17:26:42.421368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.149653+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:26:34.149653+0000","last_peered":"2024-09-13T17:26:34.149653+0000","last_clean":"2024-09-13T17:26:34.149653+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:26:34.149653+0000","last_undegraded":"2024-09-13T17:26:34.149653+0000","last_fullsized":"2024-09-13T17:26:34.149653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.951691+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:26:33.951691+0000","last_peered":"2024-09-13T17:26:33.951691+0000","last_clean":"2024-09-13T17:26:33.951691+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:26:33.951691+0000","last_undegraded":"2024-09-13T17:26:33.951691+0000","last_fullsized":"2024-09-13T17:26:33.951691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351389+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:26:42.351389+0000","last_peered":"2024-09-13T17:26:42.351389+0000","last_clean":"2024-09-13T17:26:42.351389+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:26:42.351389+0000","last_undegraded":"2024-09-13T17:26:42.351389+0000","last_fullsized":"2024-09-13T17:26:42.351389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.918018+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:26:35.918018+0000","last_peered":"2024-09-13T17:26:35.918018+0000","last_clean":"2024-09-13T17:26:35.918018+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:26:35.918018+0000","last_undegraded":"2024-09-13T17:26:35.918018+0000","last_fullsized":"2024-09-13T17:26:35.918018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:31.909648+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:26:31.909648+0000","last_peered":"2024-09-13T17:26:31.909648+0000","last_clean":"2024-09-13T17:26:31.909648+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:26:31.909648+0000","last_undegraded":"2024-09-13T17:26:31.909648+0000","last_fullsized":"2024-09-13T17:26:31.909648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817100+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:26:30.817100+0000","last_peered":"2024-09-13T17:26:30.817100+0000","last_clean":"2024-09-13T17:26:30.817100+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:26:30.817100+0000","last_undegraded":"2024-09-13T17:26:30.817100+0000","last_fullsized":"2024-09-13T17:26:30.817100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938437+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:26:32.938437+0000","last_peered":"2024-09-13T17:26:32.938437+0000","last_clean":"2024-09-13T17:26:32.938437+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:26:32.938437+0000","last_undegraded":"2024-09-13T17:26:32.938437+0000","last_fullsized":"2024-09-13T17:26:32.938437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'41","reported_seq":86,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.537171+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:26:42.537171+0000","last_peered":"2024-09-13T17:26:42.537171+0000","last_clean":"2024-09-13T17:26:42.537171+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:26:42.537171+0000","last_undegraded":"2024-09-13T17:26:42.537171+0000","last_fullsized":"2024-09-13T17:26:42.537171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":73,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:27.937937+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:26:27.937937+0000","last_peered":"2024-09-13T17:26:27.937937+0000","last_clean":"2024-09-13T17:26:27.937937+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:26:27.937937+0000","last_undegraded":"2024-09-13T17:26:27.937937+0000","last_fullsized":"2024-09-13T17:26:27.937937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:28.129039+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:26:28.129039+0000","last_peered":"2024-09-13T17:26:28.129039+0000","last_clean":"2024-09-13T17:26:28.129039+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:26:28.129039+0000","last_undegraded":"2024-09-13T17:26:28.129039+0000","last_fullsized":"2024-09-13T17:26:28.129039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.053691+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:25:43.053691+0000","last_peered":"2024-09-13T17:25:43.053691+0000","last_clean":"2024-09-13T17:25:43.053691+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:25:43.053691+0000","last_undegraded":"2024-09-13T17:25:43.053691+0000","last_fullsized":"2024-09-13T17:25:43.053691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237734+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:26:34.237734+0000","last_peered":"2024-09-13T17:26:34.237734+0000","last_clean":"2024-09-13T17:26:34.237734+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:26:34.237734+0000","last_undegraded":"2024-09-13T17:26:34.237734+0000","last_fullsized":"2024-09-13T17:26:34.237734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.981541+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:26:29.981541+0000","last_peered":"2024-09-13T17:26:29.981541+0000","last_clean":"2024-09-13T17:26:29.981541+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:26:29.981541+0000","last_undegraded":"2024-09-13T17:26:29.981541+0000","last_fullsized":"2024-09-13T17:26:29.981541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.554849+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:26:25.554849+0000","last_peered":"2024-09-13T17:26:25.554849+0000","last_clean":"2024-09-13T17:26:25.554849+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:26:25.554849+0000","last_undegraded":"2024-09-13T17:26:25.554849+0000","last_fullsized":"2024-09-13T17:26:25.554849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.149669+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:26:34.149669+0000","last_peered":"2024-09-13T17:26:34.149669+0000","last_clean":"2024-09-13T17:26:34.149669+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:26:34.149669+0000","last_undegraded":"2024-09-13T17:26:34.149669+0000","last_fullsized":"2024-09-13T17:26:34.149669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.981522+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:26:29.981522+0000","last_peered":"2024-09-13T17:26:29.981522+0000","last_clean":"2024-09-13T17:26:29.981522+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:26:29.981522+0000","last_undegraded":"2024-09-13T17:26:29.981522+0000","last_fullsized":"2024-09-13T17:26:29.981522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:26.184106+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:26:26.184106+0000","last_peered":"2024-09-13T17:26:26.184106+0000","last_clean":"2024-09-13T17:26:26.184106+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:26:26.184106+0000","last_undegraded":"2024-09-13T17:26:26.184106+0000","last_fullsized":"2024-09-13T17:26:26.184106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.129425+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:26:33.129425+0000","last_peered":"2024-09-13T17:26:33.129425+0000","last_clean":"2024-09-13T17:26:33.129425+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:26:33.129425+0000","last_undegraded":"2024-09-13T17:26:33.129425+0000","last_fullsized":"2024-09-13T17:26:33.129425+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:24.730697+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:26:24.730697+0000","last_peered":"2024-09-13T17:26:24.730697+0000","last_clean":"2024-09-13T17:26:24.730697+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:26:24.730697+0000","last_undegraded":"2024-09-13T17:26:24.730697+0000","last_fullsized":"2024-09-13T17:26:24.730697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350284+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:26:42.350284+0000","last_peered":"2024-09-13T17:26:42.350284+0000","last_clean":"2024-09-13T17:26:42.350284+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:26:42.350284+0000","last_undegraded":"2024-09-13T17:26:42.350284+0000","last_fullsized":"2024-09-13T17:26:42.350284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350448+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:26:42.350448+0000","last_peered":"2024-09-13T17:26:42.350448+0000","last_clean":"2024-09-13T17:26:42.350448+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:26:42.350448+0000","last_undegraded":"2024-09-13T17:26:42.350448+0000","last_fullsized":"2024-09-13T17:26:42.350448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:28.128970+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:26:28.128970+0000","last_peered":"2024-09-13T17:26:28.128970+0000","last_clean":"2024-09-13T17:26:28.128970+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:26:28.128970+0000","last_undegraded":"2024-09-13T17:26:28.128970+0000","last_fullsized":"2024-09-13T17:26:28.128970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:21.183486+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:26:21.183486+0000","last_peered":"2024-09-13T17:26:21.183486+0000","last_clean":"2024-09-13T17:26:21.183486+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:26:21.183486+0000","last_undegraded":"2024-09-13T17:26:21.183486+0000","last_fullsized":"2024-09-13T17:26:21.183486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351357+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:26:42.351357+0000","last_peered":"2024-09-13T17:26:42.351357+0000","last_clean":"2024-09-13T17:26:42.351357+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:26:42.351357+0000","last_undegraded":"2024-09-13T17:26:42.351357+0000","last_fullsized":"2024-09-13T17:26:42.351357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:27.937986+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:26:27.937986+0000","last_peered":"2024-09-13T17:26:27.937986+0000","last_clean":"2024-09-13T17:26:27.937986+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:26:27.937986+0000","last_undegraded":"2024-09-13T17:26:27.937986+0000","last_fullsized":"2024-09-13T17:26:27.937986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:26.909107+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:26:26.909107+0000","last_peered":"2024-09-13T17:26:26.909107+0000","last_clean":"2024-09-13T17:26:26.909107+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:26:26.909107+0000","last_undegraded":"2024-09-13T17:26:26.909107+0000","last_fullsized":"2024-09-13T17:26:26.909107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:01.181134+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:26:01.181134+0000","last_peered":"2024-09-13T17:26:01.181134+0000","last_clean":"2024-09-13T17:26:01.181134+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:26:01.181134+0000","last_undegraded":"2024-09-13T17:26:01.181134+0000","last_fullsized":"2024-09-13T17:26:01.181134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:31.184745+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:26:31.184745+0000","last_peered":"2024-09-13T17:26:31.184745+0000","last_clean":"2024-09-13T17:26:31.184745+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:26:31.184745+0000","last_undegraded":"2024-09-13T17:26:31.184745+0000","last_fullsized":"2024-09-13T17:26:31.184745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501870+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:26:42.501870+0000","last_peered":"2024-09-13T17:26:42.501870+0000","last_clean":"2024-09-13T17:26:42.501870+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:26:42.501870+0000","last_undegraded":"2024-09-13T17:26:42.501870+0000","last_fullsized":"2024-09-13T17:26:42.501870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.817900+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:26:35.817900+0000","last_peered":"2024-09-13T17:26:35.817900+0000","last_clean":"2024-09-13T17:26:35.817900+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:26:35.817900+0000","last_undegraded":"2024-09-13T17:26:35.817900+0000","last_fullsized":"2024-09-13T17:26:35.817900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500602+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:26:42.500602+0000","last_peered":"2024-09-13T17:26:42.500602+0000","last_clean":"2024-09-13T17:26:42.500602+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:26:42.500602+0000","last_undegraded":"2024-09-13T17:26:42.500602+0000","last_fullsized":"2024-09-13T17:26:42.500602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501617+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:26:42.501617+0000","last_peered":"2024-09-13T17:26:42.501617+0000","last_clean":"2024-09-13T17:26:42.501617+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:26:42.501617+0000","last_undegraded":"2024-09-13T17:26:42.501617+0000","last_fullsized":"2024-09-13T17:26:42.501617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.540986+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:26:42.540986+0000","last_peered":"2024-09-13T17:26:42.540986+0000","last_clean":"2024-09-13T17:26:42.540986+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:26:42.540986+0000","last_undegraded":"2024-09-13T17:26:42.540986+0000","last_fullsized":"2024-09-13T17:26:42.540986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.503498+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:26:42.503498+0000","last_peered":"2024-09-13T17:26:42.503498+0000","last_clean":"2024-09-13T17:26:42.503498+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:26:42.503498+0000","last_undegraded":"2024-09-13T17:26:42.503498+0000","last_fullsized":"2024-09-13T17:26:42.503498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501867+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:26:42.501867+0000","last_peered":"2024-09-13T17:26:42.501867+0000","last_clean":"2024-09-13T17:26:42.501867+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:26:42.501867+0000","last_undegraded":"2024-09-13T17:26:42.501867+0000","last_fullsized":"2024-09-13T17:26:42.501867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350561+0000","last_change":"2024-09-13T17:25:43.395166+0000","last_active":"2024-09-13T17:26:42.350561+0000","last_peered":"2024-09-13T17:26:42.350561+0000","last_clean":"2024-09-13T17:26:42.350561+0000","last_became_active":"2024-09-13T17:25:43.392868+0000","last_became_peered":"2024-09-13T17:25:43.392868+0000","last_unstale":"2024-09-13T17:26:42.350561+0000","last_undegraded":"2024-09-13T17:26:42.350561+0000","last_fullsized":"2024-09-13T17:26:42.350561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:24.860809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350994+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:26:42.350994+0000","last_peered":"2024-09-13T17:26:42.350994+0000","last_clean":"2024-09-13T17:26:42.350994+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:26:42.350994+0000","last_undegraded":"2024-09-13T17:26:42.350994+0000","last_fullsized":"2024-09-13T17:26:42.350994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.502163+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:26:42.502163+0000","last_peered":"2024-09-13T17:26:42.502163+0000","last_clean":"2024-09-13T17:26:42.502163+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:26:42.502163+0000","last_undegraded":"2024-09-13T17:26:42.502163+0000","last_fullsized":"2024-09-13T17:26:42.502163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423700+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:26:42.423700+0000","last_peered":"2024-09-13T17:26:42.423700+0000","last_clean":"2024-09-13T17:26:42.423700+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:26:42.423700+0000","last_undegraded":"2024-09-13T17:26:42.423700+0000","last_fullsized":"2024-09-13T17:26:42.423700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351063+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:26:42.351063+0000","last_peered":"2024-09-13T17:26:42.351063+0000","last_clean":"2024-09-13T17:26:42.351063+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:26:42.351063+0000","last_undegraded":"2024-09-13T17:26:42.351063+0000","last_fullsized":"2024-09-13T17:26:42.351063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351181+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:26:42.351181+0000","last_peered":"2024-09-13T17:26:42.351181+0000","last_clean":"2024-09-13T17:26:42.351181+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:26:42.351181+0000","last_undegraded":"2024-09-13T17:26:42.351181+0000","last_fullsized":"2024-09-13T17:26:42.351181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423263+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:26:42.423263+0000","last_peered":"2024-09-13T17:26:42.423263+0000","last_clean":"2024-09-13T17:26:42.423263+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:26:42.423263+0000","last_undegraded":"2024-09-13T17:26:42.423263+0000","last_fullsized":"2024-09-13T17:26:42.423263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.054174+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:25:43.054174+0000","last_peered":"2024-09-13T17:25:43.054174+0000","last_clean":"2024-09-13T17:25:43.054174+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:25:43.054174+0000","last_undegraded":"2024-09-13T17:25:43.054174+0000","last_fullsized":"2024-09-13T17:25:43.054174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501336+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:26:42.501336+0000","last_peered":"2024-09-13T17:26:42.501336+0000","last_clean":"2024-09-13T17:26:42.501336+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:26:42.501336+0000","last_undegraded":"2024-09-13T17:26:42.501336+0000","last_fullsized":"2024-09-13T17:26:42.501336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:43.351080+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:26:43.351080+0000","last_peered":"2024-09-13T17:26:43.351080+0000","last_clean":"2024-09-13T17:26:43.351080+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:26:43.351080+0000","last_undegraded":"2024-09-13T17:26:43.351080+0000","last_fullsized":"2024-09-13T17:26:43.351080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1959,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.051664+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:25:43.051664+0000","last_peered":"2024-09-13T17:25:43.051664+0000","last_clean":"2024-09-13T17:25:43.051664+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:25:43.051664+0000","last_undegraded":"2024-09-13T17:25:43.051664+0000","last_fullsized":"2024-09-13T17:25:43.051664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.567360+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:26:11.567360+0000","last_peered":"2024-09-13T17:26:11.567360+0000","last_clean":"2024-09-13T17:26:11.567360+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:26:11.567360+0000","last_undegraded":"2024-09-13T17:26:11.567360+0000","last_fullsized":"2024-09-13T17:26:11.567360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351446+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:26:42.351446+0000","last_peered":"2024-09-13T17:26:42.351446+0000","last_clean":"2024-09-13T17:26:42.351446+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:26:42.351446+0000","last_undegraded":"2024-09-13T17:26:42.351446+0000","last_fullsized":"2024-09-13T17:26:42.351446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.053590+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:43.053590+0000","last_peered":"2024-09-13T17:25:43.053590+0000","last_clean":"2024-09-13T17:25:43.053590+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:25:43.053590+0000","last_undegraded":"2024-09-13T17:25:43.053590+0000","last_fullsized":"2024-09-13T17:25:43.053590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350518+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:26:42.350518+0000","last_peered":"2024-09-13T17:26:42.350518+0000","last_clean":"2024-09-13T17:26:42.350518+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:26:42.350518+0000","last_undegraded":"2024-09-13T17:26:42.350518+0000","last_fullsized":"2024-09-13T17:26:42.350518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501769+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:26:42.501769+0000","last_peered":"2024-09-13T17:26:42.501769+0000","last_clean":"2024-09-13T17:26:42.501769+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:26:42.501769+0000","last_undegraded":"2024-09-13T17:26:42.501769+0000","last_fullsized":"2024-09-13T17:26:42.501769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.237179+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:25:43.237179+0000","last_peered":"2024-09-13T17:25:43.237179+0000","last_clean":"2024-09-13T17:25:43.237179+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:25:43.237179+0000","last_undegraded":"2024-09-13T17:25:43.237179+0000","last_fullsized":"2024-09-13T17:25:43.237179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.560671+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:26:11.560671+0000","last_peered":"2024-09-13T17:26:11.560671+0000","last_clean":"2024-09-13T17:26:11.560671+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:26:11.560671+0000","last_undegraded":"2024-09-13T17:26:11.560671+0000","last_fullsized":"2024-09-13T17:26:11.560671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'40","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:43.319859+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:26:43.319859+0000","last_peered":"2024-09-13T17:26:43.319859+0000","last_clean":"2024-09-13T17:26:43.319859+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:26:43.319859+0000","last_undegraded":"2024-09-13T17:26:43.319859+0000","last_fullsized":"2024-09-13T17:26:43.319859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68128,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":191,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.049487+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:25:43.049487+0000","last_peered":"2024-09-13T17:25:43.049487+0000","last_clean":"2024-09-13T17:25:43.049487+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:25:43.049487+0000","last_undegraded":"2024-09-13T17:25:43.049487+0000","last_fullsized":"2024-09-13T17:25:43.049487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.577017+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:26:11.577017+0000","last_peered":"2024-09-13T17:26:11.577017+0000","last_clean":"2024-09-13T17:26:11.577017+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:26:11.577017+0000","last_undegraded":"2024-09-13T17:26:11.577017+0000","last_fullsized":"2024-09-13T17:26:11.577017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.237369+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:25:43.237369+0000","last_peered":"2024-09-13T17:25:43.237369+0000","last_clean":"2024-09-13T17:25:43.237369+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:25:43.237369+0000","last_undegraded":"2024-09-13T17:25:43.237369+0000","last_fullsized":"2024-09-13T17:25:43.237369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056840+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:25:43.056840+0000","last_peered":"2024-09-13T17:25:43.056840+0000","last_clean":"2024-09-13T17:25:43.056840+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:25:43.056840+0000","last_undegraded":"2024-09-13T17:25:43.056840+0000","last_fullsized":"2024-09-13T17:25:43.056840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.568212+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:26:11.568212+0000","last_peered":"2024-09-13T17:26:11.568212+0000","last_clean":"2024-09-13T17:26:11.568212+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:26:11.568212+0000","last_undegraded":"2024-09-13T17:26:11.568212+0000","last_fullsized":"2024-09-13T17:26:11.568212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501742+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:26:42.501742+0000","last_peered":"2024-09-13T17:26:42.501742+0000","last_clean":"2024-09-13T17:26:42.501742+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:26:42.501742+0000","last_undegraded":"2024-09-13T17:26:42.501742+0000","last_fullsized":"2024-09-13T17:26:42.501742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.420590+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:26:42.420590+0000","last_peered":"2024-09-13T17:26:42.420590+0000","last_clean":"2024-09-13T17:26:42.420590+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:26:42.420590+0000","last_undegraded":"2024-09-13T17:26:42.420590+0000","last_fullsized":"2024-09-13T17:26:42.420590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572870+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:26:11.572870+0000","last_peered":"2024-09-13T17:26:11.572870+0000","last_clean":"2024-09-13T17:26:11.572870+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:26:11.572870+0000","last_undegraded":"2024-09-13T17:26:11.572870+0000","last_fullsized":"2024-09-13T17:26:11.572870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":143,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.432692+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:26:42.432692+0000","last_peered":"2024-09-13T17:26:42.432692+0000","last_clean":"2024-09-13T17:26:42.432692+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:26:42.432692+0000","last_undegraded":"2024-09-13T17:26:42.432692+0000","last_fullsized":"2024-09-13T17:26:42.432692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056274+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:25:43.056274+0000","last_peered":"2024-09-13T17:25:43.056274+0000","last_clean":"2024-09-13T17:25:43.056274+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:25:43.056274+0000","last_undegraded":"2024-09-13T17:25:43.056274+0000","last_fullsized":"2024-09-13T17:25:43.056274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.917114+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:26:25.917114+0000","last_peered":"2024-09-13T17:26:25.917114+0000","last_clean":"2024-09-13T17:26:25.917114+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:26:25.917114+0000","last_undegraded":"2024-09-13T17:26:25.917114+0000","last_fullsized":"2024-09-13T17:26:25.917114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.564988+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:26:11.564988+0000","last_peered":"2024-09-13T17:26:11.564988+0000","last_clean":"2024-09-13T17:26:11.564988+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:26:11.564988+0000","last_undegraded":"2024-09-13T17:26:11.564988+0000","last_fullsized":"2024-09-13T17:26:11.564988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.236896+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:25:43.236896+0000","last_peered":"2024-09-13T17:25:43.236896+0000","last_clean":"2024-09-13T17:25:43.236896+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:25:43.236896+0000","last_undegraded":"2024-09-13T17:25:43.236896+0000","last_fullsized":"2024-09-13T17:25:43.236896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237862+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:26:34.237862+0000","last_peered":"2024-09-13T17:26:34.237862+0000","last_clean":"2024-09-13T17:26:34.237862+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:26:34.237862+0000","last_undegraded":"2024-09-13T17:26:34.237862+0000","last_fullsized":"2024-09-13T17:26:34.237862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.564967+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:26:11.564967+0000","last_peered":"2024-09-13T17:26:11.564967+0000","last_clean":"2024-09-13T17:26:11.564967+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:26:11.564967+0000","last_undegraded":"2024-09-13T17:26:11.564967+0000","last_fullsized":"2024-09-13T17:26:11.564967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.054105+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:25:43.054105+0000","last_peered":"2024-09-13T17:25:43.054105+0000","last_clean":"2024-09-13T17:25:43.054105+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:25:43.054105+0000","last_undegraded":"2024-09-13T17:25:43.054105+0000","last_fullsized":"2024-09-13T17:25:43.054105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.050335+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:25:43.050335+0000","last_peered":"2024-09-13T17:25:43.050335+0000","last_clean":"2024-09-13T17:25:43.050335+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:25:43.050335+0000","last_undegraded":"2024-09-13T17:25:43.050335+0000","last_fullsized":"2024-09-13T17:25:43.050335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572729+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:26:11.572729+0000","last_peered":"2024-09-13T17:26:11.572729+0000","last_clean":"2024-09-13T17:26:11.572729+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:26:11.572729+0000","last_undegraded":"2024-09-13T17:26:11.572729+0000","last_fullsized":"2024-09-13T17:26:11.572729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056930+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:25:43.056930+0000","last_peered":"2024-09-13T17:25:43.056930+0000","last_clean":"2024-09-13T17:25:43.056930+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:25:43.056930+0000","last_undegraded":"2024-09-13T17:25:43.056930+0000","last_fullsized":"2024-09-13T17:25:43.056930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351902+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:26:42.351902+0000","last_peered":"2024-09-13T17:26:42.351902+0000","last_clean":"2024-09-13T17:26:42.351902+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:26:42.351902+0000","last_undegraded":"2024-09-13T17:26:42.351902+0000","last_fullsized":"2024-09-13T17:26:42.351902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.557262+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:26:11.557262+0000","last_peered":"2024-09-13T17:26:11.557262+0000","last_clean":"2024-09-13T17:26:11.557262+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:26:11.557262+0000","last_undegraded":"2024-09-13T17:26:11.557262+0000","last_fullsized":"2024-09-13T17:26:11.557262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.057073+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:25:43.057073+0000","last_peered":"2024-09-13T17:25:43.057073+0000","last_clean":"2024-09-13T17:25:43.057073+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:25:43.057073+0000","last_undegraded":"2024-09-13T17:25:43.057073+0000","last_fullsized":"2024-09-13T17:25:43.057073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.236979+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:26:29.236979+0000","last_peered":"2024-09-13T17:26:29.236979+0000","last_clean":"2024-09-13T17:26:29.236979+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:26:29.236979+0000","last_undegraded":"2024-09-13T17:26:29.236979+0000","last_fullsized":"2024-09-13T17:26:29.236979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817145+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:26:30.817145+0000","last_peered":"2024-09-13T17:26:30.817145+0000","last_clean":"2024-09-13T17:26:30.817145+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:26:30.817145+0000","last_undegraded":"2024-09-13T17:26:30.817145+0000","last_fullsized":"2024-09-13T17:26:30.817145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817167+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:26:30.817167+0000","last_peered":"2024-09-13T17:26:30.817167+0000","last_clean":"2024-09-13T17:26:30.817167+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:26:30.817167+0000","last_undegraded":"2024-09-13T17:26:30.817167+0000","last_fullsized":"2024-09-13T17:26:30.817167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917538+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:26:30.917538+0000","last_peered":"2024-09-13T17:26:30.917538+0000","last_clean":"2024-09-13T17:26:30.917538+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:26:30.917538+0000","last_undegraded":"2024-09-13T17:26:30.917538+0000","last_fullsized":"2024-09-13T17:26:30.917538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.129438+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:26:33.129438+0000","last_peered":"2024-09-13T17:26:33.129438+0000","last_clean":"2024-09-13T17:26:33.129438+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:26:33.129438+0000","last_undegraded":"2024-09-13T17:26:33.129438+0000","last_fullsized":"2024-09-13T17:26:33.129438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237831+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:26:34.237831+0000","last_peered":"2024-09-13T17:26:34.237831+0000","last_clean":"2024-09-13T17:26:34.237831+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:26:34.237831+0000","last_undegraded":"2024-09-13T17:26:34.237831+0000","last_fullsized":"2024-09-13T17:26:34.237831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.817949+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:26:35.817949+0000","last_peered":"2024-09-13T17:26:35.817949+0000","last_clean":"2024-09-13T17:26:35.817949+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:26:35.817949+0000","last_undegraded":"2024-09-13T17:26:35.817949+0000","last_fullsized":"2024-09-13T17:26:35.817949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938425+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:26:32.938425+0000","last_peered":"2024-09-13T17:26:32.938425+0000","last_clean":"2024-09-13T17:26:32.938425+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:26:32.938425+0000","last_undegraded":"2024-09-13T17:26:32.938425+0000","last_fullsized":"2024-09-13T17:26:32.938425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.731999+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:26:34.731999+0000","last_peered":"2024-09-13T17:26:34.731999+0000","last_clean":"2024-09-13T17:26:34.731999+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:26:34.731999+0000","last_undegraded":"2024-09-13T17:26:34.731999+0000","last_fullsized":"2024-09-13T17:26:34.731999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.424680+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:26:42.424680+0000","last_peered":"2024-09-13T17:26:42.424680+0000","last_clean":"2024-09-13T17:26:42.424680+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:26:42.424680+0000","last_undegraded":"2024-09-13T17:26:42.424680+0000","last_fullsized":"2024-09-13T17:26:42.424680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500772+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:26:42.500772+0000","last_peered":"2024-09-13T17:26:42.500772+0000","last_clean":"2024-09-13T17:26:42.500772+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:26:42.500772+0000","last_undegraded":"2024-09-13T17:26:42.500772+0000","last_fullsized":"2024-09-13T17:26:42.500772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350799+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:26:42.350799+0000","last_peered":"2024-09-13T17:26:42.350799+0000","last_clean":"2024-09-13T17:26:42.350799+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:26:42.350799+0000","last_undegraded":"2024-09-13T17:26:42.350799+0000","last_fullsized":"2024-09-13T17:26:42.350799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501815+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:26:42.501815+0000","last_peered":"2024-09-13T17:26:42.501815+0000","last_clean":"2024-09-13T17:26:42.501815+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:26:42.501815+0000","last_undegraded":"2024-09-13T17:26:42.501815+0000","last_fullsized":"2024-09-13T17:26:42.501815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501512+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:26:42.501512+0000","last_peered":"2024-09-13T17:26:42.501512+0000","last_clean":"2024-09-13T17:26:42.501512+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:26:42.501512+0000","last_undegraded":"2024-09-13T17:26:42.501512+0000","last_fullsized":"2024-09-13T17:26:42.501512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.421423+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:26:42.421423+0000","last_peered":"2024-09-13T17:26:42.421423+0000","last_clean":"2024-09-13T17:26:42.421423+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:26:42.421423+0000","last_undegraded":"2024-09-13T17:26:42.421423+0000","last_fullsized":"2024-09-13T17:26:42.421423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.502291+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:26:42.502291+0000","last_peered":"2024-09-13T17:26:42.502291+0000","last_clean":"2024-09-13T17:26:42.502291+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:26:42.502291+0000","last_undegraded":"2024-09-13T17:26:42.502291+0000","last_fullsized":"2024-09-13T17:26:42.502291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500771+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:26:42.500771+0000","last_peered":"2024-09-13T17:26:42.500771+0000","last_clean":"2024-09-13T17:26:42.500771+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:26:42.500771+0000","last_undegraded":"2024-09-13T17:26:42.500771+0000","last_fullsized":"2024-09-13T17:26:42.500771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.566896+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:26:11.566896+0000","last_peered":"2024-09-13T17:26:11.566896+0000","last_clean":"2024-09-13T17:26:11.566896+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:26:11.566896+0000","last_undegraded":"2024-09-13T17:26:11.566896+0000","last_fullsized":"2024-09-13T17:26:11.566896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.569146+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:26:11.569146+0000","last_peered":"2024-09-13T17:26:11.569146+0000","last_clean":"2024-09-13T17:26:11.569146+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:26:11.569146+0000","last_undegraded":"2024-09-13T17:26:11.569146+0000","last_fullsized":"2024-09-13T17:26:11.569146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.562194+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:26:11.562194+0000","last_peered":"2024-09-13T17:26:11.562194+0000","last_clean":"2024-09-13T17:26:11.562194+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:26:11.562194+0000","last_undegraded":"2024-09-13T17:26:11.562194+0000","last_fullsized":"2024-09-13T17:26:11.562194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.543425+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:26:11.543425+0000","last_peered":"2024-09-13T17:26:11.543425+0000","last_clean":"2024-09-13T17:26:11.543425+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:26:11.543425+0000","last_undegraded":"2024-09-13T17:26:11.543425+0000","last_fullsized":"2024-09-13T17:26:11.543425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.575431+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:26:11.575431+0000","last_peered":"2024-09-13T17:26:11.575431+0000","last_clean":"2024-09-13T17:26:11.575431+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:26:11.575431+0000","last_undegraded":"2024-09-13T17:26:11.575431+0000","last_fullsized":"2024-09-13T17:26:11.575431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.561159+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:26:11.561159+0000","last_peered":"2024-09-13T17:26:11.561159+0000","last_clean":"2024-09-13T17:26:11.561159+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:26:11.561159+0000","last_undegraded":"2024-09-13T17:26:11.561159+0000","last_fullsized":"2024-09-13T17:26:11.561159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.576704+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:26:11.576704+0000","last_peered":"2024-09-13T17:26:11.576704+0000","last_clean":"2024-09-13T17:26:11.576704+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:26:11.576704+0000","last_undegraded":"2024-09-13T17:26:11.576704+0000","last_fullsized":"2024-09-13T17:26:11.576704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572750+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:26:11.572750+0000","last_peered":"2024-09-13T17:26:11.572750+0000","last_clean":"2024-09-13T17:26:11.572750+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:26:11.572750+0000","last_undegraded":"2024-09-13T17:26:11.572750+0000","last_fullsized":"2024-09-13T17:26:11.572750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.573518+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:26:11.573518+0000","last_peered":"2024-09-13T17:26:11.573518+0000","last_clean":"2024-09-13T17:26:11.573518+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:26:11.573518+0000","last_undegraded":"2024-09-13T17:26:11.573518+0000","last_fullsized":"2024-09-13T17:26:11.573518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.556715+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:26:11.556715+0000","last_peered":"2024-09-13T17:26:11.556715+0000","last_clean":"2024-09-13T17:26:11.556715+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:26:11.556715+0000","last_undegraded":"2024-09-13T17:26:11.556715+0000","last_fullsized":"2024-09-13T17:26:11.556715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.565559+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:26:11.565559+0000","last_peered":"2024-09-13T17:26:11.565559+0000","last_clean":"2024-09-13T17:26:11.565559+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:26:11.565559+0000","last_undegraded":"2024-09-13T17:26:11.565559+0000","last_fullsized":"2024-09-13T17:26:11.565559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.571323+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:26:11.571323+0000","last_peered":"2024-09-13T17:26:11.571323+0000","last_clean":"2024-09-13T17:26:11.571323+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:26:11.571323+0000","last_undegraded":"2024-09-13T17:26:11.571323+0000","last_fullsized":"2024-09-13T17:26:11.571323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917553+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:26:30.917553+0000","last_peered":"2024-09-13T17:26:30.917553+0000","last_clean":"2024-09-13T17:26:30.917553+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:26:30.917553+0000","last_undegraded":"2024-09-13T17:26:30.917553+0000","last_fullsized":"2024-09-13T17:26:30.917553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917595+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:26:30.917595+0000","last_peered":"2024-09-13T17:26:30.917595+0000","last_clean":"2024-09-13T17:26:30.917595+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:26:30.917595+0000","last_undegraded":"2024-09-13T17:26:30.917595+0000","last_fullsized":"2024-09-13T17:26:30.917595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.237162+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:26:29.237162+0000","last_peered":"2024-09-13T17:26:29.237162+0000","last_clean":"2024-09-13T17:26:29.237162+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:26:29.237162+0000","last_undegraded":"2024-09-13T17:26:29.237162+0000","last_fullsized":"2024-09-13T17:26:29.237162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598371,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":832,"num_read_kb":528,"num_write":282,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1795113,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108906,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416424,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":524981,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70699999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.57899999999999996,"5min":0.745,"15min":0.745},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.30399999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.81699999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.371},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61499999999999999},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82899999999999996,"5min":1.05,"15min":1.05},"last":0.35799999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85199999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86299999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.53700000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.97099999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.50800000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1196032,"data_stored":1317091,"data_compressed":68931,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56699999999999995,"15min":0.57399999999999995},"min":{"1min":0.35499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.435},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.438,"15min":0.438},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75600000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.58799999999999997},"min":{"1min":0.38200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74399999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.376}]},{"osd":5,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.503}]},{"osd":6,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":1.073,"15min":1.073},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.91300000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.496}]},{"osd":9,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70399999999999996,"15min":0.72199999999999998},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88600000000000001,"5min":1.147,"15min":1.147},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.52700000000000002}]},{"osd":11,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82399999999999995,"5min":1.175,"15min":1.175},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91700000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.54600000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":918411,"data_compressed":45619,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45500000000000002,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64200000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.49299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68200000000000005,"5min":0.999,"15min":0.999},"last":0.24399999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.852,"5min":13.852,"15min":13.852},"last":1.04}]},{"osd":2,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":0.88300000000000001,"15min":0.77400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":17.055,"5min":17.055,"15min":17.055},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61399999999999999,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57699999999999996,"15min":0.623},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.77700000000000002,"15min":0.71699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.074999999999999,"5min":17.074999999999999,"15min":17.074999999999999},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67200000000000004,"15min":0.70899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88700000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.42199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.753,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":1.054,"15min":1.054},"last":1.1519999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86499999999999999,"5min":1.083,"15min":1.083},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.71799999999999997,"15min":0.75600000000000001},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93700000000000006,"5min":1.04,"15min":1.04},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.006,"5min":1.288,"15min":1.288},"last":0.69899999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":0.83099999999999996,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":17.100999999999999,"5min":17.100999999999999,"15min":17.100999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.78800000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103223,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":917978,"data_compressed":45619,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.83899999999999997,"15min":0.97499999999999998},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.5679999999999996,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":0.68500000000000005,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":16.945,"5min":16.945,"15min":16.945},"last":0.372}]},{"osd":1,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":0.77400000000000002,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.544,"5min":22.544,"15min":22.544},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.59299999999999997,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.56399999999999995,"15min":0.46800000000000003},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":15.054,"5min":15.054,"15min":15.054},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.125,"5min":0.56699999999999995,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":14.135999999999999,"5min":14.135999999999999,"15min":14.135999999999999},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":0.73099999999999998,"15min":0.61499999999999999},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":19.047000000000001,"5min":19.047000000000001,"15min":19.047000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.778,"5min":3.778,"15min":3.778},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.61899999999999999,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.3220000000000001,"5min":7.3220000000000001,"15min":7.3220000000000001},"last":0.496},{"interface":"front","average":{"1min":1.077,"5min":0.68700000000000006,"15min":0.62},"min":{"1min":0.50800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":9.7200000000000006,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.8170000000000002,"5min":3.8170000000000002,"15min":3.8170000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":0.68300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":16.911000000000001,"5min":16.911000000000001,"15min":16.911000000000001},"last":0.436}]},{"osd":6,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.7280000000000002,"5min":3.7280000000000002,"15min":3.7280000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.151,"5min":0.754,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.5979999999999999,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.71999999999999997,"15min":0.66000000000000003},"min":{"1min":0.51500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7279999999999998,"5min":6.7279999999999998,"15min":6.7279999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":0.80400000000000005,"15min":0.68799999999999994},"min":{"1min":0.52800000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":16.847000000000001,"5min":16.847000000000001,"15min":16.847000000000001},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 17:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.73099999999999998,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.44,"15min":0.44},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.58599999999999997}]},{"osd":10,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.57499999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.57299999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.82099999999999995,"15min":0.80300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.9239999999999999,"5min":3.9239999999999999,"15min":3.9239999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.101,"5min":0.81599999999999995,"15min":0.76800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.9370000000000003,"5min":7.9370000000000003,"15min":7.9370000000000003},"last":0.59599999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":1089536,"data_stored":1122450,"data_compressed":62775,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71699999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.39800000000000002,"15min":0.379},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.313}]},{"osd":2,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.439}]},{"osd":4,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45400000000000001,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":0.75,"15min":0.75},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.34399999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80600000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.746,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83999999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41699999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.627},"min":{"1min":0.53200000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47799999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52100000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84999999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.502}]},{"osd":11,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.67100000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59099999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":520371,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51800000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.314},{"interface":"front","average":{"1min":0.372,"5min":0.39400000000000002,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.45900000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.751,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76800000000000002,"5min":1.008,"15min":1.008},"last":0.441}]},{"osd":2,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73299999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.49199999999999999},"min":{"1min":0.36399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67300000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.66600000000000004,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.623},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":1.018,"15min":1.018},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91900000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.52200000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.046,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53100000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90200000000000002,"5min":1.298,"15min":1.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.139,"15min":1.139},"last":0.65900000000000003}]},{"osd":11,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.61099999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.93799999999999994,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.77300000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.745}]}]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1118208,"data_stored":1248267,"data_compressed":68931,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.502,"15min":0.47699999999999998},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.39800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76200000000000001,"5min":1.415,"15min":1.415},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.35999999999999999},"min":{"1min":0.28699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48999999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.36899999999999999},"min":{"1min":0.28199999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.496,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54200000000000004,"15min":0.48699999999999999},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":1.522,"15min":1.522},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.42499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90000000000000002,"5min":1.3,"15min":1.3},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.85499999999999998,"5min":1.244,"15min":1.244},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.87,"5min":1.272,"15min":1.272},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77700000000000002,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":4.327,"15min":4.327},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56000000000000005,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":1.212,"15min":1.212},"last":0.44}]},{"osd":9,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51500000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.67900000000000005},"min":{"1min":0.496,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83499999999999996,"5min":1.337,"15min":1.337},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.63600000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":526392,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.34300000000000003},"min":{"1min":0.32300000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.497,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.29699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55700000000000005,"15min":0.5},"min":{"1min":0.48999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83599999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.441},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.53200000000000003,"15min":0.502},"min":{"1min":0.45800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35899999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.50800000000000001},"min":{"1min":0.47999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78700000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57299999999999995,"15min":0.51700000000000002},"min":{"1min":0.53100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.76300000000000001,"15min":0.63},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81100000000000005,"5min":14.458,"15min":14.458},"last":0.39900000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.309,"15min":0.309},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.45600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73499999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45700000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.55000000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.432},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.495,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55200000000000005}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.58699999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.501}]},{"osd":10,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.54200000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.878,"5min":1.02,"15min":1.02},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41399999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.65400000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.60999999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":1077248,"data_stored":1045381,"data_compressed":51775,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.35999999999999999},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.19500000000000001},"max":{"1min":0.51300000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42499999999999999,"15min":0.373},"min":{"1min":0.28899999999999998,"5min":0.253,"15min":0.246},"max":{"1min":0.54000000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91600000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.64000000000000001},"min":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":4.3979999999999997},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60299999999999998,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.6040000000000001,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:26:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.115,"15min":1.115},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.65000000000000002},"min":{"1min":0.5,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.64100000000000001},"min":{"1min":0.497,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82699999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98899999999999999,"5min":1.069,"15min":1.069},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.498}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73599999999999999,"5min":1.05,"15min":1.05},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.78200000000000003,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.81000000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92800000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.85499999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":1114112,"data_stored":589874,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52900000000000003,"15min":0.46300000000000002},"min":{"1min":0.38800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.313},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.40799999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68799999999999994,"5min":0.753,"15min":0.753},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54700000000000004,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.48399999999999999},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.34699999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.44700000000000001,"15min":0.41499999999999998},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.375},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.495,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.55100000000000005,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56299999999999994,"15min":0.53200000000000003},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.625,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63700000000000001}]},{"osd":11,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.628},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54900000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":1093632,"data_stored":588409,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.313},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84499999999999997,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.57699999999999996,"15min":0.54100000000000004},"min":{"1min":0.48699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":1.536,"15min":1.536},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.50800000000000001},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.89400000000000002,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.39300000000000002},"min":{"1min":0.34000000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.40899999999999997},"min":{"1min":0.33000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73999999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81499999999999995,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.504},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.57699999999999996},"min":{"1min":0.57399999999999995,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86199999999999999,"5min":1.613,"15min":1.613},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Sep 13 17:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83699999999999997,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.86199999999999999,"5min":2.504,"15min":2.504},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.53500000000000003},"min":{"1min":0.45300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94199999999999995,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.55300000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.60399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96799999999999997,"5min":1.458,"15min":1.458},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.61699999999999999},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.67500000000000004}]},{"osd":9,"last update":"Fri Sep 13 17:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.58899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98099999999999998,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.60399999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.58799999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":1.655,"15min":1.655},"last":0.58399999999999996}]},{"osd":11,"last update":"Fri Sep 13 17:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96499999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.69099999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.008,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.69599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":521759,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.45800000000000002},"min":{"1min":0.40400000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.51000000000000001},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.73599999999999999,"15min":0.56699999999999995},"min":{"1min":0.42999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75900000000000001,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.28499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78000000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65500000000000003,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.75}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.70699999999999996,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":17.844999999999999,"5min":17.844999999999999,"15min":17.844999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42599999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.78500000000000003,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78900000000000003,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":1.766,"15min":1.766},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75700000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89400000000000002,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84899999999999998,"5min":1.472,"15min":1.472},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.54700000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89700000000000002,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86299999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85999999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72599999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.91900000000000004,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84499999999999997,"5min":1.048,"15min":1.048},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133522,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202677,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:26:47.799 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-13T17:26:47.799 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:26:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 -- 172.21.15.44:0/2873336216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8103fe0 msgr2=0x7f07a810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 --2- 172.21.15.44:0/2873336216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8103fe0 0x7f07a810bb30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009cd0 tx=0x7f07a002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 -- 172.21.15.44:0/2873336216 shutdown_connections 2024-09-13T17:26:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 --2- 172.21.15.44:0/2873336216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8103fe0 0x7f07a810bb30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 --2- 172.21.15.44:0/2873336216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a80ff840 0x7f07a8103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 --2- 172.21.15.44:0/2873336216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 0x7f07a80ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 -- 172.21.15.44:0/2873336216 >> 172.21.15.44:0/2873336216 conn(0x7f07a80fa900 msgr2=0x7f07a80fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 -- 172.21.15.44:0/2873336216 shutdown_connections 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.934+0000 7f07acb18640 1 -- 172.21.15.44:0/2873336216 wait complete. 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 Processor -- start 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 -- start start 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 0x7f07a8196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a8103fe0 0x7f07a8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 0x7f07a819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819b520 con 0x7f07a819ab30 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819b690 con 0x7f07a8103fe0 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07acb18640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819b800 con 0x7f07a80fee90 2024-09-13T17:26:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07a6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 0x7f07a819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07a6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 0x7f07a819afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40450/0 (socket says 172.21.15.44:40450) 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07a6d9d640 1 -- 172.21.15.44:0/1523176589 learned_addr learned my addr 172.21.15.44:0/1523176589 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f079ffff640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a8103fe0 0x7f07a8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.935+0000 7f07a659c640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 0x7f07a8196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a6d9d640 1 -- 172.21.15.44:0/1523176589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 msgr2=0x7f07a8196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a6d9d640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 0x7f07a8196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a6d9d640 1 -- 172.21.15.44:0/1523176589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a8103fe0 msgr2=0x7f07a8196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a6d9d640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a8103fe0 0x7f07a8196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a6d9d640 1 -- 172.21.15.44:0/1523176589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0009900 con 0x7f07a819ab30 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a659c640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 0x7f07a8196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07a6d9d640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 0x7f07a819afe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f07a0004af0 tx=0x7f07a0030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0031b80 con 0x7f07a819ab30 2024-09-13T17:26:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a81a2de0 con 0x7f07a819ab30 2024-09-13T17:26:47.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07a0003e00 con 0x7f07a819ab30 2024-09-13T17:26:47.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0004800 con 0x7f07a819ab30 2024-09-13T17:26:47.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.936+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a81a3260 con 0x7f07a819ab30 2024-09-13T17:26:47.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.937+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8100360 con 0x7f07a819ab30 2024-09-13T17:26:47.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.938+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f07a000a040 con 0x7f07a819ab30 2024-09-13T17:26:47.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.939+0000 7f079f7fe640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f07940a1320 0x7f07940a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:47.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.939+0000 7f07a659c640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f07940a1320 0x7f07940a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:47.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.939+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f07a00ee950 con 0x7f07a819ab30 2024-09-13T17:26:47.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.940+0000 7f07a659c640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f07940a1320 0x7f07940a37e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07980058c0 tx=0x7f0798005850 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:47.943+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f07a00b7830 con 0x7f07a819ab30 2024-09-13T17:26:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.110+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07a80ff8b0 con 0x7f07940a1320 2024-09-13T17:26:48.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.121+0000 7f079f7fe640 1 -- 172.21.15.44:0/1523176589 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426558 (secure 0 0 0) 0x7f07a80ff8b0 con 0x7f07940a1320 2024-09-13T17:26:48.127 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:48.130 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f07940a1320 msgr2=0x7f07940a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f07940a1320 0x7f07940a37e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07980058c0 tx=0x7f0798005850 comp rx=0 tx=0).stop 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 msgr2=0x7f07a819afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 0x7f07a819afe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f07a0004af0 tx=0x7f07a0030dd0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 shutdown_connections 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f07940a1320 0x7f07940a37e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a819ab30 0x7f07a819afe0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a8103fe0 0x7f07a8196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 --2- 172.21.15.44:0/1523176589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a80fee90 0x7f07a8196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 >> 172.21.15.44:0/1523176589 conn(0x7f07a80fa900 msgr2=0x7f07a8068960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:48.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 shutdown_connections 2024-09-13T17:26:48.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.125+0000 7f07acb18640 1 -- 172.21.15.44:0/1523176589 wait complete. 2024-09-13T17:26:48.137 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-13T17:26:46.362976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188973,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":968,"num_read_kb":647,"num_write":465,"num_write_kb":4379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386016,"kb_used_data":12632,"kb_used_omap":18,"kb_used_meta":373229,"kb_avail":1124555808,"statfs":{"total":1151940427776,"available":1151545147392,"internally_reserved":0,"allocated":12935168,"data_stored":9841364,"data_compressed":502488,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19131,"internal_metadata":382186821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989616"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.554882+0000","last_change":"2024-09-13T17:24:21.385947+0000","last_active":"2024-09-13T17:26:25.554882+0000","last_peered":"2024-09-13T17:26:25.554882+0000","last_clean":"2024-09-13T17:26:25.554882+0000","last_became_active":"2024-09-13T17:24:21.385625+0000","last_became_peered":"2024-09-13T17:24:21.385625+0000","last_unstale":"2024-09-13T17:26:25.554882+0000","last_undegraded":"2024-09-13T17:26:25.554882+0000","last_fullsized":"2024-09-13T17:26:25.554882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:24.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:22.937512+0000","last_change":"2024-09-13T17:24:21.748552+0000","last_active":"2024-09-13T17:26:22.937512+0000","last_peered":"2024-09-13T17:26:22.937512+0000","last_clean":"2024-09-13T17:26:22.937512+0000","last_became_active":"2024-09-13T17:24:21.748253+0000","last_became_peered":"2024-09-13T17:24:21.748253+0000","last_unstale":"2024-09-13T17:26:22.937512+0000","last_undegraded":"2024-09-13T17:26:22.937512+0000","last_fullsized":"2024-09-13T17:26:22.937512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:25.255538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056659+0000","last_change":"2024-09-13T17:24:25.757185+0000","last_active":"2024-09-13T17:25:43.056659+0000","last_peered":"2024-09-13T17:25:43.056659+0000","last_clean":"2024-09-13T17:25:43.056659+0000","last_became_active":"2024-09-13T17:24:25.756777+0000","last_became_peered":"2024-09-13T17:24:25.756777+0000","last_unstale":"2024-09-13T17:25:43.056659+0000","last_undegraded":"2024-09-13T17:25:43.056659+0000","last_fullsized":"2024-09-13T17:25:43.056659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:35.376782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:22.937537+0000","last_change":"2024-09-13T17:24:21.388560+0000","last_active":"2024-09-13T17:26:22.937537+0000","last_peered":"2024-09-13T17:26:22.937537+0000","last_clean":"2024-09-13T17:26:22.937537+0000","last_became_active":"2024-09-13T17:24:21.388251+0000","last_became_peered":"2024-09-13T17:24:21.388251+0000","last_unstale":"2024-09-13T17:26:22.937537+0000","last_undegraded":"2024-09-13T17:26:22.937537+0000","last_fullsized":"2024-09-13T17:26:22.937537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:40.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917611+0000","last_change":"2024-09-13T17:24:21.740380+0000","last_active":"2024-09-13T17:26:30.917611+0000","last_peered":"2024-09-13T17:26:30.917611+0000","last_clean":"2024-09-13T17:26:30.917611+0000","last_became_active":"2024-09-13T17:24:21.740078+0000","last_became_peered":"2024-09-13T17:24:21.740078+0000","last_unstale":"2024-09-13T17:26:30.917611+0000","last_undegraded":"2024-09-13T17:26:30.917611+0000","last_fullsized":"2024-09-13T17:26:30.917611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:18.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.055957+0000","last_change":"2024-09-13T17:24:25.756147+0000","last_active":"2024-09-13T17:25:43.055957+0000","last_peered":"2024-09-13T17:25:43.055957+0000","last_clean":"2024-09-13T17:25:43.055957+0000","last_became_active":"2024-09-13T17:24:25.755599+0000","last_became_peered":"2024-09-13T17:24:25.755599+0000","last_unstale":"2024-09-13T17:25:43.055957+0000","last_undegraded":"2024-09-13T17:25:43.055957+0000","last_fullsized":"2024-09-13T17:25:43.055957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:18.276921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:44.231710+0000","last_change":"2024-09-13T17:25:43.062483+0000","last_active":"2024-09-13T17:25:44.231710+0000","last_peered":"2024-09-13T17:25:44.231710+0000","last_clean":"2024-09-13T17:25:44.231710+0000","last_became_active":"2024-09-13T17:25:43.058145+0000","last_became_peered":"2024-09-13T17:25:43.058145+0000","last_unstale":"2024-09-13T17:25:44.231710+0000","last_undegraded":"2024-09-13T17:25:44.231710+0000","last_fullsized":"2024-09-13T17:25:44.231710+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:10.848768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.576853+0000","last_change":"2024-09-13T17:24:21.732954+0000","last_active":"2024-09-13T17:26:11.576853+0000","last_peered":"2024-09-13T17:26:11.576853+0000","last_clean":"2024-09-13T17:26:11.576853+0000","last_became_active":"2024-09-13T17:24:21.732362+0000","last_became_peered":"2024-09-13T17:24:21.732362+0000","last_unstale":"2024-09-13T17:26:11.576853+0000","last_undegraded":"2024-09-13T17:26:11.576853+0000","last_fullsized":"2024-09-13T17:26:11.576853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.871511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056991+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:43.056991+0000","last_peered":"2024-09-13T17:25:43.056991+0000","last_clean":"2024-09-13T17:25:43.056991+0000","last_became_active":"2024-09-13T17:24:25.755673+0000","last_became_peered":"2024-09-13T17:24:25.755673+0000","last_unstale":"2024-09-13T17:25:43.056991+0000","last_undegraded":"2024-09-13T17:25:43.056991+0000","last_fullsized":"2024-09-13T17:25:43.056991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:10.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.051509+0000","last_change":"2024-09-13T17:24:21.381297+0000","last_active":"2024-09-13T17:25:43.051509+0000","last_peered":"2024-09-13T17:25:43.051509+0000","last_clean":"2024-09-13T17:25:43.051509+0000","last_became_active":"2024-09-13T17:24:21.380882+0000","last_became_peered":"2024-09-13T17:24:21.380882+0000","last_unstale":"2024-09-13T17:25:43.051509+0000","last_undegraded":"2024-09-13T17:25:43.051509+0000","last_fullsized":"2024-09-13T17:25:43.051509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:25.327334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.569770+0000","last_change":"2024-09-13T17:24:21.749767+0000","last_active":"2024-09-13T17:26:11.569770+0000","last_peered":"2024-09-13T17:26:11.569770+0000","last_clean":"2024-09-13T17:26:11.569770+0000","last_became_active":"2024-09-13T17:24:21.749305+0000","last_became_peered":"2024-09-13T17:24:21.749305+0000","last_unstale":"2024-09-13T17:26:11.569770+0000","last_undegraded":"2024-09-13T17:26:11.569770+0000","last_fullsized":"2024-09-13T17:26:11.569770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:56.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.055992+0000","last_change":"2024-09-13T17:24:25.753498+0000","last_active":"2024-09-13T17:25:43.055992+0000","last_peered":"2024-09-13T17:25:43.055992+0000","last_clean":"2024-09-13T17:25:43.055992+0000","last_became_active":"2024-09-13T17:24:25.752916+0000","last_became_peered":"2024-09-13T17:24:25.752916+0000","last_unstale":"2024-09-13T17:25:43.055992+0000","last_undegraded":"2024-09-13T17:25:43.055992+0000","last_fullsized":"2024-09-13T17:25:43.055992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:32.604805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:36.185390+0000","last_change":"2024-09-13T17:24:21.745328+0000","last_active":"2024-09-13T17:26:36.185390+0000","last_peered":"2024-09-13T17:26:36.185390+0000","last_clean":"2024-09-13T17:26:36.185390+0000","last_became_active":"2024-09-13T17:24:21.744926+0000","last_became_peered":"2024-09-13T17:24:21.744926+0000","last_unstale":"2024-09-13T17:26:36.185390+0000","last_undegraded":"2024-09-13T17:26:36.185390+0000","last_fullsized":"2024-09-13T17:26:36.185390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:52.645942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423394+0000","last_change":"2024-09-13T17:24:21.379170+0000","last_active":"2024-09-13T17:26:42.423394+0000","last_peered":"2024-09-13T17:26:42.423394+0000","last_clean":"2024-09-13T17:26:42.423394+0000","last_became_active":"2024-09-13T17:24:21.378836+0000","last_became_peered":"2024-09-13T17:24:21.378836+0000","last_unstale":"2024-09-13T17:26:42.423394+0000","last_undegraded":"2024-09-13T17:26:42.423394+0000","last_fullsized":"2024-09-13T17:26:42.423394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:39.363210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351560+0000","last_change":"2024-09-13T17:24:25.795826+0000","last_active":"2024-09-13T17:26:42.351560+0000","last_peered":"2024-09-13T17:26:42.351560+0000","last_clean":"2024-09-13T17:26:42.351560+0000","last_became_active":"2024-09-13T17:24:25.795233+0000","last_became_peered":"2024-09-13T17:24:25.795233+0000","last_unstale":"2024-09-13T17:26:42.351560+0000","last_undegraded":"2024-09-13T17:26:42.351560+0000","last_fullsized":"2024-09-13T17:26:42.351560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:48.143530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423943+0000","last_change":"2024-09-13T17:24:21.741863+0000","last_active":"2024-09-13T17:26:42.423943+0000","last_peered":"2024-09-13T17:26:42.423943+0000","last_clean":"2024-09-13T17:26:42.423943+0000","last_became_active":"2024-09-13T17:24:21.741535+0000","last_became_peered":"2024-09-13T17:24:21.741535+0000","last_unstale":"2024-09-13T17:26:42.423943+0000","last_undegraded":"2024-09-13T17:26:42.423943+0000","last_fullsized":"2024-09-13T17:26:42.423943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:44.889524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817187+0000","last_change":"2024-09-13T17:24:21.386863+0000","last_active":"2024-09-13T17:26:30.817187+0000","last_peered":"2024-09-13T17:26:30.817187+0000","last_clean":"2024-09-13T17:26:30.817187+0000","last_became_active":"2024-09-13T17:24:21.386478+0000","last_became_peered":"2024-09-13T17:24:21.386478+0000","last_unstale":"2024-09-13T17:26:30.817187+0000","last_undegraded":"2024-09-13T17:26:30.817187+0000","last_fullsized":"2024-09-13T17:26:30.817187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:53.149858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351304+0000","last_change":"2024-09-13T17:24:42.564232+0000","last_active":"2024-09-13T17:26:42.351304+0000","last_peered":"2024-09-13T17:26:42.351304+0000","last_clean":"2024-09-13T17:26:42.351304+0000","last_became_active":"2024-09-13T17:24:42.563855+0000","last_became_peered":"2024-09-13T17:24:42.563855+0000","last_unstale":"2024-09-13T17:26:42.351304+0000","last_undegraded":"2024-09-13T17:26:42.351304+0000","last_fullsized":"2024-09-13T17:26:42.351304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:28.738333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237712+0000","last_change":"2024-09-13T17:24:21.756125+0000","last_active":"2024-09-13T17:26:34.237712+0000","last_peered":"2024-09-13T17:26:34.237712+0000","last_clean":"2024-09-13T17:26:34.237712+0000","last_became_active":"2024-09-13T17:24:21.755717+0000","last_became_peered":"2024-09-13T17:24:21.755717+0000","last_unstale":"2024-09-13T17:26:34.237712+0000","last_undegraded":"2024-09-13T17:26:34.237712+0000","last_fullsized":"2024-09-13T17:26:34.237712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:05.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938386+0000","last_change":"2024-09-13T17:24:21.383309+0000","last_active":"2024-09-13T17:26:32.938386+0000","last_peered":"2024-09-13T17:26:32.938386+0000","last_clean":"2024-09-13T17:26:32.938386+0000","last_became_active":"2024-09-13T17:24:21.382781+0000","last_became_peered":"2024-09-13T17:24:21.382781+0000","last_unstale":"2024-09-13T17:26:32.938386+0000","last_undegraded":"2024-09-13T17:26:32.938386+0000","last_fullsized":"2024-09-13T17:26:32.938386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:22.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.421368+0000","last_change":"2024-09-13T17:24:25.757337+0000","last_active":"2024-09-13T17:26:42.421368+0000","last_peered":"2024-09-13T17:26:42.421368+0000","last_clean":"2024-09-13T17:26:42.421368+0000","last_became_active":"2024-09-13T17:24:25.756813+0000","last_became_peered":"2024-09-13T17:24:25.756813+0000","last_unstale":"2024-09-13T17:26:42.421368+0000","last_undegraded":"2024-09-13T17:26:42.421368+0000","last_fullsized":"2024-09-13T17:26:42.421368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:22.951871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.149653+0000","last_change":"2024-09-13T17:24:21.759988+0000","last_active":"2024-09-13T17:26:34.149653+0000","last_peered":"2024-09-13T17:26:34.149653+0000","last_clean":"2024-09-13T17:26:34.149653+0000","last_became_active":"2024-09-13T17:24:21.759603+0000","last_became_peered":"2024-09-13T17:24:21.759603+0000","last_unstale":"2024-09-13T17:26:34.149653+0000","last_undegraded":"2024-09-13T17:26:34.149653+0000","last_fullsized":"2024-09-13T17:26:34.149653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:44.346144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.951691+0000","last_change":"2024-09-13T17:24:21.376544+0000","last_active":"2024-09-13T17:26:33.951691+0000","last_peered":"2024-09-13T17:26:33.951691+0000","last_clean":"2024-09-13T17:26:33.951691+0000","last_became_active":"2024-09-13T17:24:21.376134+0000","last_became_peered":"2024-09-13T17:24:21.376134+0000","last_unstale":"2024-09-13T17:26:33.951691+0000","last_undegraded":"2024-09-13T17:26:33.951691+0000","last_fullsized":"2024-09-13T17:26:33.951691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:57.274099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351389+0000","last_change":"2024-09-13T17:24:42.562373+0000","last_active":"2024-09-13T17:26:42.351389+0000","last_peered":"2024-09-13T17:26:42.351389+0000","last_clean":"2024-09-13T17:26:42.351389+0000","last_became_active":"2024-09-13T17:24:42.561889+0000","last_became_peered":"2024-09-13T17:24:42.561889+0000","last_unstale":"2024-09-13T17:26:42.351389+0000","last_undegraded":"2024-09-13T17:26:42.351389+0000","last_fullsized":"2024-09-13T17:26:42.351389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.918018+0000","last_change":"2024-09-13T17:24:21.748004+0000","last_active":"2024-09-13T17:26:35.918018+0000","last_peered":"2024-09-13T17:26:35.918018+0000","last_clean":"2024-09-13T17:26:35.918018+0000","last_became_active":"2024-09-13T17:24:21.747637+0000","last_became_peered":"2024-09-13T17:24:21.747637+0000","last_unstale":"2024-09-13T17:26:35.918018+0000","last_undegraded":"2024-09-13T17:26:35.918018+0000","last_fullsized":"2024-09-13T17:26:35.918018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:54.988361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:31.909648+0000","last_change":"2024-09-13T17:24:21.375358+0000","last_active":"2024-09-13T17:26:31.909648+0000","last_peered":"2024-09-13T17:26:31.909648+0000","last_clean":"2024-09-13T17:26:31.909648+0000","last_became_active":"2024-09-13T17:24:21.375058+0000","last_became_peered":"2024-09-13T17:24:21.375058+0000","last_unstale":"2024-09-13T17:26:31.909648+0000","last_undegraded":"2024-09-13T17:26:31.909648+0000","last_fullsized":"2024-09-13T17:26:31.909648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:46.016048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817100+0000","last_change":"2024-09-13T17:24:25.753103+0000","last_active":"2024-09-13T17:26:30.817100+0000","last_peered":"2024-09-13T17:26:30.817100+0000","last_clean":"2024-09-13T17:26:30.817100+0000","last_became_active":"2024-09-13T17:24:25.752678+0000","last_became_peered":"2024-09-13T17:24:25.752678+0000","last_unstale":"2024-09-13T17:26:30.817100+0000","last_undegraded":"2024-09-13T17:26:30.817100+0000","last_fullsized":"2024-09-13T17:26:30.817100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:47:04.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938437+0000","last_change":"2024-09-13T17:24:21.744523+0000","last_active":"2024-09-13T17:26:32.938437+0000","last_peered":"2024-09-13T17:26:32.938437+0000","last_clean":"2024-09-13T17:26:32.938437+0000","last_became_active":"2024-09-13T17:24:21.743239+0000","last_became_peered":"2024-09-13T17:24:21.743239+0000","last_unstale":"2024-09-13T17:26:32.938437+0000","last_undegraded":"2024-09-13T17:26:32.938437+0000","last_fullsized":"2024-09-13T17:26:32.938437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:14.950840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'41","reported_seq":86,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.537171+0000","last_change":"2024-09-13T17:24:21.391525+0000","last_active":"2024-09-13T17:26:42.537171+0000","last_peered":"2024-09-13T17:26:42.537171+0000","last_clean":"2024-09-13T17:26:42.537171+0000","last_became_active":"2024-09-13T17:24:21.391215+0000","last_became_peered":"2024-09-13T17:24:21.391215+0000","last_unstale":"2024-09-13T17:26:42.537171+0000","last_undegraded":"2024-09-13T17:26:42.537171+0000","last_fullsized":"2024-09-13T17:26:42.537171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:34.793499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":73,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:27.937937+0000","last_change":"2024-09-13T17:24:25.795447+0000","last_active":"2024-09-13T17:26:27.937937+0000","last_peered":"2024-09-13T17:26:27.937937+0000","last_clean":"2024-09-13T17:26:27.937937+0000","last_became_active":"2024-09-13T17:24:25.794969+0000","last_became_peered":"2024-09-13T17:24:25.794969+0000","last_unstale":"2024-09-13T17:26:27.937937+0000","last_undegraded":"2024-09-13T17:26:27.937937+0000","last_fullsized":"2024-09-13T17:26:27.937937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:36.014938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:28.129039+0000","last_change":"2024-09-13T17:24:21.753134+0000","last_active":"2024-09-13T17:26:28.129039+0000","last_peered":"2024-09-13T17:26:28.129039+0000","last_clean":"2024-09-13T17:26:28.129039+0000","last_became_active":"2024-09-13T17:24:21.752781+0000","last_became_peered":"2024-09-13T17:24:21.752781+0000","last_unstale":"2024-09-13T17:26:28.129039+0000","last_undegraded":"2024-09-13T17:26:28.129039+0000","last_fullsized":"2024-09-13T17:26:28.129039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:48.853195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.053691+0000","last_change":"2024-09-13T17:24:42.564732+0000","last_active":"2024-09-13T17:25:43.053691+0000","last_peered":"2024-09-13T17:25:43.053691+0000","last_clean":"2024-09-13T17:25:43.053691+0000","last_became_active":"2024-09-13T17:24:42.564384+0000","last_became_peered":"2024-09-13T17:24:42.564384+0000","last_unstale":"2024-09-13T17:25:43.053691+0000","last_undegraded":"2024-09-13T17:25:43.053691+0000","last_fullsized":"2024-09-13T17:25:43.053691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:45.701933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237734+0000","last_change":"2024-09-13T17:24:25.753890+0000","last_active":"2024-09-13T17:26:34.237734+0000","last_peered":"2024-09-13T17:26:34.237734+0000","last_clean":"2024-09-13T17:26:34.237734+0000","last_became_active":"2024-09-13T17:24:25.753459+0000","last_became_peered":"2024-09-13T17:24:25.753459+0000","last_unstale":"2024-09-13T17:26:34.237734+0000","last_undegraded":"2024-09-13T17:26:34.237734+0000","last_fullsized":"2024-09-13T17:26:34.237734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:14.529969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.981541+0000","last_change":"2024-09-13T17:24:21.746104+0000","last_active":"2024-09-13T17:26:29.981541+0000","last_peered":"2024-09-13T17:26:29.981541+0000","last_clean":"2024-09-13T17:26:29.981541+0000","last_became_active":"2024-09-13T17:24:21.745756+0000","last_became_peered":"2024-09-13T17:24:21.745756+0000","last_unstale":"2024-09-13T17:26:29.981541+0000","last_undegraded":"2024-09-13T17:26:29.981541+0000","last_fullsized":"2024-09-13T17:26:29.981541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:16.008491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.554849+0000","last_change":"2024-09-13T17:24:21.389805+0000","last_active":"2024-09-13T17:26:25.554849+0000","last_peered":"2024-09-13T17:26:25.554849+0000","last_clean":"2024-09-13T17:26:25.554849+0000","last_became_active":"2024-09-13T17:24:21.389426+0000","last_became_peered":"2024-09-13T17:24:21.389426+0000","last_unstale":"2024-09-13T17:26:25.554849+0000","last_undegraded":"2024-09-13T17:26:25.554849+0000","last_fullsized":"2024-09-13T17:26:25.554849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:14.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.149669+0000","last_change":"2024-09-13T17:24:25.795713+0000","last_active":"2024-09-13T17:26:34.149669+0000","last_peered":"2024-09-13T17:26:34.149669+0000","last_clean":"2024-09-13T17:26:34.149669+0000","last_became_active":"2024-09-13T17:24:25.795273+0000","last_became_peered":"2024-09-13T17:24:25.795273+0000","last_unstale":"2024-09-13T17:26:34.149669+0000","last_undegraded":"2024-09-13T17:26:34.149669+0000","last_fullsized":"2024-09-13T17:26:34.149669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:01.660039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.981522+0000","last_change":"2024-09-13T17:24:21.748707+0000","last_active":"2024-09-13T17:26:29.981522+0000","last_peered":"2024-09-13T17:26:29.981522+0000","last_clean":"2024-09-13T17:26:29.981522+0000","last_became_active":"2024-09-13T17:24:21.748380+0000","last_became_peered":"2024-09-13T17:24:21.748380+0000","last_unstale":"2024-09-13T17:26:29.981522+0000","last_undegraded":"2024-09-13T17:26:29.981522+0000","last_fullsized":"2024-09-13T17:26:29.981522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.676069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:26.184106+0000","last_change":"2024-09-13T17:24:21.380203+0000","last_active":"2024-09-13T17:26:26.184106+0000","last_peered":"2024-09-13T17:26:26.184106+0000","last_clean":"2024-09-13T17:26:26.184106+0000","last_became_active":"2024-09-13T17:24:21.379919+0000","last_became_peered":"2024-09-13T17:24:21.379919+0000","last_unstale":"2024-09-13T17:26:26.184106+0000","last_undegraded":"2024-09-13T17:26:26.184106+0000","last_fullsized":"2024-09-13T17:26:26.184106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:07.065159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.129425+0000","last_change":"2024-09-13T17:24:25.753396+0000","last_active":"2024-09-13T17:26:33.129425+0000","last_peered":"2024-09-13T17:26:33.129425+0000","last_clean":"2024-09-13T17:26:33.129425+0000","last_became_active":"2024-09-13T17:24:25.752982+0000","last_became_peered":"2024-09-13T17:24:25.752982+0000","last_unstale":"2024-09-13T17:26:33.129425+0000","last_undegraded":"2024-09-13T17:26:33.129425+0000","last_fullsized":"2024-09-13T17:26:33.129425+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:40.160050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:24.730697+0000","last_change":"2024-09-13T17:24:21.753518+0000","last_active":"2024-09-13T17:26:24.730697+0000","last_peered":"2024-09-13T17:26:24.730697+0000","last_clean":"2024-09-13T17:26:24.730697+0000","last_became_active":"2024-09-13T17:24:21.753206+0000","last_became_peered":"2024-09-13T17:24:21.753206+0000","last_unstale":"2024-09-13T17:26:24.730697+0000","last_undegraded":"2024-09-13T17:26:24.730697+0000","last_fullsized":"2024-09-13T17:26:24.730697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:44.901836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350284+0000","last_change":"2024-09-13T17:24:21.385134+0000","last_active":"2024-09-13T17:26:42.350284+0000","last_peered":"2024-09-13T17:26:42.350284+0000","last_clean":"2024-09-13T17:26:42.350284+0000","last_became_active":"2024-09-13T17:24:21.384823+0000","last_became_peered":"2024-09-13T17:24:21.384823+0000","last_unstale":"2024-09-13T17:26:42.350284+0000","last_undegraded":"2024-09-13T17:26:42.350284+0000","last_fullsized":"2024-09-13T17:26:42.350284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:42:55.594728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350448+0000","last_change":"2024-09-13T17:24:25.795460+0000","last_active":"2024-09-13T17:26:42.350448+0000","last_peered":"2024-09-13T17:26:42.350448+0000","last_clean":"2024-09-13T17:26:42.350448+0000","last_became_active":"2024-09-13T17:24:25.794901+0000","last_became_peered":"2024-09-13T17:24:25.794901+0000","last_unstale":"2024-09-13T17:26:42.350448+0000","last_undegraded":"2024-09-13T17:26:42.350448+0000","last_fullsized":"2024-09-13T17:26:42.350448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:14.046044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:28.128970+0000","last_change":"2024-09-13T17:24:21.745385+0000","last_active":"2024-09-13T17:26:28.128970+0000","last_peered":"2024-09-13T17:26:28.128970+0000","last_clean":"2024-09-13T17:26:28.128970+0000","last_became_active":"2024-09-13T17:24:21.745077+0000","last_became_peered":"2024-09-13T17:24:21.745077+0000","last_unstale":"2024-09-13T17:26:28.128970+0000","last_undegraded":"2024-09-13T17:26:28.128970+0000","last_fullsized":"2024-09-13T17:26:28.128970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:34.748178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:21.183486+0000","last_change":"2024-09-13T17:24:21.384060+0000","last_active":"2024-09-13T17:26:21.183486+0000","last_peered":"2024-09-13T17:26:21.183486+0000","last_clean":"2024-09-13T17:26:21.183486+0000","last_became_active":"2024-09-13T17:24:21.383560+0000","last_became_peered":"2024-09-13T17:24:21.383560+0000","last_unstale":"2024-09-13T17:26:21.183486+0000","last_undegraded":"2024-09-13T17:26:21.183486+0000","last_fullsized":"2024-09-13T17:26:21.183486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:53.338504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351357+0000","last_change":"2024-09-13T17:24:25.795873+0000","last_active":"2024-09-13T17:26:42.351357+0000","last_peered":"2024-09-13T17:26:42.351357+0000","last_clean":"2024-09-13T17:26:42.351357+0000","last_became_active":"2024-09-13T17:24:25.795424+0000","last_became_peered":"2024-09-13T17:24:25.795424+0000","last_unstale":"2024-09-13T17:26:42.351357+0000","last_undegraded":"2024-09-13T17:26:42.351357+0000","last_fullsized":"2024-09-13T17:26:42.351357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:26.588774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:27.937986+0000","last_change":"2024-09-13T17:24:21.745460+0000","last_active":"2024-09-13T17:26:27.937986+0000","last_peered":"2024-09-13T17:26:27.937986+0000","last_clean":"2024-09-13T17:26:27.937986+0000","last_became_active":"2024-09-13T17:24:21.743998+0000","last_became_peered":"2024-09-13T17:24:21.743998+0000","last_unstale":"2024-09-13T17:26:27.937986+0000","last_undegraded":"2024-09-13T17:26:27.937986+0000","last_fullsized":"2024-09-13T17:26:27.937986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:30.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:26.909107+0000","last_change":"2024-09-13T17:24:21.383784+0000","last_active":"2024-09-13T17:26:26.909107+0000","last_peered":"2024-09-13T17:26:26.909107+0000","last_clean":"2024-09-13T17:26:26.909107+0000","last_became_active":"2024-09-13T17:24:21.383278+0000","last_became_peered":"2024-09-13T17:24:21.383278+0000","last_unstale":"2024-09-13T17:26:26.909107+0000","last_undegraded":"2024-09-13T17:26:26.909107+0000","last_fullsized":"2024-09-13T17:26:26.909107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:19.274954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:01.181134+0000","last_change":"2024-09-13T17:23:12.992545+0000","last_active":"2024-09-13T17:26:01.181134+0000","last_peered":"2024-09-13T17:26:01.181134+0000","last_clean":"2024-09-13T17:26:01.181134+0000","last_became_active":"2024-09-13T17:23:12.992105+0000","last_became_peered":"2024-09-13T17:23:12.992105+0000","last_unstale":"2024-09-13T17:26:01.181134+0000","last_undegraded":"2024-09-13T17:26:01.181134+0000","last_fullsized":"2024-09-13T17:26:01.181134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:20:01.986853+0000","last_clean_scrub_stamp":"2024-09-13T17:20:01.986853+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:35.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:31.184745+0000","last_change":"2024-09-13T17:24:25.750590+0000","last_active":"2024-09-13T17:26:31.184745+0000","last_peered":"2024-09-13T17:26:31.184745+0000","last_clean":"2024-09-13T17:26:31.184745+0000","last_became_active":"2024-09-13T17:24:25.750103+0000","last_became_peered":"2024-09-13T17:24:25.750103+0000","last_unstale":"2024-09-13T17:26:31.184745+0000","last_undegraded":"2024-09-13T17:26:31.184745+0000","last_fullsized":"2024-09-13T17:26:31.184745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:51.282970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501870+0000","last_change":"2024-09-13T17:24:21.748399+0000","last_active":"2024-09-13T17:26:42.501870+0000","last_peered":"2024-09-13T17:26:42.501870+0000","last_clean":"2024-09-13T17:26:42.501870+0000","last_became_active":"2024-09-13T17:24:21.748110+0000","last_became_peered":"2024-09-13T17:24:21.748110+0000","last_unstale":"2024-09-13T17:26:42.501870+0000","last_undegraded":"2024-09-13T17:26:42.501870+0000","last_fullsized":"2024-09-13T17:26:42.501870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:52.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.817900+0000","last_change":"2024-09-13T17:24:21.384243+0000","last_active":"2024-09-13T17:26:35.817900+0000","last_peered":"2024-09-13T17:26:35.817900+0000","last_clean":"2024-09-13T17:26:35.817900+0000","last_became_active":"2024-09-13T17:24:21.383741+0000","last_became_peered":"2024-09-13T17:24:21.383741+0000","last_unstale":"2024-09-13T17:26:35.817900+0000","last_undegraded":"2024-09-13T17:26:35.817900+0000","last_fullsized":"2024-09-13T17:26:35.817900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:10.202476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500602+0000","last_change":"2024-09-13T17:24:25.750508+0000","last_active":"2024-09-13T17:26:42.500602+0000","last_peered":"2024-09-13T17:26:42.500602+0000","last_clean":"2024-09-13T17:26:42.500602+0000","last_became_active":"2024-09-13T17:24:25.750135+0000","last_became_peered":"2024-09-13T17:24:25.750135+0000","last_unstale":"2024-09-13T17:26:42.500602+0000","last_undegraded":"2024-09-13T17:26:42.500602+0000","last_fullsized":"2024-09-13T17:26:42.500602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:16.259552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501617+0000","last_change":"2024-09-13T17:24:21.746212+0000","last_active":"2024-09-13T17:26:42.501617+0000","last_peered":"2024-09-13T17:26:42.501617+0000","last_clean":"2024-09-13T17:26:42.501617+0000","last_became_active":"2024-09-13T17:24:21.745840+0000","last_became_peered":"2024-09-13T17:24:21.745840+0000","last_unstale":"2024-09-13T17:26:42.501617+0000","last_undegraded":"2024-09-13T17:26:42.501617+0000","last_fullsized":"2024-09-13T17:26:42.501617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:01.125182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.540986+0000","last_change":"2024-09-13T17:24:21.391867+0000","last_active":"2024-09-13T17:26:42.540986+0000","last_peered":"2024-09-13T17:26:42.540986+0000","last_clean":"2024-09-13T17:26:42.540986+0000","last_became_active":"2024-09-13T17:24:21.391524+0000","last_became_peered":"2024-09-13T17:24:21.391524+0000","last_unstale":"2024-09-13T17:26:42.540986+0000","last_undegraded":"2024-09-13T17:26:42.540986+0000","last_fullsized":"2024-09-13T17:26:42.540986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.270894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.503498+0000","last_change":"2024-09-13T17:24:25.749461+0000","last_active":"2024-09-13T17:26:42.503498+0000","last_peered":"2024-09-13T17:26:42.503498+0000","last_clean":"2024-09-13T17:26:42.503498+0000","last_became_active":"2024-09-13T17:24:25.746811+0000","last_became_peered":"2024-09-13T17:24:25.746811+0000","last_unstale":"2024-09-13T17:26:42.503498+0000","last_undegraded":"2024-09-13T17:26:42.503498+0000","last_fullsized":"2024-09-13T17:26:42.503498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:09.837341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501867+0000","last_change":"2024-09-13T17:24:21.749232+0000","last_active":"2024-09-13T17:26:42.501867+0000","last_peered":"2024-09-13T17:26:42.501867+0000","last_clean":"2024-09-13T17:26:42.501867+0000","last_became_active":"2024-09-13T17:24:21.748917+0000","last_became_peered":"2024-09-13T17:24:21.748917+0000","last_unstale":"2024-09-13T17:26:42.501867+0000","last_undegraded":"2024-09-13T17:26:42.501867+0000","last_fullsized":"2024-09-13T17:26:42.501867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:53.969448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350561+0000","last_change":"2024-09-13T17:25:43.395166+0000","last_active":"2024-09-13T17:26:42.350561+0000","last_peered":"2024-09-13T17:26:42.350561+0000","last_clean":"2024-09-13T17:26:42.350561+0000","last_became_active":"2024-09-13T17:25:43.392868+0000","last_became_peered":"2024-09-13T17:25:43.392868+0000","last_unstale":"2024-09-13T17:26:42.350561+0000","last_undegraded":"2024-09-13T17:26:42.350561+0000","last_fullsized":"2024-09-13T17:26:42.350561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:24.860809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350994+0000","last_change":"2024-09-13T17:24:25.750956+0000","last_active":"2024-09-13T17:26:42.350994+0000","last_peered":"2024-09-13T17:26:42.350994+0000","last_clean":"2024-09-13T17:26:42.350994+0000","last_became_active":"2024-09-13T17:24:25.750087+0000","last_became_peered":"2024-09-13T17:24:25.750087+0000","last_unstale":"2024-09-13T17:26:42.350994+0000","last_undegraded":"2024-09-13T17:26:42.350994+0000","last_fullsized":"2024-09-13T17:26:42.350994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:15.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.502163+0000","last_change":"2024-09-13T17:24:21.753368+0000","last_active":"2024-09-13T17:26:42.502163+0000","last_peered":"2024-09-13T17:26:42.502163+0000","last_clean":"2024-09-13T17:26:42.502163+0000","last_became_active":"2024-09-13T17:24:21.753056+0000","last_became_peered":"2024-09-13T17:24:21.753056+0000","last_unstale":"2024-09-13T17:26:42.502163+0000","last_undegraded":"2024-09-13T17:26:42.502163+0000","last_fullsized":"2024-09-13T17:26:42.502163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:34.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423700+0000","last_change":"2024-09-13T17:24:21.383193+0000","last_active":"2024-09-13T17:26:42.423700+0000","last_peered":"2024-09-13T17:26:42.423700+0000","last_clean":"2024-09-13T17:26:42.423700+0000","last_became_active":"2024-09-13T17:24:21.382878+0000","last_became_peered":"2024-09-13T17:24:21.382878+0000","last_unstale":"2024-09-13T17:26:42.423700+0000","last_undegraded":"2024-09-13T17:26:42.423700+0000","last_fullsized":"2024-09-13T17:26:42.423700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:05.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351063+0000","last_change":"2024-09-13T17:24:25.757855+0000","last_active":"2024-09-13T17:26:42.351063+0000","last_peered":"2024-09-13T17:26:42.351063+0000","last_clean":"2024-09-13T17:26:42.351063+0000","last_became_active":"2024-09-13T17:24:25.757212+0000","last_became_peered":"2024-09-13T17:24:25.757212+0000","last_unstale":"2024-09-13T17:26:42.351063+0000","last_undegraded":"2024-09-13T17:26:42.351063+0000","last_fullsized":"2024-09-13T17:26:42.351063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:50.290884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351181+0000","last_change":"2024-09-13T17:24:21.746447+0000","last_active":"2024-09-13T17:26:42.351181+0000","last_peered":"2024-09-13T17:26:42.351181+0000","last_clean":"2024-09-13T17:26:42.351181+0000","last_became_active":"2024-09-13T17:24:21.746078+0000","last_became_peered":"2024-09-13T17:24:21.746078+0000","last_unstale":"2024-09-13T17:26:42.351181+0000","last_undegraded":"2024-09-13T17:26:42.351181+0000","last_fullsized":"2024-09-13T17:26:42.351181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:06.858181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.423263+0000","last_change":"2024-09-13T17:24:21.379018+0000","last_active":"2024-09-13T17:26:42.423263+0000","last_peered":"2024-09-13T17:26:42.423263+0000","last_clean":"2024-09-13T17:26:42.423263+0000","last_became_active":"2024-09-13T17:24:21.378653+0000","last_became_peered":"2024-09-13T17:24:21.378653+0000","last_unstale":"2024-09-13T17:26:42.423263+0000","last_undegraded":"2024-09-13T17:26:42.423263+0000","last_fullsized":"2024-09-13T17:26:42.423263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:36.534170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.054174+0000","last_change":"2024-09-13T17:24:42.780454+0000","last_active":"2024-09-13T17:25:43.054174+0000","last_peered":"2024-09-13T17:25:43.054174+0000","last_clean":"2024-09-13T17:25:43.054174+0000","last_became_active":"2024-09-13T17:24:42.779939+0000","last_became_peered":"2024-09-13T17:24:42.779939+0000","last_unstale":"2024-09-13T17:25:43.054174+0000","last_undegraded":"2024-09-13T17:25:43.054174+0000","last_fullsized":"2024-09-13T17:25:43.054174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:31.832880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501336+0000","last_change":"2024-09-13T17:24:21.741084+0000","last_active":"2024-09-13T17:26:42.501336+0000","last_peered":"2024-09-13T17:26:42.501336+0000","last_clean":"2024-09-13T17:26:42.501336+0000","last_became_active":"2024-09-13T17:24:21.740757+0000","last_became_peered":"2024-09-13T17:24:21.740757+0000","last_unstale":"2024-09-13T17:26:42.501336+0000","last_undegraded":"2024-09-13T17:26:42.501336+0000","last_fullsized":"2024-09-13T17:26:42.501336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:51.314598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:43.351080+0000","last_change":"2024-09-13T17:24:21.378898+0000","last_active":"2024-09-13T17:26:43.351080+0000","last_peered":"2024-09-13T17:26:43.351080+0000","last_clean":"2024-09-13T17:26:43.351080+0000","last_became_active":"2024-09-13T17:24:21.378565+0000","last_became_peered":"2024-09-13T17:24:21.378565+0000","last_unstale":"2024-09-13T17:26:43.351080+0000","last_undegraded":"2024-09-13T17:26:43.351080+0000","last_fullsized":"2024-09-13T17:26:43.351080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:52.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1959,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.051664+0000","last_change":"2024-09-13T17:24:42.780760+0000","last_active":"2024-09-13T17:25:43.051664+0000","last_peered":"2024-09-13T17:25:43.051664+0000","last_clean":"2024-09-13T17:25:43.051664+0000","last_became_active":"2024-09-13T17:24:42.780073+0000","last_became_peered":"2024-09-13T17:24:42.780073+0000","last_unstale":"2024-09-13T17:25:43.051664+0000","last_undegraded":"2024-09-13T17:25:43.051664+0000","last_fullsized":"2024-09-13T17:25:43.051664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:45.227599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.567360+0000","last_change":"2024-09-13T17:24:21.745282+0000","last_active":"2024-09-13T17:26:11.567360+0000","last_peered":"2024-09-13T17:26:11.567360+0000","last_clean":"2024-09-13T17:26:11.567360+0000","last_became_active":"2024-09-13T17:24:21.744898+0000","last_became_peered":"2024-09-13T17:24:21.744898+0000","last_unstale":"2024-09-13T17:26:11.567360+0000","last_undegraded":"2024-09-13T17:26:11.567360+0000","last_fullsized":"2024-09-13T17:26:11.567360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:47.308692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351446+0000","last_change":"2024-09-13T17:24:21.390160+0000","last_active":"2024-09-13T17:26:42.351446+0000","last_peered":"2024-09-13T17:26:42.351446+0000","last_clean":"2024-09-13T17:26:42.351446+0000","last_became_active":"2024-09-13T17:24:21.389819+0000","last_became_peered":"2024-09-13T17:24:21.389819+0000","last_unstale":"2024-09-13T17:26:42.351446+0000","last_undegraded":"2024-09-13T17:26:42.351446+0000","last_fullsized":"2024-09-13T17:26:42.351446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:06.095818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.053590+0000","last_change":"2024-09-13T17:24:25.756159+0000","last_active":"2024-09-13T17:25:43.053590+0000","last_peered":"2024-09-13T17:25:43.053590+0000","last_clean":"2024-09-13T17:25:43.053590+0000","last_became_active":"2024-09-13T17:24:25.755713+0000","last_became_peered":"2024-09-13T17:24:25.755713+0000","last_unstale":"2024-09-13T17:25:43.053590+0000","last_undegraded":"2024-09-13T17:25:43.053590+0000","last_fullsized":"2024-09-13T17:25:43.053590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:37.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350518+0000","last_change":"2024-09-13T17:24:21.739232+0000","last_active":"2024-09-13T17:26:42.350518+0000","last_peered":"2024-09-13T17:26:42.350518+0000","last_clean":"2024-09-13T17:26:42.350518+0000","last_became_active":"2024-09-13T17:24:21.738937+0000","last_became_peered":"2024-09-13T17:24:21.738937+0000","last_unstale":"2024-09-13T17:26:42.350518+0000","last_undegraded":"2024-09-13T17:26:42.350518+0000","last_fullsized":"2024-09-13T17:26:42.350518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:03.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501769+0000","last_change":"2024-09-13T17:24:21.380646+0000","last_active":"2024-09-13T17:26:42.501769+0000","last_peered":"2024-09-13T17:26:42.501769+0000","last_clean":"2024-09-13T17:26:42.501769+0000","last_became_active":"2024-09-13T17:24:21.380016+0000","last_became_peered":"2024-09-13T17:24:21.380016+0000","last_unstale":"2024-09-13T17:26:42.501769+0000","last_undegraded":"2024-09-13T17:26:42.501769+0000","last_fullsized":"2024-09-13T17:26:42.501769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:42.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.237179+0000","last_change":"2024-09-13T17:24:25.753614+0000","last_active":"2024-09-13T17:25:43.237179+0000","last_peered":"2024-09-13T17:25:43.237179+0000","last_clean":"2024-09-13T17:25:43.237179+0000","last_became_active":"2024-09-13T17:24:25.753278+0000","last_became_peered":"2024-09-13T17:24:25.753278+0000","last_unstale":"2024-09-13T17:25:43.237179+0000","last_undegraded":"2024-09-13T17:25:43.237179+0000","last_fullsized":"2024-09-13T17:25:43.237179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:59.400038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.560671+0000","last_change":"2024-09-13T17:24:21.760326+0000","last_active":"2024-09-13T17:26:11.560671+0000","last_peered":"2024-09-13T17:26:11.560671+0000","last_clean":"2024-09-13T17:26:11.560671+0000","last_became_active":"2024-09-13T17:24:21.760017+0000","last_became_peered":"2024-09-13T17:24:21.760017+0000","last_unstale":"2024-09-13T17:26:11.560671+0000","last_undegraded":"2024-09-13T17:26:11.560671+0000","last_fullsized":"2024-09-13T17:26:11.560671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:36.669183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'40","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:43.319859+0000","last_change":"2024-09-13T17:24:21.378584+0000","last_active":"2024-09-13T17:26:43.319859+0000","last_peered":"2024-09-13T17:26:43.319859+0000","last_clean":"2024-09-13T17:26:43.319859+0000","last_became_active":"2024-09-13T17:24:21.378012+0000","last_became_peered":"2024-09-13T17:24:21.378012+0000","last_unstale":"2024-09-13T17:26:43.319859+0000","last_undegraded":"2024-09-13T17:26:43.319859+0000","last_fullsized":"2024-09-13T17:26:43.319859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:07.104914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68128,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":191,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.049487+0000","last_change":"2024-09-13T17:24:25.756678+0000","last_active":"2024-09-13T17:25:43.049487+0000","last_peered":"2024-09-13T17:25:43.049487+0000","last_clean":"2024-09-13T17:25:43.049487+0000","last_became_active":"2024-09-13T17:24:25.754955+0000","last_became_peered":"2024-09-13T17:24:25.754955+0000","last_unstale":"2024-09-13T17:25:43.049487+0000","last_undegraded":"2024-09-13T17:25:43.049487+0000","last_fullsized":"2024-09-13T17:25:43.049487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:33.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.577017+0000","last_change":"2024-09-13T17:24:21.740115+0000","last_active":"2024-09-13T17:26:11.577017+0000","last_peered":"2024-09-13T17:26:11.577017+0000","last_clean":"2024-09-13T17:26:11.577017+0000","last_became_active":"2024-09-13T17:24:21.739233+0000","last_became_peered":"2024-09-13T17:24:21.739233+0000","last_unstale":"2024-09-13T17:26:11.577017+0000","last_undegraded":"2024-09-13T17:26:11.577017+0000","last_fullsized":"2024-09-13T17:26:11.577017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:17.494215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.237369+0000","last_change":"2024-09-13T17:24:21.389987+0000","last_active":"2024-09-13T17:25:43.237369+0000","last_peered":"2024-09-13T17:25:43.237369+0000","last_clean":"2024-09-13T17:25:43.237369+0000","last_became_active":"2024-09-13T17:24:21.389689+0000","last_became_peered":"2024-09-13T17:24:21.389689+0000","last_unstale":"2024-09-13T17:25:43.237369+0000","last_undegraded":"2024-09-13T17:25:43.237369+0000","last_fullsized":"2024-09-13T17:25:43.237369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:05.702036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056840+0000","last_change":"2024-09-13T17:24:25.756585+0000","last_active":"2024-09-13T17:25:43.056840+0000","last_peered":"2024-09-13T17:25:43.056840+0000","last_clean":"2024-09-13T17:25:43.056840+0000","last_became_active":"2024-09-13T17:24:25.755991+0000","last_became_peered":"2024-09-13T17:24:25.755991+0000","last_unstale":"2024-09-13T17:25:43.056840+0000","last_undegraded":"2024-09-13T17:25:43.056840+0000","last_fullsized":"2024-09-13T17:25:43.056840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:16.903090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.568212+0000","last_change":"2024-09-13T17:24:21.745838+0000","last_active":"2024-09-13T17:26:11.568212+0000","last_peered":"2024-09-13T17:26:11.568212+0000","last_clean":"2024-09-13T17:26:11.568212+0000","last_became_active":"2024-09-13T17:24:21.745553+0000","last_became_peered":"2024-09-13T17:24:21.745553+0000","last_unstale":"2024-09-13T17:26:11.568212+0000","last_undegraded":"2024-09-13T17:26:11.568212+0000","last_fullsized":"2024-09-13T17:26:11.568212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:22.774109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501742+0000","last_change":"2024-09-13T17:24:21.381381+0000","last_active":"2024-09-13T17:26:42.501742+0000","last_peered":"2024-09-13T17:26:42.501742+0000","last_clean":"2024-09-13T17:26:42.501742+0000","last_became_active":"2024-09-13T17:24:21.381095+0000","last_became_peered":"2024-09-13T17:24:21.381095+0000","last_unstale":"2024-09-13T17:26:42.501742+0000","last_undegraded":"2024-09-13T17:26:42.501742+0000","last_fullsized":"2024-09-13T17:26:42.501742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:27.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.420590+0000","last_change":"2024-09-13T17:24:25.759458+0000","last_active":"2024-09-13T17:26:42.420590+0000","last_peered":"2024-09-13T17:26:42.420590+0000","last_clean":"2024-09-13T17:26:42.420590+0000","last_became_active":"2024-09-13T17:24:25.757664+0000","last_became_peered":"2024-09-13T17:24:25.757664+0000","last_unstale":"2024-09-13T17:26:42.420590+0000","last_undegraded":"2024-09-13T17:26:42.420590+0000","last_fullsized":"2024-09-13T17:26:42.420590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:21.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572870+0000","last_change":"2024-09-13T17:24:21.732576+0000","last_active":"2024-09-13T17:26:11.572870+0000","last_peered":"2024-09-13T17:26:11.572870+0000","last_clean":"2024-09-13T17:26:11.572870+0000","last_became_active":"2024-09-13T17:24:21.732252+0000","last_became_peered":"2024-09-13T17:24:21.732252+0000","last_unstale":"2024-09-13T17:26:11.572870+0000","last_undegraded":"2024-09-13T17:26:11.572870+0000","last_fullsized":"2024-09-13T17:26:11.572870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:07.173985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":143,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.432692+0000","last_change":"2024-09-13T17:24:21.384570+0000","last_active":"2024-09-13T17:26:42.432692+0000","last_peered":"2024-09-13T17:26:42.432692+0000","last_clean":"2024-09-13T17:26:42.432692+0000","last_became_active":"2024-09-13T17:24:21.384267+0000","last_became_peered":"2024-09-13T17:24:21.384267+0000","last_unstale":"2024-09-13T17:26:42.432692+0000","last_undegraded":"2024-09-13T17:26:42.432692+0000","last_fullsized":"2024-09-13T17:26:42.432692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:14.262636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056274+0000","last_change":"2024-09-13T17:24:25.752829+0000","last_active":"2024-09-13T17:25:43.056274+0000","last_peered":"2024-09-13T17:25:43.056274+0000","last_clean":"2024-09-13T17:25:43.056274+0000","last_became_active":"2024-09-13T17:24:25.752450+0000","last_became_peered":"2024-09-13T17:24:25.752450+0000","last_unstale":"2024-09-13T17:25:43.056274+0000","last_undegraded":"2024-09-13T17:25:43.056274+0000","last_fullsized":"2024-09-13T17:25:43.056274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:43.611171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:25.917114+0000","last_change":"2024-09-13T17:24:25.745031+0000","last_active":"2024-09-13T17:26:25.917114+0000","last_peered":"2024-09-13T17:26:25.917114+0000","last_clean":"2024-09-13T17:26:25.917114+0000","last_became_active":"2024-09-13T17:24:25.744594+0000","last_became_peered":"2024-09-13T17:24:25.744594+0000","last_unstale":"2024-09-13T17:26:25.917114+0000","last_undegraded":"2024-09-13T17:26:25.917114+0000","last_fullsized":"2024-09-13T17:26:25.917114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:09.500346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.564988+0000","last_change":"2024-09-13T17:24:21.741559+0000","last_active":"2024-09-13T17:26:11.564988+0000","last_peered":"2024-09-13T17:26:11.564988+0000","last_clean":"2024-09-13T17:26:11.564988+0000","last_became_active":"2024-09-13T17:24:21.741279+0000","last_became_peered":"2024-09-13T17:24:21.741279+0000","last_unstale":"2024-09-13T17:26:11.564988+0000","last_undegraded":"2024-09-13T17:26:11.564988+0000","last_fullsized":"2024-09-13T17:26:11.564988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:02.511805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.236896+0000","last_change":"2024-09-13T17:24:21.385649+0000","last_active":"2024-09-13T17:25:43.236896+0000","last_peered":"2024-09-13T17:25:43.236896+0000","last_clean":"2024-09-13T17:25:43.236896+0000","last_became_active":"2024-09-13T17:24:21.385349+0000","last_became_peered":"2024-09-13T17:24:21.385349+0000","last_unstale":"2024-09-13T17:25:43.236896+0000","last_undegraded":"2024-09-13T17:25:43.236896+0000","last_fullsized":"2024-09-13T17:25:43.236896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:31.270379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237862+0000","last_change":"2024-09-13T17:24:25.757789+0000","last_active":"2024-09-13T17:26:34.237862+0000","last_peered":"2024-09-13T17:26:34.237862+0000","last_clean":"2024-09-13T17:26:34.237862+0000","last_became_active":"2024-09-13T17:24:25.757432+0000","last_became_peered":"2024-09-13T17:24:25.757432+0000","last_unstale":"2024-09-13T17:26:34.237862+0000","last_undegraded":"2024-09-13T17:26:34.237862+0000","last_fullsized":"2024-09-13T17:26:34.237862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:40.774010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.564967+0000","last_change":"2024-09-13T17:24:21.743446+0000","last_active":"2024-09-13T17:26:11.564967+0000","last_peered":"2024-09-13T17:26:11.564967+0000","last_clean":"2024-09-13T17:26:11.564967+0000","last_became_active":"2024-09-13T17:24:21.743173+0000","last_became_peered":"2024-09-13T17:24:21.743173+0000","last_unstale":"2024-09-13T17:26:11.564967+0000","last_undegraded":"2024-09-13T17:26:11.564967+0000","last_fullsized":"2024-09-13T17:26:11.564967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:02.890775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.054105+0000","last_change":"2024-09-13T17:24:21.384438+0000","last_active":"2024-09-13T17:25:43.054105+0000","last_peered":"2024-09-13T17:25:43.054105+0000","last_clean":"2024-09-13T17:25:43.054105+0000","last_became_active":"2024-09-13T17:24:21.384046+0000","last_became_peered":"2024-09-13T17:24:21.384046+0000","last_unstale":"2024-09-13T17:25:43.054105+0000","last_undegraded":"2024-09-13T17:25:43.054105+0000","last_fullsized":"2024-09-13T17:25:43.054105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:39.442106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.050335+0000","last_change":"2024-09-13T17:24:21.384138+0000","last_active":"2024-09-13T17:25:43.050335+0000","last_peered":"2024-09-13T17:25:43.050335+0000","last_clean":"2024-09-13T17:25:43.050335+0000","last_became_active":"2024-09-13T17:24:21.383254+0000","last_became_peered":"2024-09-13T17:24:21.383254+0000","last_unstale":"2024-09-13T17:25:43.050335+0000","last_undegraded":"2024-09-13T17:25:43.050335+0000","last_fullsized":"2024-09-13T17:25:43.050335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:39.604480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572729+0000","last_change":"2024-09-13T17:24:21.749892+0000","last_active":"2024-09-13T17:26:11.572729+0000","last_peered":"2024-09-13T17:26:11.572729+0000","last_clean":"2024-09-13T17:26:11.572729+0000","last_became_active":"2024-09-13T17:24:21.749584+0000","last_became_peered":"2024-09-13T17:24:21.749584+0000","last_unstale":"2024-09-13T17:26:11.572729+0000","last_undegraded":"2024-09-13T17:26:11.572729+0000","last_fullsized":"2024-09-13T17:26:11.572729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:38.826420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.056930+0000","last_change":"2024-09-13T17:24:25.756239+0000","last_active":"2024-09-13T17:25:43.056930+0000","last_peered":"2024-09-13T17:25:43.056930+0000","last_clean":"2024-09-13T17:25:43.056930+0000","last_became_active":"2024-09-13T17:24:25.755832+0000","last_became_peered":"2024-09-13T17:24:25.755832+0000","last_unstale":"2024-09-13T17:25:43.056930+0000","last_undegraded":"2024-09-13T17:25:43.056930+0000","last_fullsized":"2024-09-13T17:25:43.056930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:16.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.351902+0000","last_change":"2024-09-13T17:24:21.387771+0000","last_active":"2024-09-13T17:26:42.351902+0000","last_peered":"2024-09-13T17:26:42.351902+0000","last_clean":"2024-09-13T17:26:42.351902+0000","last_became_active":"2024-09-13T17:24:21.387428+0000","last_became_peered":"2024-09-13T17:24:21.387428+0000","last_unstale":"2024-09-13T17:26:42.351902+0000","last_undegraded":"2024-09-13T17:26:42.351902+0000","last_fullsized":"2024-09-13T17:26:42.351902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:20.352859+0000","last_clean_scrub_stamp":"2024-09-13T17:24:20.352859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:30.235890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.557262+0000","last_change":"2024-09-13T17:24:21.741393+0000","last_active":"2024-09-13T17:26:11.557262+0000","last_peered":"2024-09-13T17:26:11.557262+0000","last_clean":"2024-09-13T17:26:11.557262+0000","last_became_active":"2024-09-13T17:24:21.740933+0000","last_became_peered":"2024-09-13T17:24:21.740933+0000","last_unstale":"2024-09-13T17:26:11.557262+0000","last_undegraded":"2024-09-13T17:26:11.557262+0000","last_fullsized":"2024-09-13T17:26:11.557262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:42.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:25:43.057073+0000","last_change":"2024-09-13T17:24:25.754003+0000","last_active":"2024-09-13T17:25:43.057073+0000","last_peered":"2024-09-13T17:25:43.057073+0000","last_clean":"2024-09-13T17:25:43.057073+0000","last_became_active":"2024-09-13T17:24:25.753583+0000","last_became_peered":"2024-09-13T17:24:25.753583+0000","last_unstale":"2024-09-13T17:25:43.057073+0000","last_undegraded":"2024-09-13T17:25:43.057073+0000","last_fullsized":"2024-09-13T17:25:43.057073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:24.730757+0000","last_clean_scrub_stamp":"2024-09-13T17:24:24.730757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:46.925317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.236979+0000","last_change":"2024-09-13T17:24:21.756098+0000","last_active":"2024-09-13T17:26:29.236979+0000","last_peered":"2024-09-13T17:26:29.236979+0000","last_clean":"2024-09-13T17:26:29.236979+0000","last_became_active":"2024-09-13T17:24:21.755729+0000","last_became_peered":"2024-09-13T17:24:21.755729+0000","last_unstale":"2024-09-13T17:26:29.236979+0000","last_undegraded":"2024-09-13T17:26:29.236979+0000","last_fullsized":"2024-09-13T17:26:29.236979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:27.157061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817145+0000","last_change":"2024-09-13T17:24:21.749962+0000","last_active":"2024-09-13T17:26:30.817145+0000","last_peered":"2024-09-13T17:26:30.817145+0000","last_clean":"2024-09-13T17:26:30.817145+0000","last_became_active":"2024-09-13T17:24:21.749644+0000","last_became_peered":"2024-09-13T17:24:21.749644+0000","last_unstale":"2024-09-13T17:26:30.817145+0000","last_undegraded":"2024-09-13T17:26:30.817145+0000","last_fullsized":"2024-09-13T17:26:30.817145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:10.027876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.817167+0000","last_change":"2024-09-13T17:24:21.744289+0000","last_active":"2024-09-13T17:26:30.817167+0000","last_peered":"2024-09-13T17:26:30.817167+0000","last_clean":"2024-09-13T17:26:30.817167+0000","last_became_active":"2024-09-13T17:24:21.743939+0000","last_became_peered":"2024-09-13T17:24:21.743939+0000","last_unstale":"2024-09-13T17:26:30.817167+0000","last_undegraded":"2024-09-13T17:26:30.817167+0000","last_fullsized":"2024-09-13T17:26:30.817167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:40.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917538+0000","last_change":"2024-09-13T17:24:42.562202+0000","last_active":"2024-09-13T17:26:30.917538+0000","last_peered":"2024-09-13T17:26:30.917538+0000","last_clean":"2024-09-13T17:26:30.917538+0000","last_became_active":"2024-09-13T17:24:42.561722+0000","last_became_peered":"2024-09-13T17:24:42.561722+0000","last_unstale":"2024-09-13T17:26:30.917538+0000","last_undegraded":"2024-09-13T17:26:30.917538+0000","last_fullsized":"2024-09-13T17:26:30.917538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:42.554496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:33.129438+0000","last_change":"2024-09-13T17:24:21.745816+0000","last_active":"2024-09-13T17:26:33.129438+0000","last_peered":"2024-09-13T17:26:33.129438+0000","last_clean":"2024-09-13T17:26:33.129438+0000","last_became_active":"2024-09-13T17:24:21.745499+0000","last_became_peered":"2024-09-13T17:24:21.745499+0000","last_unstale":"2024-09-13T17:26:33.129438+0000","last_undegraded":"2024-09-13T17:26:33.129438+0000","last_fullsized":"2024-09-13T17:26:33.129438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:09:27.481551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.237831+0000","last_change":"2024-09-13T17:24:42.564605+0000","last_active":"2024-09-13T17:26:34.237831+0000","last_peered":"2024-09-13T17:26:34.237831+0000","last_clean":"2024-09-13T17:26:34.237831+0000","last_became_active":"2024-09-13T17:24:42.564180+0000","last_became_peered":"2024-09-13T17:24:42.564180+0000","last_unstale":"2024-09-13T17:26:34.237831+0000","last_undegraded":"2024-09-13T17:26:34.237831+0000","last_fullsized":"2024-09-13T17:26:34.237831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:16.587263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:35.817949+0000","last_change":"2024-09-13T17:24:21.745212+0000","last_active":"2024-09-13T17:26:35.817949+0000","last_peered":"2024-09-13T17:26:35.817949+0000","last_clean":"2024-09-13T17:26:35.817949+0000","last_became_active":"2024-09-13T17:24:21.744891+0000","last_became_peered":"2024-09-13T17:24:21.744891+0000","last_unstale":"2024-09-13T17:26:35.817949+0000","last_undegraded":"2024-09-13T17:26:35.817949+0000","last_fullsized":"2024-09-13T17:26:35.817949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:32.938425+0000","last_change":"2024-09-13T17:24:21.744307+0000","last_active":"2024-09-13T17:26:32.938425+0000","last_peered":"2024-09-13T17:26:32.938425+0000","last_clean":"2024-09-13T17:26:32.938425+0000","last_became_active":"2024-09-13T17:24:21.743845+0000","last_became_peered":"2024-09-13T17:24:21.743845+0000","last_unstale":"2024-09-13T17:26:32.938425+0000","last_undegraded":"2024-09-13T17:26:32.938425+0000","last_fullsized":"2024-09-13T17:26:32.938425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:52.069174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:34.731999+0000","last_change":"2024-09-13T17:24:21.744064+0000","last_active":"2024-09-13T17:26:34.731999+0000","last_peered":"2024-09-13T17:26:34.731999+0000","last_clean":"2024-09-13T17:26:34.731999+0000","last_became_active":"2024-09-13T17:24:21.743709+0000","last_became_peered":"2024-09-13T17:24:21.743709+0000","last_unstale":"2024-09-13T17:26:34.731999+0000","last_undegraded":"2024-09-13T17:26:34.731999+0000","last_fullsized":"2024-09-13T17:26:34.731999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:05.922560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.424680+0000","last_change":"2024-09-13T17:24:21.752441+0000","last_active":"2024-09-13T17:26:42.424680+0000","last_peered":"2024-09-13T17:26:42.424680+0000","last_clean":"2024-09-13T17:26:42.424680+0000","last_became_active":"2024-09-13T17:24:21.752034+0000","last_became_peered":"2024-09-13T17:24:21.752034+0000","last_unstale":"2024-09-13T17:26:42.424680+0000","last_undegraded":"2024-09-13T17:26:42.424680+0000","last_fullsized":"2024-09-13T17:26:42.424680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:00.403667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500772+0000","last_change":"2024-09-13T17:24:21.739356+0000","last_active":"2024-09-13T17:26:42.500772+0000","last_peered":"2024-09-13T17:26:42.500772+0000","last_clean":"2024-09-13T17:26:42.500772+0000","last_became_active":"2024-09-13T17:24:21.739081+0000","last_became_peered":"2024-09-13T17:24:21.739081+0000","last_unstale":"2024-09-13T17:26:42.500772+0000","last_undegraded":"2024-09-13T17:26:42.500772+0000","last_fullsized":"2024-09-13T17:26:42.500772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:11.824424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.350799+0000","last_change":"2024-09-13T17:24:21.738874+0000","last_active":"2024-09-13T17:26:42.350799+0000","last_peered":"2024-09-13T17:26:42.350799+0000","last_clean":"2024-09-13T17:26:42.350799+0000","last_became_active":"2024-09-13T17:24:21.738586+0000","last_became_peered":"2024-09-13T17:24:21.738586+0000","last_unstale":"2024-09-13T17:26:42.350799+0000","last_undegraded":"2024-09-13T17:26:42.350799+0000","last_fullsized":"2024-09-13T17:26:42.350799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:14.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501815+0000","last_change":"2024-09-13T17:24:21.760519+0000","last_active":"2024-09-13T17:26:42.501815+0000","last_peered":"2024-09-13T17:26:42.501815+0000","last_clean":"2024-09-13T17:26:42.501815+0000","last_became_active":"2024-09-13T17:24:21.760196+0000","last_became_peered":"2024-09-13T17:24:21.760196+0000","last_unstale":"2024-09-13T17:26:42.501815+0000","last_undegraded":"2024-09-13T17:26:42.501815+0000","last_fullsized":"2024-09-13T17:26:42.501815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:56.273751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.501512+0000","last_change":"2024-09-13T17:24:21.744105+0000","last_active":"2024-09-13T17:26:42.501512+0000","last_peered":"2024-09-13T17:26:42.501512+0000","last_clean":"2024-09-13T17:26:42.501512+0000","last_became_active":"2024-09-13T17:24:21.743545+0000","last_became_peered":"2024-09-13T17:24:21.743545+0000","last_unstale":"2024-09-13T17:26:42.501512+0000","last_undegraded":"2024-09-13T17:26:42.501512+0000","last_fullsized":"2024-09-13T17:26:42.501512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:58.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.421423+0000","last_change":"2024-09-13T17:24:21.732618+0000","last_active":"2024-09-13T17:26:42.421423+0000","last_peered":"2024-09-13T17:26:42.421423+0000","last_clean":"2024-09-13T17:26:42.421423+0000","last_became_active":"2024-09-13T17:24:21.732315+0000","last_became_peered":"2024-09-13T17:24:21.732315+0000","last_unstale":"2024-09-13T17:26:42.421423+0000","last_undegraded":"2024-09-13T17:26:42.421423+0000","last_fullsized":"2024-09-13T17:26:42.421423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:35.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.502291+0000","last_change":"2024-09-13T17:24:21.748702+0000","last_active":"2024-09-13T17:26:42.502291+0000","last_peered":"2024-09-13T17:26:42.502291+0000","last_clean":"2024-09-13T17:26:42.502291+0000","last_became_active":"2024-09-13T17:24:21.748444+0000","last_became_peered":"2024-09-13T17:24:21.748444+0000","last_unstale":"2024-09-13T17:26:42.502291+0000","last_undegraded":"2024-09-13T17:26:42.502291+0000","last_fullsized":"2024-09-13T17:26:42.502291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:58.542924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T17:26:42.500771+0000","last_change":"2024-09-13T17:24:21.737974+0000","last_active":"2024-09-13T17:26:42.500771+0000","last_peered":"2024-09-13T17:26:42.500771+0000","last_clean":"2024-09-13T17:26:42.500771+0000","last_became_active":"2024-09-13T17:24:21.737663+0000","last_became_peered":"2024-09-13T17:24:21.737663+0000","last_unstale":"2024-09-13T17:26:42.500771+0000","last_undegraded":"2024-09-13T17:26:42.500771+0000","last_fullsized":"2024-09-13T17:26:42.500771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:41.437204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.566896+0000","last_change":"2024-09-13T17:24:21.747236+0000","last_active":"2024-09-13T17:26:11.566896+0000","last_peered":"2024-09-13T17:26:11.566896+0000","last_clean":"2024-09-13T17:26:11.566896+0000","last_became_active":"2024-09-13T17:24:21.746162+0000","last_became_peered":"2024-09-13T17:24:21.746162+0000","last_unstale":"2024-09-13T17:26:11.566896+0000","last_undegraded":"2024-09-13T17:26:11.566896+0000","last_fullsized":"2024-09-13T17:26:11.566896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:14:17.567128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.569146+0000","last_change":"2024-09-13T17:24:21.741841+0000","last_active":"2024-09-13T17:26:11.569146+0000","last_peered":"2024-09-13T17:26:11.569146+0000","last_clean":"2024-09-13T17:26:11.569146+0000","last_became_active":"2024-09-13T17:24:21.741376+0000","last_became_peered":"2024-09-13T17:24:21.741376+0000","last_unstale":"2024-09-13T17:26:11.569146+0000","last_undegraded":"2024-09-13T17:26:11.569146+0000","last_fullsized":"2024-09-13T17:26:11.569146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:12.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.562194+0000","last_change":"2024-09-13T17:24:21.746532+0000","last_active":"2024-09-13T17:26:11.562194+0000","last_peered":"2024-09-13T17:26:11.562194+0000","last_clean":"2024-09-13T17:26:11.562194+0000","last_became_active":"2024-09-13T17:24:21.746228+0000","last_became_peered":"2024-09-13T17:24:21.746228+0000","last_unstale":"2024-09-13T17:26:11.562194+0000","last_undegraded":"2024-09-13T17:26:11.562194+0000","last_fullsized":"2024-09-13T17:26:11.562194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.618514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.543425+0000","last_change":"2024-09-13T17:24:21.745833+0000","last_active":"2024-09-13T17:26:11.543425+0000","last_peered":"2024-09-13T17:26:11.543425+0000","last_clean":"2024-09-13T17:26:11.543425+0000","last_became_active":"2024-09-13T17:24:21.745549+0000","last_became_peered":"2024-09-13T17:24:21.745549+0000","last_unstale":"2024-09-13T17:26:11.543425+0000","last_undegraded":"2024-09-13T17:26:11.543425+0000","last_fullsized":"2024-09-13T17:26:11.543425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:52.599227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.575431+0000","last_change":"2024-09-13T17:24:21.742651+0000","last_active":"2024-09-13T17:26:11.575431+0000","last_peered":"2024-09-13T17:26:11.575431+0000","last_clean":"2024-09-13T17:26:11.575431+0000","last_became_active":"2024-09-13T17:24:21.742368+0000","last_became_peered":"2024-09-13T17:24:21.742368+0000","last_unstale":"2024-09-13T17:26:11.575431+0000","last_undegraded":"2024-09-13T17:26:11.575431+0000","last_fullsized":"2024-09-13T17:26:11.575431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:28.741304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.561159+0000","last_change":"2024-09-13T17:24:42.565853+0000","last_active":"2024-09-13T17:26:11.561159+0000","last_peered":"2024-09-13T17:26:11.561159+0000","last_clean":"2024-09-13T17:26:11.561159+0000","last_became_active":"2024-09-13T17:24:42.565483+0000","last_became_peered":"2024-09-13T17:24:42.565483+0000","last_unstale":"2024-09-13T17:26:11.561159+0000","last_undegraded":"2024-09-13T17:26:11.561159+0000","last_fullsized":"2024-09-13T17:26:11.561159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:41.437228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.576704+0000","last_change":"2024-09-13T17:24:21.741010+0000","last_active":"2024-09-13T17:26:11.576704+0000","last_peered":"2024-09-13T17:26:11.576704+0000","last_clean":"2024-09-13T17:26:11.576704+0000","last_became_active":"2024-09-13T17:24:21.740652+0000","last_became_peered":"2024-09-13T17:24:21.740652+0000","last_unstale":"2024-09-13T17:26:11.576704+0000","last_undegraded":"2024-09-13T17:26:11.576704+0000","last_fullsized":"2024-09-13T17:26:11.576704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.436554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.572750+0000","last_change":"2024-09-13T17:24:21.742786+0000","last_active":"2024-09-13T17:26:11.572750+0000","last_peered":"2024-09-13T17:26:11.572750+0000","last_clean":"2024-09-13T17:26:11.572750+0000","last_became_active":"2024-09-13T17:24:21.742468+0000","last_became_peered":"2024-09-13T17:24:21.742468+0000","last_unstale":"2024-09-13T17:26:11.572750+0000","last_undegraded":"2024-09-13T17:26:11.572750+0000","last_fullsized":"2024-09-13T17:26:11.572750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:43.850055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.573518+0000","last_change":"2024-09-13T17:24:21.741069+0000","last_active":"2024-09-13T17:26:11.573518+0000","last_peered":"2024-09-13T17:26:11.573518+0000","last_clean":"2024-09-13T17:26:11.573518+0000","last_became_active":"2024-09-13T17:24:21.740723+0000","last_became_peered":"2024-09-13T17:24:21.740723+0000","last_unstale":"2024-09-13T17:26:11.573518+0000","last_undegraded":"2024-09-13T17:26:11.573518+0000","last_fullsized":"2024-09-13T17:26:11.573518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:33:50.000723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.556715+0000","last_change":"2024-09-13T17:24:21.749142+0000","last_active":"2024-09-13T17:26:11.556715+0000","last_peered":"2024-09-13T17:26:11.556715+0000","last_clean":"2024-09-13T17:26:11.556715+0000","last_became_active":"2024-09-13T17:24:21.748786+0000","last_became_peered":"2024-09-13T17:24:21.748786+0000","last_unstale":"2024-09-13T17:26:11.556715+0000","last_undegraded":"2024-09-13T17:26:11.556715+0000","last_fullsized":"2024-09-13T17:26:11.556715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:25.688436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.565559+0000","last_change":"2024-09-13T17:24:21.742152+0000","last_active":"2024-09-13T17:26:11.565559+0000","last_peered":"2024-09-13T17:26:11.565559+0000","last_clean":"2024-09-13T17:26:11.565559+0000","last_became_active":"2024-09-13T17:24:21.741811+0000","last_became_peered":"2024-09-13T17:24:21.741811+0000","last_unstale":"2024-09-13T17:26:11.565559+0000","last_undegraded":"2024-09-13T17:26:11.565559+0000","last_fullsized":"2024-09-13T17:26:11.565559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:02.386562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:11.571323+0000","last_change":"2024-09-13T17:24:42.564529+0000","last_active":"2024-09-13T17:26:11.571323+0000","last_peered":"2024-09-13T17:26:11.571323+0000","last_clean":"2024-09-13T17:26:11.571323+0000","last_became_active":"2024-09-13T17:24:42.564150+0000","last_became_peered":"2024-09-13T17:24:42.564150+0000","last_unstale":"2024-09-13T17:26:11.571323+0000","last_undegraded":"2024-09-13T17:26:11.571323+0000","last_fullsized":"2024-09-13T17:26:11.571323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:42.037884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917553+0000","last_change":"2024-09-13T17:24:21.742833+0000","last_active":"2024-09-13T17:26:30.917553+0000","last_peered":"2024-09-13T17:26:30.917553+0000","last_clean":"2024-09-13T17:26:30.917553+0000","last_became_active":"2024-09-13T17:24:21.742536+0000","last_became_peered":"2024-09-13T17:24:21.742536+0000","last_unstale":"2024-09-13T17:26:30.917553+0000","last_undegraded":"2024-09-13T17:26:30.917553+0000","last_fullsized":"2024-09-13T17:26:30.917553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:01.929525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:30.917595+0000","last_change":"2024-09-13T17:24:21.747767+0000","last_active":"2024-09-13T17:26:30.917595+0000","last_peered":"2024-09-13T17:26:30.917595+0000","last_clean":"2024-09-13T17:26:30.917595+0000","last_became_active":"2024-09-13T17:24:21.747453+0000","last_became_peered":"2024-09-13T17:24:21.747453+0000","last_unstale":"2024-09-13T17:26:30.917595+0000","last_undegraded":"2024-09-13T17:26:30.917595+0000","last_fullsized":"2024-09-13T17:26:30.917595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:08.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T17:26:29.237162+0000","last_change":"2024-09-13T17:24:21.748687+0000","last_active":"2024-09-13T17:26:29.237162+0000","last_peered":"2024-09-13T17:26:29.237162+0000","last_clean":"2024-09-13T17:26:29.237162+0000","last_became_active":"2024-09-13T17:24:21.748363+0000","last_became_peered":"2024-09-13T17:24:21.748363+0000","last_unstale":"2024-09-13T17:26:29.237162+0000","last_undegraded":"2024-09-13T17:26:29.237162+0000","last_fullsized":"2024-09-13T17:26:29.237162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:24:21.358787+0000","last_clean_scrub_stamp":"2024-09-13T17:24:21.358787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:58.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598371,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":832,"num_read_kb":528,"num_write":282,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1795113,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108906,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416424,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":524981,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70699999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.57899999999999996,"5min":0.745,"15min":0.745},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.30399999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.81699999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.371},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61499999999999999},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82899999999999996,"5min":1.05,"15min":1.05},"last":0.35799999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85199999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86299999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.53700000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.97099999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.50800000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1196032,"data_stored":1317091,"data_compressed":68931,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56699999999999995,"15min":0.57399999999999995},"min":{"1min":0.35499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.435},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.438,"15min":0.438},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75600000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.58799999999999997},"min":{"1min":0.38200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74399999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.376}]},{"osd":5,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.503}]},{"osd":6,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":1.073,"15min":1.073},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.91300000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.496}]},{"osd":9,"last update":"Fri Sep 13 17:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70399999999999996,"15min":0.72199999999999998},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88600000000000001,"5min":1.147,"15min":1.147},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.52700000000000002}]},{"osd":11,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82399999999999995,"5min":1.175,"15min":1.175},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91700000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.54600000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":918411,"data_compressed":45619,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45500000000000002,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64200000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.49299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68200000000000005,"5min":0.999,"15min":0.999},"last":0.24399999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.852,"5min":13.852,"15min":13.852},"last":1.04}]},{"osd":2,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":0.88300000000000001,"15min":0.77400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":17.055,"5min":17.055,"15min":17.055},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61399999999999999,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57699999999999996,"15min":0.623},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.77700000000000002,"15min":0.71699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.074999999999999,"5min":17.074999999999999,"15min":17.074999999999999},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67200000000000004,"15min":0.70899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88700000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.42199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:23:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.753,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":1.054,"15min":1.054},"last":1.1519999999999999}]},{"osd":8,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86499999999999999,"5min":1.083,"15min":1.083},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.71799999999999997,"15min":0.75600000000000001},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93700000000000006,"5min":1.04,"15min":1.04},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.006,"5min":1.288,"15min":1.288},"last":0.69899999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":0.83099999999999996,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":17.100999999999999,"5min":17.100999999999999,"15min":17.100999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.78800000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103223,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":917978,"data_compressed":45619,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.83899999999999997,"15min":0.97499999999999998},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.5679999999999996,"5min":12.994999999999999,"15min":12.994999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":0.68500000000000005,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":16.945,"5min":16.945,"15min":16.945},"last":0.372}]},{"osd":1,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":0.77400000000000002,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.544,"5min":22.544,"15min":22.544},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.59299999999999997,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.56399999999999995,"15min":0.46800000000000003},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":15.054,"5min":15.054,"15min":15.054},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.125,"5min":0.56699999999999995,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":14.135999999999999,"5min":14.135999999999999,"15min":14.135999999999999},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":0.73099999999999998,"15min":0.61499999999999999},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":19.047000000000001,"5min":19.047000000000001,"15min":19.047000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.778,"5min":3.778,"15min":3.778},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.61899999999999999,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.3220000000000001,"5min":7.3220000000000001,"15min":7.3220000000000001},"last":0.496},{"interface":"front","average":{"1min":1.077,"5min":0.68700000000000006,"15min":0.62},"min":{"1min":0.50800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":9.7200000000000006,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.8170000000000002,"5min":3.8170000000000002,"15min":3.8170000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":0.68300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":16.911000000000001,"5min":16.911000000000001,"15min":16.911000000000001},"last":0.436}]},{"osd":6,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.7280000000000002,"5min":3.7280000000000002,"15min":3.7280000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.151,"5min":0.754,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.5979999999999999,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.71999999999999997,"15min":0.66000000000000003},"min":{"1min":0.51500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7279999999999998,"5min":6.7279999999999998,"15min":6.7279999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":0.80400000000000005,"15min":0.68799999999999994},"min":{"1min":0.52800000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":16.847000000000001,"5min":16.847000000000001,"15min":16.847000000000001},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 17:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.73099999999999998,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.44,"15min":0.44},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.58599999999999997}]},{"osd":10,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.57499999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.57299999999999995}]},{"osd":11,"last update":"Fri Sep 13 17:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.82099999999999995,"15min":0.80300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.9239999999999999,"5min":3.9239999999999999,"15min":3.9239999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.101,"5min":0.81599999999999995,"15min":0.76800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.9370000000000003,"5min":7.9370000000000003,"15min":7.9370000000000003},"last":0.59599999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":1089536,"data_stored":1122450,"data_compressed":62775,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71699999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.39800000000000002,"15min":0.379},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.313}]},{"osd":2,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.439}]},{"osd":4,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45400000000000001,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":0.75,"15min":0.75},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.34399999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80600000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:26:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.746,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83999999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41699999999999998}]},{"osd":8,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.627},"min":{"1min":0.53200000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47799999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52100000000000002}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84999999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.502}]},{"osd":11,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.67100000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59099999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":520371,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51800000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.314},{"interface":"front","average":{"1min":0.372,"5min":0.39400000000000002,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.45900000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.751,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76800000000000002,"5min":1.008,"15min":1.008},"last":0.441}]},{"osd":2,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73299999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.49199999999999999},"min":{"1min":0.36399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67300000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.66600000000000004,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.623},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":1.018,"15min":1.018},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91900000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.52200000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.046,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53100000000000003}]},{"osd":10,"last update":"Fri Sep 13 17:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90200000000000002,"5min":1.298,"15min":1.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.139,"15min":1.139},"last":0.65900000000000003}]},{"osd":11,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.61099999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.93799999999999994,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.77300000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.745}]}]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1118208,"data_stored":1248267,"data_compressed":68931,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.502,"15min":0.47699999999999998},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.39800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76200000000000001,"5min":1.415,"15min":1.415},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.35999999999999999},"min":{"1min":0.28699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48999999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.36899999999999999},"min":{"1min":0.28199999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.496,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54200000000000004,"15min":0.48699999999999999},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":1.522,"15min":1.522},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.42499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90000000000000002,"5min":1.3,"15min":1.3},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.85499999999999998,"5min":1.244,"15min":1.244},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.87,"5min":1.272,"15min":1.272},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77700000000000002,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":4.327,"15min":4.327},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56000000000000005,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":1.212,"15min":1.212},"last":0.44}]},{"osd":9,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51500000000000001}]},{"osd":10,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.67900000000000005},"min":{"1min":0.496,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83499999999999996,"5min":1.337,"15min":1.337},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.63600000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":526392,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:23:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80900000000000005,"5min":0.876,"15min":0.876},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.34300000000000003},"min":{"1min":0.32300000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.497,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.29699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 17:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55700000000000005,"15min":0.5},"min":{"1min":0.48999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83599999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.441},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.53200000000000003,"15min":0.502},"min":{"1min":0.45800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35899999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.50800000000000001},"min":{"1min":0.47999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78700000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57299999999999995,"15min":0.51700000000000002},"min":{"1min":0.53100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.76300000000000001,"15min":0.63},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81100000000000005,"5min":14.458,"15min":14.458},"last":0.39900000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.309,"15min":0.309},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.45600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73499999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45700000000000002}]},{"osd":8,"last update":"Fri Sep 13 17:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.55000000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.432},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.495,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55200000000000005}]},{"osd":9,"last update":"Fri Sep 13 17:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.58699999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.501}]},{"osd":10,"last update":"Fri Sep 13 17:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.54200000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.878,"5min":1.02,"15min":1.02},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41399999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.65400000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.60999999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":1077248,"data_stored":1045381,"data_compressed":51775,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.35999999999999999},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.19500000000000001},"max":{"1min":0.51300000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42499999999999999,"15min":0.373},"min":{"1min":0.28899999999999998,"5min":0.253,"15min":0.246},"max":{"1min":0.54000000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91600000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.64000000000000001},"min":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":4.3979999999999997},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 17:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60299999999999998,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.6040000000000001,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:26:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.115,"15min":1.115},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.65000000000000002},"min":{"1min":0.5,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.64100000000000001},"min":{"1min":0.497,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82699999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Sep 13 17:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.627}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98899999999999999,"5min":1.069,"15min":1.069},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.498}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73599999999999999,"5min":1.05,"15min":1.05},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.78200000000000003,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.81000000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92800000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.85499999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":1114112,"data_stored":589874,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52900000000000003,"15min":0.46300000000000002},"min":{"1min":0.38800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.313},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.40799999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68799999999999994,"5min":0.753,"15min":0.753},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54700000000000004,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.48399999999999999},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.34699999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.44700000000000001,"15min":0.41499999999999998},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.375},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.495,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.55100000000000005,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56299999999999994,"15min":0.53200000000000003},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":9,"last update":"Fri Sep 13 17:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.625,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Sep 13 17:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63700000000000001}]},{"osd":11,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.628},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54900000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":1093632,"data_stored":588409,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.313},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84499999999999997,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.57699999999999996,"15min":0.54100000000000004},"min":{"1min":0.48699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":1.536,"15min":1.536},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.50800000000000001},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.89400000000000002,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.39300000000000002},"min":{"1min":0.34000000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.40899999999999997},"min":{"1min":0.33000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73999999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81499999999999995,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.504},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.57699999999999996},"min":{"1min":0.57399999999999995,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86199999999999999,"5min":1.613,"15min":1.613},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Sep 13 17:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83699999999999997,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.86199999999999999,"5min":2.504,"15min":2.504},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.53500000000000003},"min":{"1min":0.45300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94199999999999995,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.55300000000000005}]},{"osd":8,"last update":"Fri Sep 13 17:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.60399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96799999999999997,"5min":1.458,"15min":1.458},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.61699999999999999},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.67500000000000004}]},{"osd":9,"last update":"Fri Sep 13 17:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.58899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98099999999999998,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.60399999999999998}]},{"osd":10,"last update":"Fri Sep 13 17:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.58799999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":1.655,"15min":1.655},"last":0.58399999999999996}]},{"osd":11,"last update":"Fri Sep 13 17:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96499999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.69099999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.008,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.69599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":521759,"data_compressed":26473,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.45800000000000002},"min":{"1min":0.40400000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.51000000000000001},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.73599999999999999,"15min":0.56699999999999995},"min":{"1min":0.42999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75900000000000001,"5min":15.816000000000001,"15min":15.816000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.28499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78000000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65500000000000003,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.75}]},{"osd":4,"last update":"Fri Sep 13 17:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.70699999999999996,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":17.844999999999999,"5min":17.844999999999999,"15min":17.844999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42599999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.78500000000000003,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78900000000000003,"5min":15.723000000000001,"15min":15.723000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72899999999999998,"5min":1.766,"15min":1.766},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75700000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89400000000000002,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Sep 13 17:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84899999999999998,"5min":1.472,"15min":1.472},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.54700000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri Sep 13 17:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89700000000000002,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Sep 13 17:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86299999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85999999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72599999999999998}]},{"osd":11,"last update":"Fri Sep 13 17:26:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.91900000000000004,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84499999999999997,"5min":1.048,"15min":1.048},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36302,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133522,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202677,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:26:48.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 -- 172.21.15.44:0/2594615952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9940ff860 msgr2=0x7fc9941038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/2594615952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9940ff860 0x7fc9941038a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc984009930 tx=0x7fc98402a420 comp rx=0 tx=0).stop 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 -- 172.21.15.44:0/2594615952 shutdown_connections 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/2594615952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc994104000 0x7fc99410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/2594615952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9940ff860 0x7fc9941038a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/2594615952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940feeb0 0x7fc9940ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 -- 172.21.15.44:0/2594615952 >> 172.21.15.44:0/2594615952 conn(0x7fc9940fa900 msgr2=0x7fc9940fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 -- 172.21.15.44:0/2594615952 shutdown_connections 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 -- 172.21.15.44:0/2594615952 wait complete. 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.279+0000 7fc99b5f4640 1 Processor -- start 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 -- start start 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940feeb0 0x7fc994100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 0x7fc994100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941013c0 0x7fc9941a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a4af0 con 0x7fc994104000 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a4c60 con 0x7fc9940feeb0 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc99b5f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a4e20 con 0x7fc9941013c0 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 0x7fc994100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 0x7fc994100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40464/0 (socket says 172.21.15.44:40464) 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 -- 172.21.15.44:0/3924053170 learned_addr learned my addr 172.21.15.44:0/3924053170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc999390640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940feeb0 0x7fc994100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc999b91640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941013c0 0x7fc9941a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 -- 172.21.15.44:0/3924053170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941013c0 msgr2=0x7fc9941a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941013c0 0x7fc9941a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 -- 172.21.15.44:0/3924053170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940feeb0 msgr2=0x7fc994100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940feeb0 0x7fc994100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 -- 172.21.15.44:0/3924053170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984009560 con 0x7fc994104000 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.280+0000 7fc998b8f640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 0x7fc994100e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc984005ab0 tx=0x7fc984032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.281+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9840044d0 con 0x7fc994104000 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.281+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc984032e10 con 0x7fc994104000 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.281+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9941a50a0 con 0x7fc994104000 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.281+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984003850 con 0x7fc994104000 2024-09-13T17:26:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.281+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9941a55a0 con 0x7fc994104000 2024-09-13T17:26:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.282+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994100380 con 0x7fc994104000 2024-09-13T17:26:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.283+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fc984004850 con 0x7fc994104000 2024-09-13T17:26:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.284+0000 7fc98a7fc640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9800a13a0 0x7fc9800a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.284+0000 7fc999390640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9800a13a0 0x7fc9800a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.284+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc9840ee090 con 0x7fc994104000 2024-09-13T17:26:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.284+0000 7fc999390640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9800a13a0 0x7fc9800a3860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc990009bd0 tx=0x7fc990005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:48.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.287+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc9840b7040 con 0x7fc994104000 2024-09-13T17:26:48.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:48 smithi044 ceph-mon[31175]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:48.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:48 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3480678730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:48.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:48 smithi044 ceph-mon[31175]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-13T17:26:48.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:48 smithi070 ceph-mon[34502]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:48.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:48 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3480678730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:48.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:48 smithi070 ceph-mon[34502]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-13T17:26:48.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.463+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9940ff290 con 0x7fc994104000 2024-09-13T17:26:48.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.464+0000 7fc98a7fc640 1 -- 172.21.15.44:0/3924053170 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24868 (secure 0 0 0) 0x7fc984040090 con 0x7fc994104000 2024-09-13T17:26:48.466 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:48.466 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":93,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:26:42.344870+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:48.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9800a13a0 msgr2=0x7fc9800a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9800a13a0 0x7fc9800a3860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc990009bd0 tx=0x7fc990005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 msgr2=0x7fc994100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 0x7fc994100e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc984005ab0 tx=0x7fc984032c20 comp rx=0 tx=0).stop 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 shutdown_connections 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9800a13a0 0x7fc9800a3860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941013c0 0x7fc9941a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994104000 0x7fc994100e80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 --2- 172.21.15.44:0/3924053170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940feeb0 0x7fc994100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 >> 172.21.15.44:0/3924053170 conn(0x7fc9940fa900 msgr2=0x7fc994068930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.466+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 shutdown_connections 2024-09-13T17:26:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.467+0000 7fc99b5f4640 1 -- 172.21.15.44:0/3924053170 wait complete. 2024-09-13T17:26:48.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:26:48.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:26:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:48 smithi100 ceph-mon[35055]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:48 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3480678730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:48 smithi100 ceph-mon[35055]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-13T17:26:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.584+0000 7f8014f7d640 1 -- 172.21.15.44:0/4035136690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010100e50 msgr2=0x7f8010101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.584+0000 7f8014f7d640 1 --2- 172.21.15.44:0/4035136690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010100e50 0x7f8010101230 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f800002a410 comp rx=0 tx=0).stop 2024-09-13T17:26:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 -- 172.21.15.44:0/4035136690 shutdown_connections 2024-09-13T17:26:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 --2- 172.21.15.44:0/4035136690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801010a350 0x7f801010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 --2- 172.21.15.44:0/4035136690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8010101800 0x7f8010109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 --2- 172.21.15.44:0/4035136690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010100e50 0x7f8010101230 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 -- 172.21.15.44:0/4035136690 >> 172.21.15.44:0/4035136690 conn(0x7f80100fa930 msgr2=0x7f80100fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 -- 172.21.15.44:0/4035136690 shutdown_connections 2024-09-13T17:26:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 -- 172.21.15.44:0/4035136690 wait complete. 2024-09-13T17:26:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 Processor -- start 2024-09-13T17:26:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.585+0000 7f8014f7d640 1 -- start start 2024-09-13T17:26:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8010100e50 0x7f8010196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 0x7f8010196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801010a350 0x7f801019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801019b060 con 0x7f8010101800 2024-09-13T17:26:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801019b1d0 con 0x7f801010a350 2024-09-13T17:26:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801019b390 con 0x7f8010100e50 2024-09-13T17:26:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 0x7f8010196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 0x7f8010196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40480/0 (socket says 172.21.15.44:40480) 2024-09-13T17:26:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 -- 172.21.15.44:0/391731122 learned_addr learned my addr 172.21.15.44:0/391731122 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800ed9d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801010a350 0x7f801019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800e59c640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8010100e50 0x7f8010196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 -- 172.21.15.44:0/391731122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8010100e50 msgr2=0x7f8010196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8010100e50 0x7f8010196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 -- 172.21.15.44:0/391731122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801010a350 msgr2=0x7f801019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801010a350 0x7f801019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 -- 172.21.15.44:0/391731122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009560 con 0x7f8010101800 2024-09-13T17:26:48.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f800dd9b640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 0x7f8010196670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f800400e9f0 tx=0x7f800400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:48.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004009700 con 0x7f8010101800 2024-09-13T17:26:48.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f800400cd80 con 0x7f8010101800 2024-09-13T17:26:48.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801019b670 con 0x7f8010101800 2024-09-13T17:26:48.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.586+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040187e0 con 0x7f8010101800 2024-09-13T17:26:48.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.587+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80101a2ef0 con 0x7f8010101800 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.588+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8010068e50 con 0x7f8010101800 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.589+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f8004010430 con 0x7f8010101800 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.589+0000 7f7fff7fe640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7ff80a1320 0x7f7ff80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.590+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8004014070 con 0x7f8010101800 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.590+0000 7f800e59c640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7ff80a1320 0x7f7ff80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.590+0000 7f800e59c640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7ff80a1320 0x7f7ff80a37e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f800002aa00 tx=0x7f8000005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:48.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.593+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8004052e20 con 0x7f8010101800 2024-09-13T17:26:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.763+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8010105380 con 0x7f8010101800 2024-09-13T17:26:48.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.764+0000 7f7fff7fe640 1 -- 172.21.15.44:0/391731122 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8004096540 con 0x7f8010101800 2024-09-13T17:26:48.765 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7ff80a1320 msgr2=0x7f7ff80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7ff80a1320 0x7f7ff80a37e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f800002aa00 tx=0x7f8000005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 msgr2=0x7f8010196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 0x7f8010196670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f800400e9f0 tx=0x7f800400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 shutdown_connections 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7ff80a1320 0x7f7ff80a37e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801010a350 0x7f801019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8010101800 0x7f8010196670 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 --2- 172.21.15.44:0/391731122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8010100e50 0x7f8010196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 >> 172.21.15.44:0/391731122 conn(0x7f80100fa930 msgr2=0x7f80100fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:48.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 shutdown_connections 2024-09-13T17:26:48.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.766+0000 7f8014f7d640 1 -- 172.21.15.44:0/391731122 wait complete. 2024-09-13T17:26:48.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:26:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 -- 172.21.15.44:0/735208133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 msgr2=0x7f9b0810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 --2- 172.21.15.44:0/735208133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0810ea80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009ca0 tx=0x7f9af802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 -- 172.21.15.44:0/735208133 shutdown_connections 2024-09-13T17:26:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 --2- 172.21.15.44:0/735208133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0810ea80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 --2- 172.21.15.44:0/735208133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b08067ae0 0x7f9b08067f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 --2- 172.21.15.44:0/735208133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b08100b70 0x7f9b080675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 -- 172.21.15.44:0/735208133 >> 172.21.15.44:0/735208133 conn(0x7f9b08074980 msgr2=0x7f9b08074d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 -- 172.21.15.44:0/735208133 shutdown_connections 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 -- 172.21.15.44:0/735208133 wait complete. 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 Processor -- start 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.884+0000 7f9b0759e640 1 -- start start 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b08067ae0 0x7f9b081982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b08100b70 0x7f9b08198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0759e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0819d200 con 0x7f9b0810c690 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0819d370 con 0x7f9b08067ae0 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0759e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0819d530 con 0x7f9b08100b70 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40492/0 (socket says 172.21.15.44:40492) 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 -- 172.21.15.44:0/3039604073 learned_addr learned my addr 172.21.15.44:0/3039604073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b0659c640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b08067ae0 0x7f9b081982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b05d9b640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b08100b70 0x7f9b08198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 -- 172.21.15.44:0/3039604073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b08100b70 msgr2=0x7f9b08198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b08100b70 0x7f9b08198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 -- 172.21.15.44:0/3039604073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b08067ae0 msgr2=0x7f9b081982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b08067ae0 0x7f9b081982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 -- 172.21.15.44:0/3039604073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009900 con 0x7f9b0810c690 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.885+0000 7f9b06d9d640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0819cc00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005cc0 tx=0x7f9af8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.886+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8003b70 con 0x7f9b0810c690 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.886+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9af8040070 con 0x7f9b0810c690 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.886+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0819d7b0 con 0x7f9b0810c690 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.886+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8004240 con 0x7f9b0810c690 2024-09-13T17:26:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.886+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b081a5090 con 0x7f9b0810c690 2024-09-13T17:26:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.887+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b080ff230 con 0x7f9b0810c690 2024-09-13T17:26:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.888+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f9af8042030 con 0x7f9b0810c690 2024-09-13T17:26:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.889+0000 7f9aff7fe640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9af40a1320 0x7f9af40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.889+0000 7f9b0659c640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9af40a1320 0x7f9af40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.889+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9af80ee640 con 0x7f9b0810c690 2024-09-13T17:26:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.889+0000 7f9b0659c640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9af40a1320 0x7f9af40a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009bd0 tx=0x7f9b00005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:48.892+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9af80b76c0 con 0x7f9b0810c690 2024-09-13T17:26:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.063+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b08199620 con 0x7f9b0810c690 2024-09-13T17:26:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.063+0000 7f9aff7fe640 1 -- 172.21.15.44:0/3039604073 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9af80bace0 con 0x7f9b0810c690 2024-09-13T17:26:49.065 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9af40a1320 msgr2=0x7f9af40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9af40a1320 0x7f9af40a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009bd0 tx=0x7f9b00005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 msgr2=0x7f9b0819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0819cc00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005cc0 tx=0x7f9af8009360 comp rx=0 tx=0).stop 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 shutdown_connections 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9af40a1320 0x7f9af40a37e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b08100b70 0x7f9b08198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b0810c690 0x7f9b0819cc00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 --2- 172.21.15.44:0/3039604073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b08067ae0 0x7f9b081982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.065+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 >> 172.21.15.44:0/3039604073 conn(0x7f9b08074980 msgr2=0x7f9b0810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.066+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 shutdown_connections 2024-09-13T17:26:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.066+0000 7f9b0759e640 1 -- 172.21.15.44:0/3039604073 wait complete. 2024-09-13T17:26:49.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:26:49.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 -- 172.21.15.44:0/631036403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 msgr2=0x7f6a7810c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/631036403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7810c7a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009ca0 tx=0x7f6a7002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 -- 172.21.15.44:0/631036403 shutdown_connections 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/631036403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a78101800 0x7f6a78109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/631036403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7810c7a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/631036403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a78100e50 0x7f6a78101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 -- 172.21.15.44:0/631036403 >> 172.21.15.44:0/631036403 conn(0x7f6a780fa930 msgr2=0x7f6a780fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 -- 172.21.15.44:0/631036403 shutdown_connections 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 -- 172.21.15.44:0/631036403 wait complete. 2024-09-13T17:26:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.182+0000 7f6a7e06b640 1 Processor -- start 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 -- start start 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a78100e50 0x7f6a78196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a78101800 0x7f6a781966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819b060 con 0x7f6a7810a350 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819b1d0 con 0x7f6a78101800 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a7e06b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819b390 con 0x7f6a78100e50 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40504/0 (socket says 172.21.15.44:40504) 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 -- 172.21.15.44:0/2010514726 learned_addr learned my addr 172.21.15.44:0/2010514726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a76ffd640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a78101800 0x7f6a781966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a777fe640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a78100e50 0x7f6a78196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 -- 172.21.15.44:0/2010514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a78100e50 msgr2=0x7f6a78196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a78100e50 0x7f6a78196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 -- 172.21.15.44:0/2010514726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a78101800 msgr2=0x7f6a781966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a78101800 0x7f6a781966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.183+0000 7f6a77fff640 1 -- 172.21.15.44:0/2010514726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a70009900 con 0x7f6a7810a350 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.184+0000 7f6a77fff640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7819aa60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009430 tx=0x7f6a70031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.184+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70003b70 con 0x7f6a7810a350 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.184+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a70040070 con 0x7f6a7810a350 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.184+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004260 con 0x7f6a7810a350 2024-09-13T17:26:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.184+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7819b610 con 0x7f6a7810a350 2024-09-13T17:26:49.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.184+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a781a2ef0 con 0x7f6a7810a350 2024-09-13T17:26:49.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.185+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a78068e50 con 0x7f6a7810a350 2024-09-13T17:26:49.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.186+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f6a70042030 con 0x7f6a7810a350 2024-09-13T17:26:49.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.187+0000 7f6a74ff9640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a600a13a0 0x7f6a600a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.187+0000 7f6a777fe640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a600a13a0 0x7f6a600a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.187+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6a700ee500 con 0x7f6a7810a350 2024-09-13T17:26:49.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.187+0000 7f6a777fe640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a600a13a0 0x7f6a600a3860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009bd0 tx=0x7f6a68005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.191+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6a700b74f0 con 0x7f6a7810a350 2024-09-13T17:26:49.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:49 smithi044 ceph-mon[31175]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:49 smithi044 ceph-mon[31175]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:49 smithi044 ceph-mon[31175]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:49 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3924053170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:49.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:49 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/391731122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:49.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.361+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a78197480 con 0x7f6a7810a350 2024-09-13T17:26:49.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.361+0000 7f6a74ff9640 1 -- 172.21.15.44:0/2010514726 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a700bab10 con 0x7f6a7810a350 2024-09-13T17:26:49.362 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:26:49.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a600a13a0 msgr2=0x7f6a600a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a600a13a0 0x7f6a600a3860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009bd0 tx=0x7f6a68005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:49.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 msgr2=0x7f6a7819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7819aa60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009430 tx=0x7f6a70031680 comp rx=0 tx=0).stop 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 shutdown_connections 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a600a13a0 0x7f6a600a3860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a7810a350 0x7f6a7819aa60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a78101800 0x7f6a781966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 --2- 172.21.15.44:0/2010514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a78100e50 0x7f6a78196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 >> 172.21.15.44:0/2010514726 conn(0x7f6a780fa930 msgr2=0x7f6a780fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 shutdown_connections 2024-09-13T17:26:49.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.363+0000 7f6a7e06b640 1 -- 172.21.15.44:0/2010514726 wait complete. 2024-09-13T17:26:49.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:26:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:49 smithi070 ceph-mon[34502]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:49 smithi070 ceph-mon[34502]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:49 smithi070 ceph-mon[34502]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:49 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3924053170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:49 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/391731122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 -- 172.21.15.44:0/1734746055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640fee80 msgr2=0x7ff1640ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 --2- 172.21.15.44:0/1734746055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640fee80 0x7ff1640ff260 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff160009930 tx=0x7ff16002a420 comp rx=0 tx=0).stop 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 -- 172.21.15.44:0/1734746055 shutdown_connections 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 --2- 172.21.15.44:0/1734746055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff164103fd0 0x7ff16410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 --2- 172.21.15.44:0/1734746055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1640ff830 0x7ff164103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 --2- 172.21.15.44:0/1734746055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640fee80 0x7ff1640ff260 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 -- 172.21.15.44:0/1734746055 >> 172.21.15.44:0/1734746055 conn(0x7ff1640fa930 msgr2=0x7ff1640fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 -- 172.21.15.44:0/1734746055 shutdown_connections 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 -- 172.21.15.44:0/1734746055 wait complete. 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.479+0000 7ff16c019640 1 Processor -- start 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 -- start start 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1640fee80 0x7ff1641961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff164103fd0 0x7ff1641966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 0x7ff16419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419b4c0 con 0x7ff16419aad0 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419b630 con 0x7ff164103fd0 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16c019640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419b7a0 con 0x7ff1640fee80 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 0x7ff16419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 0x7ff16419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54224/0 (socket says 172.21.15.44:54224) 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 -- 172.21.15.44:0/603885082 learned_addr learned my addr 172.21.15.44:0/603885082 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff1695b4640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff164103fd0 0x7ff1641966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff169db5640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1640fee80 0x7ff1641961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 -- 172.21.15.44:0/603885082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1640fee80 msgr2=0x7ff1641961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1640fee80 0x7ff1641961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 -- 172.21.15.44:0/603885082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff164103fd0 msgr2=0x7ff1641966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff164103fd0 0x7ff1641966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 -- 172.21.15.44:0/603885082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff160009560 con 0x7ff16419aad0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.480+0000 7ff16a5b6640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 0x7ff16419af80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cad0 tx=0x7ff15c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.481+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00ae20 con 0x7ff16419aad0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.481+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff15c005020 con 0x7ff16419aad0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.481+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1641a2d80 con 0x7ff16419aad0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.481+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00f6b0 con 0x7ff16419aad0 2024-09-13T17:26:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.481+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1641a31e0 con 0x7ff16419aad0 2024-09-13T17:26:49.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.482+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164100350 con 0x7ff16419aad0 2024-09-13T17:26:49.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.483+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7ff15c005190 con 0x7ff16419aad0 2024-09-13T17:26:49.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.484+0000 7ff15affd640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1500a1320 0x7ff1500a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.484+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff15c0cd2e0 con 0x7ff16419aad0 2024-09-13T17:26:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.484+0000 7ff169db5640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1500a1320 0x7ff1500a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.484+0000 7ff169db5640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1500a1320 0x7ff1500a37e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff160006010 tx=0x7ff160005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.488+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff15c096160 con 0x7ff16419aad0 2024-09-13T17:26:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:49 smithi100 ceph-mon[35055]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:49 smithi100 ceph-mon[35055]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:49 smithi100 ceph-mon[35055]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T17:26:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:49 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3924053170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:49.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:49 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/391731122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:26:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.659+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff164108190 con 0x7ff16419aad0 2024-09-13T17:26:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.660+0000 7ff15affd640 1 -- 172.21.15.44:0/603885082 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff15c01d0a0 con 0x7ff16419aad0 2024-09-13T17:26:49.661 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:26:49.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1500a1320 msgr2=0x7ff1500a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1500a1320 0x7ff1500a37e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff160006010 tx=0x7ff160005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 msgr2=0x7ff16419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 0x7ff16419af80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cad0 tx=0x7ff15c00cf90 comp rx=0 tx=0).stop 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 shutdown_connections 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1500a1320 0x7ff1500a37e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16419aad0 0x7ff16419af80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff164103fd0 0x7ff1641966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 --2- 172.21.15.44:0/603885082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1640fee80 0x7ff1641961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 >> 172.21.15.44:0/603885082 conn(0x7ff1640fa930 msgr2=0x7ff1640689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 shutdown_connections 2024-09-13T17:26:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.662+0000 7ff16c019640 1 -- 172.21.15.44:0/603885082 wait complete. 2024-09-13T17:26:49.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:26:49.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 -- 172.21.15.44:0/3339297444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6330100940 msgr2=0x7f6330104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 --2- 172.21.15.44:0/3339297444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6330100940 0x7f6330104980 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6324009930 tx=0x7f632402a420 comp rx=0 tx=0).stop 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 -- 172.21.15.44:0/3339297444 shutdown_connections 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 --2- 172.21.15.44:0/3339297444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63301050e0 0x7f633010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 --2- 172.21.15.44:0/3339297444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6330100940 0x7f6330104980 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 --2- 172.21.15.44:0/3339297444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63300fff90 0x7f6330100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 -- 172.21.15.44:0/3339297444 >> 172.21.15.44:0/3339297444 conn(0x7f63300fba20 msgr2=0x7f63300fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 -- 172.21.15.44:0/3339297444 shutdown_connections 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.777+0000 7f63368fe640 1 -- 172.21.15.44:0/3339297444 wait complete. 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 Processor -- start 2024-09-13T17:26:49.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 -- start start 2024-09-13T17:26:49.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300fff90 0x7f6330196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63301050e0 0x7f6330196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 0x7f633019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633019b540 con 0x7f633019ab50 2024-09-13T17:26:49.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633019b6b0 con 0x7f63300fff90 2024-09-13T17:26:49.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f63368fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633019b820 con 0x7f63301050e0 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f6334e9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 0x7f633019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f6334e9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 0x7f633019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54248/0 (socket says 172.21.15.44:54248) 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f6334e9b640 1 -- 172.21.15.44:0/3890928905 learned_addr learned my addr 172.21.15.44:0/3890928905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f632f7fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63301050e0 0x7f6330196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.778+0000 7f632ffff640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300fff90 0x7f6330196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f6334e9b640 1 -- 172.21.15.44:0/3890928905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63301050e0 msgr2=0x7f6330196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f6334e9b640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63301050e0 0x7f6330196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f6334e9b640 1 -- 172.21.15.44:0/3890928905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300fff90 msgr2=0x7f6330196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f6334e9b640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300fff90 0x7f6330196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f6334e9b640 1 -- 172.21.15.44:0/3890928905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009560 con 0x7f633019ab50 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f6334e9b640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 0x7f633019b000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f632800eaa0 tx=0x7f632800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800cdb0 con 0x7f633019ab50 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6328005020 con 0x7f633019ab50 2024-09-13T17:26:49.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280116d0 con 0x7f633019ab50 2024-09-13T17:26:49.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63301a2e00 con 0x7f633019ab50 2024-09-13T17:26:49.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.779+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63301a3230 con 0x7f633019ab50 2024-09-13T17:26:49.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.780+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6330101460 con 0x7f633019ab50 2024-09-13T17:26:49.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.781+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f6328005190 con 0x7f633019ab50 2024-09-13T17:26:49.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.782+0000 7f632d7fa640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63180a13a0 0x7f63180a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:49.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.782+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f63280cd330 con 0x7f633019ab50 2024-09-13T17:26:49.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.782+0000 7f632ffff640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63180a13a0 0x7f63180a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:49.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.783+0000 7f632ffff640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63180a13a0 0x7f63180a3860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6320009bd0 tx=0x7f6320005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:49.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.786+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f63280962c0 con 0x7f633019ab50 2024-09-13T17:26:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.997+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6330100370 con 0x7f633019ab50 2024-09-13T17:26:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:49.998+0000 7f632d7fa640 1 -- 172.21.15.44:0/3890928905 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f632801f090 con 0x7f633019ab50 2024-09-13T17:26:49.999 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:49.999 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:26:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:26:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63180a13a0 msgr2=0x7f63180a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63180a13a0 0x7f63180a3860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6320009bd0 tx=0x7f6320005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 msgr2=0x7f633019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 0x7f633019b000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f632800eaa0 tx=0x7f632800ef60 comp rx=0 tx=0).stop 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 shutdown_connections 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63180a13a0 0x7f63180a3860 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63301050e0 0x7f6330196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f633019ab50 0x7f633019b000 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 --2- 172.21.15.44:0/3890928905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300fff90 0x7f6330196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 >> 172.21.15.44:0/3890928905 conn(0x7f63300fba20 msgr2=0x7f63300fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 shutdown_connections 2024-09-13T17:26:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.000+0000 7f63368fe640 1 -- 172.21.15.44:0/3890928905 wait complete. 2024-09-13T17:26:50.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 -- 172.21.15.44:0/889664170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc0ff830 msgr2=0x7f45dc103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 --2- 172.21.15.44:0/889664170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc0ff830 0x7f45dc103870 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009930 tx=0x7f45d002a420 comp rx=0 tx=0).stop 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 -- 172.21.15.44:0/889664170 shutdown_connections 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 --2- 172.21.15.44:0/889664170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc103fd0 0x7f45dc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 --2- 172.21.15.44:0/889664170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc0ff830 0x7f45dc103870 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 --2- 172.21.15.44:0/889664170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc0fee80 0x7f45dc0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 -- 172.21.15.44:0/889664170 >> 172.21.15.44:0/889664170 conn(0x7f45dc075c60 msgr2=0x7f45dc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 -- 172.21.15.44:0/889664170 shutdown_connections 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 -- 172.21.15.44:0/889664170 wait complete. 2024-09-13T17:26:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 Processor -- start 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.115+0000 7f45e307d640 1 -- start start 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc0fee80 0x7f45dc1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 0x7f45dc196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc19aaf0 0x7f45dc19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19b4e0 con 0x7f45dc103fd0 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19b650 con 0x7f45dc19aaf0 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19b7c0 con 0x7f45dc0fee80 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 0x7f45dc196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 0x7f45dc196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54260/0 (socket says 172.21.15.44:54260) 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 -- 172.21.15.44:0/3753410799 learned_addr learned my addr 172.21.15.44:0/3753410799 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e161a640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc19aaf0 0x7f45dc19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e0e19640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc0fee80 0x7f45dc1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 -- 172.21.15.44:0/3753410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc0fee80 msgr2=0x7f45dc1961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc0fee80 0x7f45dc1961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 -- 172.21.15.44:0/3753410799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc19aaf0 msgr2=0x7f45dc19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc19aaf0 0x7f45dc19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 -- 172.21.15.44:0/3753410799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0009560 con 0x7f45dc103fd0 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e0e19640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc0fee80 0x7f45dc1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e161a640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc19aaf0 0x7f45dc19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45dbfff640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 0x7f45dc196700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005ab0 tx=0x7f45d0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00044d0 con 0x7f45dc103fd0 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45d0032e10 con 0x7f45dc103fd0 2024-09-13T17:26:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc1a2da0 con 0x7f45dc103fd0 2024-09-13T17:26:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.116+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003850 con 0x7f45dc103fd0 2024-09-13T17:26:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.117+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a31c0 con 0x7f45dc103fd0 2024-09-13T17:26:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.118+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc100350 con 0x7f45dc103fd0 2024-09-13T17:26:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.119+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f45d0004850 con 0x7f45dc103fd0 2024-09-13T17:26:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.119+0000 7f45d9ffb640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45c40a1320 0x7f45c40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.119+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f45d00ee210 con 0x7f45dc103fd0 2024-09-13T17:26:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.120+0000 7f45e0e19640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45c40a1320 0x7f45c40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.120+0000 7f45e0e19640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45c40a1320 0x7f45c40a37e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0058c0 tx=0x7f45cc005850 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.123+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f45d00b71c0 con 0x7f45dc103fd0 2024-09-13T17:26:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.295+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45dc0ff8f0 con 0x7f45dc103fd0 2024-09-13T17:26:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.296+0000 7f45d9ffb640 1 -- 172.21.15.44:0/3753410799 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24868 (secure 0 0 0) 0x7f45d0004e60 con 0x7f45dc103fd0 2024-09-13T17:26:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:26:50.298 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":93,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:26:42.344870+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45c40a1320 msgr2=0x7f45c40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45c40a1320 0x7f45c40a37e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0058c0 tx=0x7f45cc005850 comp rx=0 tx=0).stop 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 msgr2=0x7f45dc196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 0x7f45dc196700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005ab0 tx=0x7f45d0032c20 comp rx=0 tx=0).stop 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 shutdown_connections 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45c40a1320 0x7f45c40a37e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45dc19aaf0 0x7f45dc19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45dc103fd0 0x7f45dc196700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 --2- 172.21.15.44:0/3753410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45dc0fee80 0x7f45dc1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 >> 172.21.15.44:0/3753410799 conn(0x7f45dc075c60 msgr2=0x7f45dc067800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 shutdown_connections 2024-09-13T17:26:50.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.298+0000 7f45e307d640 1 -- 172.21.15.44:0/3753410799 wait complete. 2024-09-13T17:26:50.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:50 smithi044 ceph-mon[31175]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:50.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3039604073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2010514726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/603885082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3890928905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:50.311 DEBUG:tasks.fwd_scrub:Yielding 2024-09-13T17:26:50.311 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T17:26:50.320 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T17:26:50.320 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:26:50.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-13T17:26:50.321 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-13T17:26:50.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:26:50.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.425+0000 7f766c94c640 1 -- 172.21.15.44:0/2691605770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 msgr2=0x7f766810ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.425+0000 7f766c94c640 1 --2- 172.21.15.44:0/2691605770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f766810ba40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7660009890 tx=0x7f766002a380 comp rx=0 tx=0).stop 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 -- 172.21.15.44:0/2691605770 shutdown_connections 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 --2- 172.21.15.44:0/2691605770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76680feff0 0x7f76680ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 --2- 172.21.15.44:0/2691605770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f766810ba40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 --2- 172.21.15.44:0/2691605770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7668104c30 0x7f7668105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 -- 172.21.15.44:0/2691605770 >> 172.21.15.44:0/2691605770 conn(0x7f7668074940 msgr2=0x7f7668074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 -- 172.21.15.44:0/2691605770 shutdown_connections 2024-09-13T17:26:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 -- 172.21.15.44:0/2691605770 wait complete. 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 Processor -- start 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.426+0000 7f766c94c640 1 -- start start 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766c94c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 0x7f7668198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766c94c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f7668198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766c94c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7668104c30 0x7f766819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766c94c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766819d280 con 0x7f76680ff970 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766c94c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766819d3f0 con 0x7f76680feff0 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766c94c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766819d5b0 con 0x7f7668104c30 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f7668198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 0x7f7668198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f7668198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54280/0 (socket says 172.21.15.44:54280) 2024-09-13T17:26:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 0x7f7668198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:59132/0 (socket says 172.21.15.44:59132) 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 -- 172.21.15.44:0/1088360350 learned_addr learned my addr 172.21.15.44:0/1088360350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7666d9d640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7668104c30 0x7f766819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 -- 172.21.15.44:0/1088360350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7668104c30 msgr2=0x7f766819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7668104c30 0x7f766819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 -- 172.21.15.44:0/1088360350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 msgr2=0x7f7668198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 0x7f7668198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 -- 172.21.15.44:0/1088360350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658009560 con 0x7f76680ff970 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f766659c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 0x7f7668198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.427+0000 7f7665d9b640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f7668198890 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f765c00eb90 tx=0x7f765c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0099b0 con 0x7f76680ff970 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f765c00ee90 con 0x7f76680ff970 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7660009490 con 0x7f76680ff970 2024-09-13T17:26:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c010690 con 0x7f76680ff970 2024-09-13T17:26:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76681a4f30 con 0x7f76680ff970 2024-09-13T17:26:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 -- 172.21.15.44:0/53776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc0fee60 msgr2=0x7f3fcc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 --2- 172.21.15.44:0/53776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc0ff240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009930 tx=0x7f3fc802a420 comp rx=0 tx=0).stop 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 -- 172.21.15.44:0/53776182 shutdown_connections 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 --2- 172.21.15.44:0/53776182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc103fb0 0x7f3fcc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 --2- 172.21.15.44:0/53776182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0ff810 0x7f3fcc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 --2- 172.21.15.44:0/53776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc0ff240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 -- 172.21.15.44:0/53776182 >> 172.21.15.44:0/53776182 conn(0x7f3fcc075c60 msgr2=0x7f3fcc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 -- 172.21.15.44:0/53776182 shutdown_connections 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.428+0000 7f3fd371f640 1 -- 172.21.15.44:0/53776182 wait complete. 2024-09-13T17:26:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 Processor -- start 2024-09-13T17:26:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76681002b0 con 0x7f76680ff970 2024-09-13T17:26:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 -- start start 2024-09-13T17:26:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc103fb0 0x7f3fcc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc19ab40 0x7f3fcc19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc19b530 con 0x7f3fcc0fee60 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc19b6a0 con 0x7f3fcc19ab40 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd371f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc19b810 con 0x7f3fcc103fb0 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd14bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd14bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54292/0 (socket says 172.21.15.44:54292) 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd14bb640 1 -- 172.21.15.44:0/3988262720 learned_addr learned my addr 172.21.15.44:0/3988262720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc19ab40 0x7f3fcc19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.429+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc103fb0 0x7f3fcc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd14bb640 1 -- 172.21.15.44:0/3988262720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc103fb0 msgr2=0x7f3fcc196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc103fb0 0x7f3fcc196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd14bb640 1 -- 172.21.15.44:0/3988262720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc19ab40 msgr2=0x7f3fcc19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc19ab40 0x7f3fcc19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd14bb640 1 -- 172.21.15.44:0/3988262720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4009840 con 0x7f3fcc0fee60 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc196210 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005ab0 tx=0x7f3fc8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fbe7fc640 1 -- 172.21.15.44:0/3988262720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80044f0 con 0x7f3fcc0fee60 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8009560 con 0x7f3fcc0fee60 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fbe7fc640 1 -- 172.21.15.44:0/3988262720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc8032e10 con 0x7f3fcc0fee60 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fbe7fc640 1 -- 172.21.15.44:0/3988262720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8003850 con 0x7f3fcc0fee60 2024-09-13T17:26:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1a3090 con 0x7f3fcc0fee60 2024-09-13T17:26:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.430+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f765c005020 con 0x7f76680ff970 2024-09-13T17:26:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.431+0000 7f76577fe640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f76500a1320 0x7f76500a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.431+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f765c014070 con 0x7f76680ff970 2024-09-13T17:26:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.431+0000 7f766659c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f76500a1320 0x7f76500a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.432+0000 7f766659c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f76500a1320 0x7f76500a37e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7658009530 tx=0x7f7658009260 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.432+0000 7f3fbe7fc640 1 -- 172.21.15.44:0/3988262720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f3fc8005210 con 0x7f3fcc0fee60 2024-09-13T17:26:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.433+0000 7f3fbe7fc640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fb80a1320 0x7f3fb80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.433+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fb80a1320 0x7f3fb80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.433+0000 7f3fbe7fc640 1 -- 172.21.15.44:0/3988262720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3fc80edd40 con 0x7f3fcc0fee60 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 Processor -- start 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fb80a1320 0x7f3fb80a37e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc1977b0 tx=0x7f3fc000b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 -- start start 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b207ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b212440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b214d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b172c60 con 0x55ae3b2080e0 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b1703f0 con 0x55ae3b212980 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd6447740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b16bce0 con 0x55ae3b2077c0 2024-09-13T17:26:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b212440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd1cbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b214d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b212440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54304/0 (socket says 172.21.15.44:54304) 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2208059095 learned_addr learned my addr 172.21.15.44:0/2208059095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b207ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2208059095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 msgr2=0x55ae3b207ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b207ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2208059095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 msgr2=0x55ae3b214d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b214d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2208059095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae3b127970 con 0x55ae3b2080e0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.434+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b214d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.435+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b212440 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000bf80 tx=0x7f3fc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cba733bb6ef979 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.437+0000 7f3fb77fe640 1 -- 172.21.15.44:0/2208059095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc001e620 con 0x55ae3b2080e0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.437+0000 7f3fb77fe640 1 -- 172.21.15.44:0/2208059095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc0015070 con 0x55ae3b2080e0 2024-09-13T17:26:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.437+0000 7f3fb77fe640 1 -- 172.21.15.44:0/2208059095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc001c970 con 0x55ae3b2080e0 2024-09-13T17:26:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.437+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f765c0967c0 con 0x7f76680ff970 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 -- 172.21.15.44:0/2208059095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 msgr2=0x55ae3b212440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b212440 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000bf80 tx=0x7f3fc0013000 comp rx=0 tx=0).stop 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 -- 172.21.15.44:0/2208059095 shutdown_connections 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b214d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b212440 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2208059095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b207ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 -- 172.21.15.44:0/2208059095 >> 172.21.15.44:0/2208059095 conn(0x55ae3b203d50 msgr2=0x55ae3b204130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 -- 172.21.15.44:0/2208059095 shutdown_connections 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 -- 172.21.15.44:0/2208059095 wait complete. 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 -- start start 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.438+0000 7f3fd6447740 1 Processor -- start 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2077c0 0x55ae3b212280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b20f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b212980 0x55ae3b20fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b129f80 con 0x55ae3b2080e0 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b129c20 con 0x55ae3b2077c0 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b0f6ab0 con 0x55ae3b212980 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd6447740 10 client.0 _ll_register_callbacks cb 0x55ae3b165350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b20f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b20f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54308/0 (socket says 172.21.15.44:54308) 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2191356818 learned_addr learned my addr 172.21.15.44:0/2191356818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b212980 0x55ae3b20fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2077c0 0x55ae3b212280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b212980 msgr2=0x55ae3b20fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b212980 0x55ae3b20fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2191356818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2077c0 msgr2=0x55ae3b212280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2077c0 0x55ae3b212280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 -- 172.21.15.44:0/2191356818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc000bc60 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b20f550 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0004770 tx=0x7f3fc001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2976d8684d1873e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.439+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/2191356818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc002a070 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/2191356818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3fc0017070 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae3ae5f590 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/2191356818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc001c3b0 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 10 client.15675 fetch_fsmap learned FSMap version 25 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae3b16a360 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/2191356818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ae3ae5f590 con 0x55ae3b2080e0 2024-09-13T17:26:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/2191356818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fc0024020 con 0x55ae3b2080e0 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 10 client.15675 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 20 client.15675 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b2225e0 0x55ae3b224a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 4 client.15675 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae3ae70860 con 0x55ae3b2225e0 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.440+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b2225e0 0x55ae3b224a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.441+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b2225e0 0x55ae3b224a00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247fd0edf4dcd789 server_cookie=aecea4ab46021c93 in_seq=0 out_seq=0 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.441+0000 7f3fb4ff9640 10 client.15675 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.441+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/2191356818 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ae3ae70860 con 0x55ae3b2225e0 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.441+0000 7f3fb4ff9640 10 client.15675 handle_command_reply: tid=0 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.441+0000 7f3fd6447740 1 client.15675 shutdown 2024-09-13T17:26:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 msgr2=0x55ae3b20f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b20f550 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0004770 tx=0x7f3fc001c9a0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 shutdown_connections 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b2225e0 0x55ae3b224a00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b212980 0x55ae3b20fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2080e0 0x55ae3b20f550 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fb4ff9640 0 client.15675 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2191356818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2077c0 0x55ae3b212280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 >> 172.21.15.44:0/2191356818 conn(0x55ae3b203d50 msgr2=0x55ae3b2091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 shutdown_connections 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 1 -- 172.21.15.44:0/2191356818 wait complete. 2024-09-13T17:26:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.442+0000 7f3fd6447740 20 client.15675 trim_cache size 0 max 16384 2024-09-13T17:26:50.452 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:50 smithi070 ceph-mon[34502]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:50.452 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3039604073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.452 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2010514726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.452 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/603885082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.452 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3890928905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 Processor -- start 2024-09-13T17:26:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 -- start start 2024-09-13T17:26:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b20c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2080e0 0x55ae3b20cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 0x55ae3b2091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ae70860 con 0x55ae3b212980 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ae5f590 con 0x55ae3b2080e0 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.456+0000 7f3fd6447740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b208b40 con 0x55ae3b2077c0 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 0x55ae3b2091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 0x55ae3b2091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54310/0 (socket says 172.21.15.44:54310) 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 -- 172.21.15.44:0/2143107398 learned_addr learned my addr 172.21.15.44:0/2143107398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2080e0 0x55ae3b20cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b20c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 -- 172.21.15.44:0/2143107398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 msgr2=0x55ae3b20c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b20c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 -- 172.21.15.44:0/2143107398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2080e0 msgr2=0x55ae3b20cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2080e0 0x55ae3b20cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 -- 172.21.15.44:0/2143107398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae3b10c1c0 con 0x55ae3b212980 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2080e0 0x55ae3b20cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 0x55ae3b2091a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4003c90 tx=0x7f3fc4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712e510eeadfd7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fb7fff640 1 -- 172.21.15.44:0/2143107398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4016070 con 0x55ae3b212980 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fb7fff640 1 -- 172.21.15.44:0/2143107398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc4017040 con 0x55ae3b212980 2024-09-13T17:26:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.457+0000 7f3fb7fff640 1 -- 172.21.15.44:0/2143107398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4010960 con 0x55ae3b212980 2024-09-13T17:26:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 -- 172.21.15.44:0/2143107398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 msgr2=0x55ae3b2091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 0x55ae3b2091a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4003c90 tx=0x7f3fc4004410 comp rx=0 tx=0).stop 2024-09-13T17:26:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 -- 172.21.15.44:0/2143107398 shutdown_connections 2024-09-13T17:26:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b212980 0x55ae3b2091a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b2080e0 0x55ae3b20cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 --2- 172.21.15.44:0/2143107398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2077c0 0x55ae3b20c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 -- 172.21.15.44:0/2143107398 >> 172.21.15.44:0/2143107398 conn(0x55ae3b203d50 msgr2=0x55ae3b20de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 -- 172.21.15.44:0/2143107398 shutdown_connections 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 -- 172.21.15.44:0/2143107398 wait complete. 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 -- start start 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.458+0000 7f3fd6447740 1 Processor -- start 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 0x55ae3b218910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2080e0 0x55ae3b218e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b278af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b2790d0 con 0x55ae3b2077c0 2024-09-13T17:26:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b279240 con 0x55ae3b212980 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3b279400 con 0x55ae3b2080e0 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd6447740 10 client.0 _ll_register_callbacks cb 0x7f3fc000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 0x55ae3b218910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 0x55ae3b218910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54322/0 (socket says 172.21.15.44:54322) 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 -- 172.21.15.44:0/828659831 learned_addr learned my addr 172.21.15.44:0/828659831 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2080e0 0x55ae3b218e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b278af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 -- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2080e0 msgr2=0x55ae3b218e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2080e0 0x55ae3b218e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 -- 172.21.15.44:0/828659831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 msgr2=0x55ae3b278af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b278af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 -- 172.21.15.44:0/828659831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae3b16a360 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd1cbc640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b278af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fd14bb640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 0x55ae3b218910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80eb100 tx=0x7f3fc80baa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbbe4027e000008 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.459+0000 7f3fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/828659831 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80c2e30 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/828659831 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3fc80b8e00 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fc0015070 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/828659831 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc80b7170 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 10 client.15681 fetch_fsmap learned FSMap version 25 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f3fc4003970 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/828659831 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc809f080 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/828659831 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fc80c2a30 con 0x55ae3b2077c0 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 10 client.15681 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:26:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 20 client.15681 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b27a9b0 0x55ae3b27cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 4 client.15681 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ae3aec3eb0 con 0x55ae3b27a9b0 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.460+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b27a9b0 0x55ae3b27cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.461+0000 7f3fd0cba640 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b27a9b0 0x55ae3b27cda0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0acb3c8f1ef9c1c server_cookie=9159d5c0fc5bed6d in_seq=0 out_seq=0 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.461+0000 7f3fb4ff9640 10 client.15681 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fb4ff9640 1 -- 172.21.15.44:0/828659831 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ae3aec3eb0 con 0x55ae3b27a9b0 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fb4ff9640 10 client.15681 handle_command_reply: tid=0 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 client.15681 shutdown 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 msgr2=0x55ae3b218910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 0x55ae3b218910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80eb100 tx=0x7f3fc80baa60 comp rx=0 tx=0).stop 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 shutdown_connections 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ae3b27a9b0 0x55ae3b27cda0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae3b2080e0 0x55ae3b218e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae3b212980 0x55ae3b278af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fb4ff9640 0 client.15681 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 --2- 172.21.15.44:0/828659831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae3b2077c0 0x55ae3b218910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 >> 172.21.15.44:0/828659831 conn(0x55ae3b203d50 msgr2=0x55ae3b2115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 shutdown_connections 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 1 -- 172.21.15.44:0/828659831 wait complete. 2024-09-13T17:26:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.462+0000 7f3fd6447740 20 client.15681 trim_cache size 0 max 16384 2024-09-13T17:26:50.470 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:50.470 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:26:50.470 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "b3789354-bda6-425b-8724-d96841503a55", 2024-09-13T17:26:50.470 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:26:50.470 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fb80a1320 msgr2=0x7f3fb80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fb80a1320 0x7f3fb80a37e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc1977b0 tx=0x7f3fc000b410 comp rx=0 tx=0).stop 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 msgr2=0x7f3fcc196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc196210 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005ab0 tx=0x7f3fc8032c20 comp rx=0 tx=0).stop 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 shutdown_connections 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fb80a1320 0x7f3fb80a37e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fcc103fb0 0x7f3fcc196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc19ab40 0x7f3fcc19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 --2- 172.21.15.44:0/3988262720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fcc0fee60 0x7f3fcc196210 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 >> 172.21.15.44:0/3988262720 conn(0x7f3fcc075c60 msgr2=0x7f3fcc102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 shutdown_connections 2024-09-13T17:26:50.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.463+0000 7f3fd371f640 1 -- 172.21.15.44:0/3988262720 wait complete. 2024-09-13T17:26:50.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:26:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:50 smithi100 ceph-mon[35055]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:26:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3039604073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2010514726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/603885082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:26:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3890928905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:26:50.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 -- 172.21.15.44:0/4239300294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 msgr2=0x7f489810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 --2- 172.21.15.44:0/4239300294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f489810bb00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4890009cd0 tx=0x7f489002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 -- 172.21.15.44:0/4239300294 shutdown_connections 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 --2- 172.21.15.44:0/4239300294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f489810bb00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 --2- 172.21.15.44:0/4239300294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980ff810 0x7f4898103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 --2- 172.21.15.44:0/4239300294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48980fee60 0x7f48980ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 -- 172.21.15.44:0/4239300294 >> 172.21.15.44:0/4239300294 conn(0x7f4898075c60 msgr2=0x7f48980fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 -- 172.21.15.44:0/4239300294 shutdown_connections 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.580+0000 7f489f583640 1 -- 172.21.15.44:0/4239300294 wait complete. 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 Processor -- start 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 -- start start 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980fee60 0x7f4898196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f4898197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4898197a70 0x7f48981a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48981a2710 con 0x7f4898103fb0 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48981a2880 con 0x7f48980fee60 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489f583640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48981a2a40 con 0x7f4898197a70 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489cb1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f4898197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489cb1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f4898197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54346/0 (socket says 172.21.15.44:54346) 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489cb1e640 1 -- 172.21.15.44:0/1088662117 learned_addr learned my addr 172.21.15.44:0/1088662117 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.581+0000 7f489d31f640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980fee60 0x7f4898196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489db20640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4898197a70 0x7f48981a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489cb1e640 1 -- 172.21.15.44:0/1088662117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4898197a70 msgr2=0x7f48981a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4898197a70 0x7f48981a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489cb1e640 1 -- 172.21.15.44:0/1088662117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980fee60 msgr2=0x7f4898196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980fee60 0x7f4898196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489cb1e640 1 -- 172.21.15.44:0/1088662117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890009900 con 0x7f4898103fb0 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489d31f640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980fee60 0x7f4898196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f4898197530 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d770 tx=0x7f488c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f488a7fc640 1 -- 172.21.15.44:0/1088662117 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c009c30 con 0x7f4898103fb0 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f488a7fc640 1 -- 172.21.15.44:0/1088662117 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488c00de80 con 0x7f4898103fb0 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48981a2d20 con 0x7f4898103fb0 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f488a7fc640 1 -- 172.21.15.44:0/1088662117 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00f620 con 0x7f4898103fb0 2024-09-13T17:26:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.582+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48981a31d0 con 0x7f4898103fb0 2024-09-13T17:26:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 Processor -- start 2024-09-13T17:26:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 -- start start 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb8311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb83e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb79bf20 con 0x5599fb831730 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb7996b0 con 0x5599fb83bfd0 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f48a22a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb794fa0 con 0x5599fb830e10 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54358/0 (socket says 172.21.15.44:54358) 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 -- 172.21.15.44:0/638095038 learned_addr learned my addr 172.21.15.44:0/638095038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489db20640 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb8311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 -- 172.21.15.44:0/638095038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 msgr2=0x5599fb8311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb8311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 -- 172.21.15.44:0/638095038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 msgr2=0x5599fb83e3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb83e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f489d31f640 1 -- 172.21.15.44:0/638095038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599fb74f840 con 0x5599fb831730 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.585+0000 7f488a7fc640 1 -- 172.21.15.44:0/1088662117 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f488c010460 con 0x7f4898103fb0 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f489d31f640 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83ba90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f489400bd40 tx=0x7f489400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2463410bcefa5872 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f48837fe640 1 -- 172.21.15.44:0/638095038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489401a660 con 0x5599fb831730 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f48837fe640 1 -- 172.21.15.44:0/638095038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f489400be30 con 0x5599fb831730 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f48837fe640 1 -- 172.21.15.44:0/638095038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894011a00 con 0x5599fb831730 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f488a7fc640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f48840a1320 0x7f48840a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f489db20640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f48840a1320 0x7f48840a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f488a7fc640 1 -- 172.21.15.44:0/1088662117 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f488c0cd0d0 con 0x7f4898103fb0 2024-09-13T17:26:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f48a22a5740 1 -- 172.21.15.44:0/638095038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 msgr2=0x5599fb83ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.586+0000 7f48a22a5740 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83ba90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f489400bd40 tx=0x7f489400f730 comp rx=0 tx=0).stop 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 -- 172.21.15.44:0/638095038 shutdown_connections 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb83e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83ba90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 --2- 172.21.15.44:0/638095038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb8311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 -- 172.21.15.44:0/638095038 >> 172.21.15.44:0/638095038 conn(0x5599fb82d380 msgr2=0x5599fb82d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 -- 172.21.15.44:0/638095038 shutdown_connections 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f489db20640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f48840a1320 0x7f48840a37e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4890005e30 tx=0x7f4890005dc0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 -- 172.21.15.44:0/638095038 wait complete. 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 -- start start 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.587+0000 7f48a22a5740 1 Processor -- start 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb838d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb839480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb7362a0 con 0x5599fb831730 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb46d430 con 0x5599fb83bfd0 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb718680 con 0x5599fb830e10 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 10 client.0 _ll_register_callbacks cb 0x5599fb78e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb838d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb838d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54368/0 (socket says 172.21.15.44:54368) 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 -- 172.21.15.44:0/3880004745 learned_addr learned my addr 172.21.15.44:0/3880004745 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489d31f640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489db20640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb839480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 -- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 msgr2=0x5599fb83ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 -- 172.21.15.44:0/3880004745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 msgr2=0x5599fb839480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb839480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 -- 172.21.15.44:0/3880004745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489400b9a0 con 0x5599fb831730 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f489cb1e640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb838d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f488c000c00 tx=0x7f488c09a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1affd084b99b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f4880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f4880ff9640 1 -- 172.21.15.44:0/3880004745 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0a20f0 con 0x5599fb831730 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f4880ff9640 1 -- 172.21.15.44:0/3880004745 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f488c010710 con 0x5599fb831730 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.588+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599fb4884e0 con 0x5599fb831730 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f4880ff9640 1 -- 172.21.15.44:0/3880004745 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488c0108f0 con 0x5599fb831730 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 10 client.15693 fetch_fsmap learned FSMap version 25 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599fb793620 con 0x5599fb831730 2024-09-13T17:26:50.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f4880ff9640 1 -- 172.21.15.44:0/3880004745 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599fb4884e0 con 0x5599fb831730 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f4880ff9640 1 -- 172.21.15.44:0/3880004745 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f488c0ca6f0 con 0x5599fb831730 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 10 client.15693 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 20 client.15693 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb84bdc0 0x5599fb84e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 4 client.15693 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599fb4997e0 con 0x5599fb84bdc0 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.589+0000 7f489d31f640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb84bdc0 0x5599fb84e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f4880ff9640 10 client.15693 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f489d31f640 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb84bdc0 0x5599fb84e1b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d344b16538a5a10a server_cookie=c70c628260005c3d in_seq=0 out_seq=0 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f4880ff9640 1 -- 172.21.15.44:0/3880004745 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5599fb4997e0 con 0x5599fb84bdc0 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f4880ff9640 10 client.15693 handle_command_reply: tid=0 2024-09-13T17:26:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 client.15693 shutdown 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 msgr2=0x5599fb838d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb838d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f488c000c00 tx=0x7f488c09a430 comp rx=0 tx=0).stop 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 shutdown_connections 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb84bdc0 0x5599fb84e1b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb839480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f4880ff9640 0 client.15693 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb838d40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3880004745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.590+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 >> 172.21.15.44:0/3880004745 conn(0x5599fb82d380 msgr2=0x5599fb833090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.591+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 shutdown_connections 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.591+0000 7f48a22a5740 1 -- 172.21.15.44:0/3880004745 wait complete. 2024-09-13T17:26:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.591+0000 7f48a22a5740 20 client.15693 trim_cache size 0 max 16384 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 Processor -- start 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 -- start start 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb830e10 0x5599fb8477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb831730 0x5599fb833fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 0x5599fb834510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb82e070 con 0x5599fb83bfd0 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb83a1b0 con 0x5599fb830e10 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f48a22a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb84bbe0 con 0x5599fb831730 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f489db20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 0x5599fb834510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f489db20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 0x5599fb834510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54376/0 (socket says 172.21.15.44:54376) 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f489db20640 1 -- 172.21.15.44:0/1162768732 learned_addr learned my addr 172.21.15.44:0/1162768732 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb831730 0x5599fb833fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.605+0000 7f489d31f640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb830e10 0x5599fb8477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489cb1e640 1 -- 172.21.15.44:0/1162768732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb830e10 msgr2=0x5599fb8477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb830e10 0x5599fb8477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489cb1e640 1 -- 172.21.15.44:0/1162768732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 msgr2=0x5599fb834510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 0x5599fb834510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489cb1e640 1 -- 172.21.15.44:0/1162768732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599fb735790 con 0x5599fb831730 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489db20640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 0x5599fb834510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489d31f640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb830e10 0x5599fb8477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f489cb1e640 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb831730 0x5599fb833fd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f488c096170 tx=0x7f488c09dc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4289d554c40332c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f4883fff640 1 -- 172.21.15.44:0/1162768732 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c099a80 con 0x5599fb831730 2024-09-13T17:26:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f4883fff640 1 -- 172.21.15.44:0/1162768732 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488c0b8020 con 0x5599fb831730 2024-09-13T17:26:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.606+0000 7f4883fff640 1 -- 172.21.15.44:0/1162768732 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c010710 con 0x5599fb831730 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 -- 172.21.15.44:0/1162768732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb831730 msgr2=0x5599fb833fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb831730 0x5599fb833fd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f488c096170 tx=0x7f488c09dc20 comp rx=0 tx=0).stop 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 -- 172.21.15.44:0/1162768732 shutdown_connections 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb831730 0x5599fb833fd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb83bfd0 0x5599fb834510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 --2- 172.21.15.44:0/1162768732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb830e10 0x5599fb8477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 -- 172.21.15.44:0/1162768732 >> 172.21.15.44:0/1162768732 conn(0x5599fb82d380 msgr2=0x5599fb852b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 -- 172.21.15.44:0/1162768732 shutdown_connections 2024-09-13T17:26:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 -- 172.21.15.44:0/1162768732 wait complete. 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 -- start start 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.607+0000 7f48a22a5740 1 Processor -- start 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb89a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb89a7b0 con 0x5599fb831730 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb89a920 con 0x5599fb83bfd0 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fb89aae0 con 0x5599fb830e10 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 10 client.0 _ll_register_callbacks cb 0x7f488c010bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54392/0 (socket says 172.21.15.44:54392) 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 -- 172.21.15.44:0/3490874095 learned_addr learned my addr 172.21.15.44:0/3490874095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489d31f640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489db20640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb89a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 -- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 msgr2=0x5599fb83f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 -- 172.21.15.44:0/3490874095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 msgr2=0x5599fb89a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb89a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 -- 172.21.15.44:0/3490874095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599fb74f840 con 0x5599fb831730 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489db20640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb89a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f489cb1e640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83f880 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f488c099740 tx=0x7f488c00d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff30b6e9b1b7d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f4880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f4880ff9640 1 -- 172.21.15.44:0/3490874095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0a1e60 con 0x5599fb831730 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f4880ff9640 1 -- 172.21.15.44:0/3490874095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f488c0a20f0 con 0x5599fb831730 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f489400be30 con 0x5599fb831730 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f4880ff9640 1 -- 172.21.15.44:0/3490874095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488c010920 con 0x5599fb831730 2024-09-13T17:26:50.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 10 client.15699 fetch_fsmap learned FSMap version 25 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.608+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f489810c340 con 0x5599fb831730 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f4880ff9640 1 -- 172.21.15.44:0/3490874095 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f489400be30 con 0x5599fb831730 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f4880ff9640 1 -- 172.21.15.44:0/3490874095 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f488c095a60 con 0x5599fb831730 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f48a22a5740 10 client.15699 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f48a22a5740 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f48a22a5740 20 client.15699 populate_metadata read hostname 'smithi044' 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb8a2fe0 0x5599fb8a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f48a22a5740 4 client.15699 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599fb4ece30 con 0x5599fb8a2fe0 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f489d31f640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb8a2fe0 0x5599fb8a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f489d31f640 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb8a2fe0 0x5599fb8a5400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662805fb82a31e3b server_cookie=709a22d0ec60bf29 in_seq=0 out_seq=0 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.609+0000 7f4880ff9640 10 client.15699 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f4880ff9640 1 -- 172.21.15.44:0/3490874095 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5599fb4ece30 con 0x5599fb8a2fe0 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f4880ff9640 10 client.15699 handle_command_reply: tid=0 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 client.15699 shutdown 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 msgr2=0x5599fb83f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83f880 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f488c099740 tx=0x7f488c00d700 comp rx=0 tx=0).stop 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 shutdown_connections 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599fb8a2fe0 0x5599fb8a5400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599fb83bfd0 0x5599fb89a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599fb831730 0x5599fb83f880 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f4880ff9640 0 client.15699 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 --2- 172.21.15.44:0/3490874095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599fb830e10 0x5599fb83f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 >> 172.21.15.44:0/3490874095 conn(0x5599fb82d380 msgr2=0x5599fb833fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 shutdown_connections 2024-09-13T17:26:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 1 -- 172.21.15.44:0/3490874095 wait complete. 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.610+0000 7f48a22a5740 20 client.15699 trim_cache size 0 max 16384 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: "b3789354-bda6-425b-8724-d96841503a55": { 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "b3789354-bda6-425b-8724-d96841503a55", 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:26:50.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:26:50.619 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:26:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f48840a1320 msgr2=0x7f48840a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f48840a1320 0x7f48840a37e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4890005e30 tx=0x7f4890005dc0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 msgr2=0x7f4898197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f4898197530 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d770 tx=0x7f488c00dc30 comp rx=0 tx=0).stop 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 shutdown_connections 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f48840a1320 0x7f48840a37e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4898197a70 0x7f48981a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4898103fb0 0x7f4898197530 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 --2- 172.21.15.44:0/1088662117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48980fee60 0x7f4898196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 >> 172.21.15.44:0/1088662117 conn(0x7f4898075c60 msgr2=0x7f4898102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 shutdown_connections 2024-09-13T17:26:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.611+0000 7f489f583640 1 -- 172.21.15.44:0/1088662117 wait complete. 2024-09-13T17:26:50.622 INFO:tasks.cephfs.filesystem:scrub status for tag:b3789354-bda6-425b-8724-d96841503a55 - {'path': '/', 'tag': 'b3789354-bda6-425b-8724-d96841503a55', 'options': 'recursive,force'} 2024-09-13T17:26:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.622+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) v1 -- 0x7f76681996a0 con 0x7f76680ff970 2024-09-13T17:26:50.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.627+0000 7f76577fe640 1 -- 172.21.15.44:0/1088360350 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) v1 ==== 129+0+0 (secure 0 0 0) 0x7f765c099de0 con 0x7f76680ff970 2024-09-13T17:26:50.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f76500a1320 msgr2=0x7f76500a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f76500a1320 0x7f76500a37e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7658009530 tx=0x7f7658009260 comp rx=0 tx=0).stop 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 msgr2=0x7f7668198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f7668198890 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f765c00eb90 tx=0x7f765c00c6a0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 shutdown_connections 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f76500a1320 0x7f76500a37e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7668104c30 0x7f766819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680ff970 0x7f7668198890 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 --2- 172.21.15.44:0/1088360350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680feff0 0x7f7668198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.629+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 >> 172.21.15.44:0/1088360350 conn(0x7f7668074940 msgr2=0x7f7668067930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.630+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 shutdown_connections 2024-09-13T17:26:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.630+0000 7f766c94c640 1 -- 172.21.15.44:0/1088360350 wait complete. 2024-09-13T17:26:50.643 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-13T17:26:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 -- 172.21.15.44:0/1505114713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8073180 msgr2=0x7f06f810c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 --2- 172.21.15.44:0/1505114713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8073180 0x7f06f810c030 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009900 tx=0x7f06ec02a480 comp rx=0 tx=0).stop 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 -- 172.21.15.44:0/1505114713 shutdown_connections 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 --2- 172.21.15.44:0/1505114713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f810c570 0x7f06f810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 --2- 172.21.15.44:0/1505114713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8073180 0x7f06f810c030 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 --2- 172.21.15.44:0/1505114713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f8072860 0x7f06f8072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 -- 172.21.15.44:0/1505114713 >> 172.21.15.44:0/1505114713 conn(0x7f06f80fb9b0 msgr2=0x7f06f80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 -- 172.21.15.44:0/1505114713 shutdown_connections 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.786+0000 7f06fdc19640 1 -- 172.21.15.44:0/1505114713 wait complete. 2024-09-13T17:26:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 Processor -- start 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 -- start start 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 0x7f06f81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f8073180 0x7f06f81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f810c570 0x7f06f819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f819d2e0 con 0x7f06f810c570 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f819d450 con 0x7f06f8073180 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06fdc19640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f819d610 con 0x7f06f8072860 2024-09-13T17:26:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 0x7f06f81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f810c570 0x7f06f819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 0x7f06f81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:48652/0 (socket says 172.21.15.44:48652) 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 -- 172.21.15.44:0/2840627244 learned_addr learned my addr 172.21.15.44:0/2840627244 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f6ffd640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f8073180 0x7f06f81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 -- 172.21.15.44:0/2840627244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f8073180 msgr2=0x7f06f81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f8073180 0x7f06f81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 -- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f810c570 msgr2=0x7f06f819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f810c570 0x7f06f819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.787+0000 7f06f77fe640 1 -- 172.21.15.44:0/2840627244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec009560 con 0x7f06f8072860 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06f7fff640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f810c570 0x7f06f819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06f77fe640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 0x7f06f81983b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f06e800e9f0 tx=0x7f06e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8009700 con 0x7f06f8072860 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f819d8f0 con 0x7f06f8072860 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06e800cd80 con 0x7f06f8072860 2024-09-13T17:26:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80187e0 con 0x7f06f8072860 2024-09-13T17:26:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.788+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f81a5170 con 0x7f06f8072860 2024-09-13T17:26:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.789+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f8073f10 con 0x7f06f8072860 2024-09-13T17:26:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.790+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f06e8010430 con 0x7f06f8072860 2024-09-13T17:26:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.791+0000 7f06f4ff9640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06e00a1320 0x7f06e00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.791+0000 7f06f6ffd640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06e00a1320 0x7f06e00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.791+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f06e8014070 con 0x7f06f8072860 2024-09-13T17:26:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.791+0000 7f06f6ffd640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06e00a1320 0x7f06e00a37e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000c00 tx=0x7f06ec004040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.794+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f06e80969c0 con 0x7f06f8072860 2024-09-13T17:26:50.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.967+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f06f8199700 con 0x7f06f8072860 2024-09-13T17:26:50.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.974+0000 7f06f4ff9640 1 -- 172.21.15.44:0/2840627244 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f06e8099fe0 con 0x7f06f8072860 2024-09-13T17:26:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06e00a1320 msgr2=0x7f06e00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06e00a1320 0x7f06e00a37e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000c00 tx=0x7f06ec004040 comp rx=0 tx=0).stop 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 msgr2=0x7f06f81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 0x7f06f81983b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f06e800e9f0 tx=0x7f06e800eeb0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 shutdown_connections 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06e00a1320 0x7f06e00a37e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f810c570 0x7f06f819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f8073180 0x7f06f81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 --2- 172.21.15.44:0/2840627244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f8072860 0x7f06f81983b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 >> 172.21.15.44:0/2840627244 conn(0x7f06f80fb9b0 msgr2=0x7f06f80fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 shutdown_connections 2024-09-13T17:26:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:50.976+0000 7f06fdc19640 1 -- 172.21.15.44:0/2840627244 wait complete. 2024-09-13T17:26:50.989 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 -- 172.21.15.44:0/527916941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc0ff7d0 msgr2=0x7f0bdc103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 --2- 172.21.15.44:0/527916941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc0ff7d0 0x7f0bdc103810 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009930 tx=0x7f0bd002a420 comp rx=0 tx=0).stop 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 -- 172.21.15.44:0/527916941 shutdown_connections 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 --2- 172.21.15.44:0/527916941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bdc103f70 0x7f0bdc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 --2- 172.21.15.44:0/527916941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc0ff7d0 0x7f0bdc103810 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 --2- 172.21.15.44:0/527916941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc0fee20 0x7f0bdc0ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 -- 172.21.15.44:0/527916941 >> 172.21.15.44:0/527916941 conn(0x7f0bdc0fa8f0 msgr2=0x7f0bdc0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 -- 172.21.15.44:0/527916941 shutdown_connections 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.093+0000 7f0be27a3640 1 -- 172.21.15.44:0/527916941 wait complete. 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 Processor -- start 2024-09-13T17:26:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 -- start start 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc0fee20 0x7f0bdc196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bdc103f70 0x7f0bdc197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 0x7f0bdc1a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc1a2710 con 0x7f0bdc197a70 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc1a2880 con 0x7f0bdc0fee20 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be27a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc1a2a40 con 0x7f0bdc103f70 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be0d40640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 0x7f0bdc1a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be0d40640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 0x7f0bdc1a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54426/0 (socket says 172.21.15.44:54426) 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0be0d40640 1 -- 172.21.15.44:0/3212793265 learned_addr learned my addr 172.21.15.44:0/3212793265 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0bdb7fe640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bdc103f70 0x7f0bdc197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.094+0000 7f0bdbfff640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc0fee20 0x7f0bdc196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be0d40640 1 -- 172.21.15.44:0/3212793265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bdc103f70 msgr2=0x7f0bdc197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be0d40640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bdc103f70 0x7f0bdc197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be0d40640 1 -- 172.21.15.44:0/3212793265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc0fee20 msgr2=0x7f0bdc196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be0d40640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc0fee20 0x7f0bdc196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be0d40640 1 -- 172.21.15.44:0/3212793265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0009560 con 0x7f0bdc197a70 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be0d40640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 0x7f0bdc1a2070 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400eaa0 tx=0x7f0bd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400cdb0 con 0x7f0bdc197a70 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bd4005020 con 0x7f0bdc197a70 2024-09-13T17:26:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bdc1a2d20 con 0x7f0bdc197a70 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40116d0 con 0x7f0bdc197a70 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.095+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bdc1a3250 con 0x7f0bdc197a70 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.096+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bdc1002f0 con 0x7f0bdc197a70 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.097+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f0bd4005190 con 0x7f0bdc197a70 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.098+0000 7f0bd97fa640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0bc40a13a0 0x7f0bc40a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.098+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0bd40cd660 con 0x7f0bdc197a70 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.098+0000 7f0bdbfff640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0bc40a13a0 0x7f0bc40a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.098+0000 7f0bdbfff640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0bc40a13a0 0x7f0bc40a3860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005d00 tx=0x7f0bcc00b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:51.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.103+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0bd40965f0 con 0x7f0bdc197a70 2024-09-13T17:26:51.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.275+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f0bdc0ff200 con 0x7f0bdc197a70 2024-09-13T17:26:51.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.280+0000 7f0bd97fa640 1 -- 172.21.15.44:0/3212793265 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0bd4099c10 con 0x7f0bdc197a70 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.282+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0bc40a13a0 msgr2=0x7f0bc40a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.282+0000 7f0be27a3640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0bc40a13a0 0x7f0bc40a3860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005d00 tx=0x7f0bcc00b410 comp rx=0 tx=0).stop 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.282+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 msgr2=0x7f0bdc1a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.282+0000 7f0be27a3640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 0x7f0bdc1a2070 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400eaa0 tx=0x7f0bd400ef60 comp rx=0 tx=0).stop 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 shutdown_connections 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0bc40a13a0 0x7f0bc40a3860 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bdc103f70 0x7f0bdc197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bdc197a70 0x7f0bdc1a2070 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 --2- 172.21.15.44:0/3212793265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc0fee20 0x7f0bdc196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 >> 172.21.15.44:0/3212793265 conn(0x7f0bdc0fa8f0 msgr2=0x7f0bdc0fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 shutdown_connections 2024-09-13T17:26:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.283+0000 7f0be27a3640 1 -- 172.21.15.44:0/3212793265 wait complete. 2024-09-13T17:26:51.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3753410799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:26:51.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:26:51.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1088360350' entity='client.admin' 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='client.? ' entity='client.admin' 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:51.295 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:51 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:51.296 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-13T17:26:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 -- 172.21.15.44:0/2581296191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d340fedf0 msgr2=0x7f7d340ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 --2- 172.21.15.44:0/2581296191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d340fedf0 0x7f7d340ff1d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009930 tx=0x7f7d2402a420 comp rx=0 tx=0).stop 2024-09-13T17:26:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 -- 172.21.15.44:0/2581296191 shutdown_connections 2024-09-13T17:26:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 --2- 172.21.15.44:0/2581296191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d340ff7a0 0x7f7d341037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 --2- 172.21.15.44:0/2581296191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d34103f40 0x7f7d3410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 --2- 172.21.15.44:0/2581296191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d340fedf0 0x7f7d340ff1d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 -- 172.21.15.44:0/2581296191 >> 172.21.15.44:0/2581296191 conn(0x7f7d340fa8a0 msgr2=0x7f7d340fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 -- 172.21.15.44:0/2581296191 shutdown_connections 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.400+0000 7f7d3908e640 1 -- 172.21.15.44:0/2581296191 wait complete. 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 Processor -- start 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 -- start start 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d340fedf0 0x7f7d34196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d34103f40 0x7f7d34196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 0x7f7d3419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3419b420 con 0x7f7d3419aa30 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3419b590 con 0x7f7d340fedf0 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.401+0000 7f7d3908e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3419b700 con 0x7f7d34103f40 2024-09-13T17:26:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 0x7f7d3419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d34103f40 0x7f7d34196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 0x7f7d3419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54448/0 (socket says 172.21.15.44:54448) 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 -- 172.21.15.44:0/1749235095 learned_addr learned my addr 172.21.15.44:0/1749235095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d32d9d640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d340fedf0 0x7f7d34196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 -- 172.21.15.44:0/1749235095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d34103f40 msgr2=0x7f7d34196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d34103f40 0x7f7d34196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 -- 172.21.15.44:0/1749235095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d340fedf0 msgr2=0x7f7d34196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d340fedf0 0x7f7d34196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 -- 172.21.15.44:0/1749235095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d24009560 con 0x7f7d3419aa30 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3259c640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d34103f40 0x7f7d34196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d32d9d640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d340fedf0 0x7f7d34196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3359e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 0x7f7d3419aee0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00cce0 tx=0x7f7d2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00f450 con 0x7f7d3419aa30 2024-09-13T17:26:51.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d2c00fa40 con 0x7f7d3419aa30 2024-09-13T17:26:51.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d341a2ce0 con 0x7f7d3419aa30 2024-09-13T17:26:51.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0189a0 con 0x7f7d3419aa30 2024-09-13T17:26:51.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.402+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d341a3140 con 0x7f7d3419aa30 2024-09-13T17:26:51.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.403+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d341002c0 con 0x7f7d3419aa30 2024-09-13T17:26:51.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.405+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f7d2c00fbb0 con 0x7f7d3419aa30 2024-09-13T17:26:51.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.405+0000 7f7d1bfff640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7d1c0a1320 0x7f7d1c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.405+0000 7f7d32d9d640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7d1c0a1320 0x7f7d1c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.405+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7d2c0cd660 con 0x7f7d3419aa30 2024-09-13T17:26:51.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.406+0000 7f7d32d9d640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7d1c0a1320 0x7f7d1c0a37e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006010 tx=0x7f7d24005a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:51.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.408+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7d2c0965d0 con 0x7f7d3419aa30 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3753410799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1088360350' entity='client.admin' 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:51.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:51.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:51.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='client.? ' entity='client.admin' 2024-09-13T17:26:51.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:51.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:51.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:51 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3753410799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1088360350' entity='client.admin' 2024-09-13T17:26:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='client.? ' entity='client.admin' 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:51.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:51 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:51.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.603+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d340ff880 con 0x7f7d1c0a1320 2024-09-13T17:26:51.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.664+0000 7f7d1bfff640 1 -- 172.21.15.44:0/1749235095 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7d340ff880 con 0x7f7d1c0a1320 2024-09-13T17:26:51.665 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path / 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7d1c0a1320 msgr2=0x7f7d1c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7d1c0a1320 0x7f7d1c0a37e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006010 tx=0x7f7d24005a40 comp rx=0 tx=0).stop 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 msgr2=0x7f7d3419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 0x7f7d3419aee0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00cce0 tx=0x7f7d2c00a620 comp rx=0 tx=0).stop 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 shutdown_connections 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7d1c0a1320 0x7f7d1c0a37e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d34103f40 0x7f7d34196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d3419aa30 0x7f7d3419aee0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 --2- 172.21.15.44:0/1749235095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d340fedf0 0x7f7d34196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.666+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 >> 172.21.15.44:0/1749235095 conn(0x7f7d340fa8a0 msgr2=0x7f7d340fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.667+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 shutdown_connections 2024-09-13T17:26:51.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.667+0000 7f7d3908e640 1 -- 172.21.15.44:0/1749235095 wait complete. 2024-09-13T17:26:51.678 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 -- 172.21.15.44:0/924799908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 msgr2=0x7f2fd80681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 --2- 172.21.15.44:0/924799908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd80681d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009900 tx=0x7f2fc802a410 comp rx=0 tx=0).stop 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 -- 172.21.15.44:0/924799908 shutdown_connections 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 --2- 172.21.15.44:0/924799908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd810c660 0x7f2fd810ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 --2- 172.21.15.44:0/924799908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd80681d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 --2- 172.21.15.44:0/924799908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd8100ca0 0x7f2fd8067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 -- 172.21.15.44:0/924799908 >> 172.21.15.44:0/924799908 conn(0x7f2fd8074940 msgr2=0x7f2fd8074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 -- 172.21.15.44:0/924799908 shutdown_connections 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 -- 172.21.15.44:0/924799908 wait complete. 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 Processor -- start 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.782+0000 7f2fdf290640 1 -- start start 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8100ca0 0x7f2fd8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd810c660 0x7f2fd819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd819d270 con 0x7f2fd8067d70 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd819d3e0 con 0x7f2fd810c660 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd819d5a0 con 0x7f2fd8100ca0 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd8198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54472/0 (socket says 172.21.15.44:54472) 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 -- 172.21.15.44:0/1347348211 learned_addr learned my addr 172.21.15.44:0/1347348211 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd82d640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd810c660 0x7f2fd819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdc82b640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8100ca0 0x7f2fd8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 -- 172.21.15.44:0/1347348211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8100ca0 msgr2=0x7f2fd8198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8100ca0 0x7f2fd8198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 -- 172.21.15.44:0/1347348211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd810c660 msgr2=0x7f2fd819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd810c660 0x7f2fd819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 -- 172.21.15.44:0/1347348211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009560 con 0x7f2fd8067d70 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdd02c640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd8198340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c920 tx=0x7f2fd400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4007c70 con 0x7f2fd8067d70 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.783+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd819d880 con 0x7f2fd8067d70 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.784+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fd4007dd0 con 0x7f2fd8067d70 2024-09-13T17:26:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.784+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4018730 con 0x7f2fd8067d70 2024-09-13T17:26:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.784+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd81a5100 con 0x7f2fd8067d70 2024-09-13T17:26:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.784+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd80ff200 con 0x7f2fd8067d70 2024-09-13T17:26:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.786+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f2fd400f450 con 0x7f2fd8067d70 2024-09-13T17:26:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.786+0000 7f2fce7fc640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2fc40a1320 0x7f2fc40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.787+0000 7f2fdc82b640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2fc40a1320 0x7f2fc40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.787+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2fd40cd360 con 0x7f2fd8067d70 2024-09-13T17:26:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.787+0000 7f2fdc82b640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2fc40a1320 0x7f2fc40a37e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fc802aa00 tx=0x7f2fc8005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:51.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.790+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2fd4096240 con 0x7f2fd8067d70 2024-09-13T17:26:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:51.983+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fd8111e80 con 0x7f2fc40a1320 2024-09-13T17:26:52.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.041+0000 7f2fce7fc640 1 -- 172.21.15.44:0/1347348211 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f2fd8111e80 con 0x7f2fc40a1320 2024-09-13T17:26:52.042 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path / 2024-09-13T17:26:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2fc40a1320 msgr2=0x7f2fc40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2fc40a1320 0x7f2fc40a37e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fc802aa00 tx=0x7f2fc8005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 msgr2=0x7f2fd8198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd8198340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c920 tx=0x7f2fd400cde0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 shutdown_connections 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2fc40a1320 0x7f2fc40a37e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8100ca0 0x7f2fd8198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd810c660 0x7f2fd819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 --2- 172.21.15.44:0/1347348211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fd8067d70 0x7f2fd8198340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 >> 172.21.15.44:0/1347348211 conn(0x7f2fd8074940 msgr2=0x7f2fd810d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 shutdown_connections 2024-09-13T17:26:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.043+0000 7f2fdf290640 1 -- 172.21.15.44:0/1347348211 wait complete. 2024-09-13T17:26:52.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-13T17:26:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 -- 172.21.15.44:0/4107624635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6100fee20 msgr2=0x7fe6100ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 --2- 172.21.15.44:0/4107624635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6100fee20 0x7fe6100ff200 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009930 tx=0x7fe60c02a420 comp rx=0 tx=0).stop 2024-09-13T17:26:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 -- 172.21.15.44:0/4107624635 shutdown_connections 2024-09-13T17:26:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 --2- 172.21.15.44:0/4107624635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6100ff7d0 0x7fe610103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 --2- 172.21.15.44:0/4107624635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe610103f70 0x7fe61010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 --2- 172.21.15.44:0/4107624635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6100fee20 0x7fe6100ff200 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 -- 172.21.15.44:0/4107624635 >> 172.21.15.44:0/4107624635 conn(0x7fe6100fa8f0 msgr2=0x7fe6100fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 -- 172.21.15.44:0/4107624635 shutdown_connections 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.157+0000 7fe617a87640 1 -- 172.21.15.44:0/4107624635 wait complete. 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 Processor -- start 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 -- start start 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6100fee20 0x7fe6101961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 0x7fe610196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61019ab10 0x7fe61019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61019b500 con 0x7fe610103f70 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61019b670 con 0x7fe61019ab10 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe617a87640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61019b7e0 con 0x7fe6100fee20 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 0x7fe610196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 0x7fe610196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54484/0 (socket says 172.21.15.44:54484) 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 -- 172.21.15.44:0/1638550362 learned_addr learned my addr 172.21.15.44:0/1638550362 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe616024640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61019ab10 0x7fe61019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615823640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6100fee20 0x7fe6101961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 -- 172.21.15.44:0/1638550362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6100fee20 msgr2=0x7fe6101961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6100fee20 0x7fe6101961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 -- 172.21.15.44:0/1638550362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61019ab10 msgr2=0x7fe61019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61019ab10 0x7fe61019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 -- 172.21.15.44:0/1638550362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c009560 con 0x7fe610103f70 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe616024640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61019ab10 0x7fe61019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.158+0000 7fe615022640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 0x7fe610196720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe60000d740 tx=0x7fe60000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.159+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600009c30 con 0x7fe610103f70 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.159+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe60000de50 con 0x7fe610103f70 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.159+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6101a2dc0 con 0x7fe610103f70 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.159+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600004d30 con 0x7fe610103f70 2024-09-13T17:26:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.159+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6101a3210 con 0x7fe610103f70 2024-09-13T17:26:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.160+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6101002f0 con 0x7fe610103f70 2024-09-13T17:26:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.161+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fe600004e90 con 0x7fe610103f70 2024-09-13T17:26:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.161+0000 7fe606ffd640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5fc0a13a0 0x7fe5fc0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.162+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe6000cd040 con 0x7fe610103f70 2024-09-13T17:26:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.162+0000 7fe615823640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5fc0a13a0 0x7fe5fc0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.162+0000 7fe615823640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5fc0a13a0 0x7fe5fc0a3860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe60c006010 tx=0x7fe60c005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:52.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.164+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe600095fa0 con 0x7fe610103f70 2024-09-13T17:26:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:52 smithi044 ceph-mon[31175]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 KiB/s wr, 23 op/s 2024-09-13T17:26:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:52 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3212793265' entity='client.admin' 2024-09-13T17:26:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:52 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.357+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6100ff7d0 con 0x7fe5fc0a13a0 2024-09-13T17:26:52.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.363+0000 7fe606ffd640 1 -- 172.21.15.44:0/1638550362 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fe6100ff7d0 con 0x7fe5fc0a13a0 2024-09-13T17:26:52.364 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-13T00:00:00", "created": "2024-09-13T17:26:51", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-13T17:26:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5fc0a13a0 msgr2=0x7fe5fc0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5fc0a13a0 0x7fe5fc0a3860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe60c006010 tx=0x7fe60c005a20 comp rx=0 tx=0).stop 2024-09-13T17:26:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 msgr2=0x7fe610196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 0x7fe610196720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe60000d740 tx=0x7fe60000dc00 comp rx=0 tx=0).stop 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 shutdown_connections 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5fc0a13a0 0x7fe5fc0a3860 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61019ab10 0x7fe61019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe610103f70 0x7fe610196720 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 --2- 172.21.15.44:0/1638550362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6100fee20 0x7fe6101961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 >> 172.21.15.44:0/1638550362 conn(0x7fe6100fa8f0 msgr2=0x7fe6100689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 shutdown_connections 2024-09-13T17:26:52.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.365+0000 7fe617a87640 1 -- 172.21.15.44:0/1638550362 wait complete. 2024-09-13T17:26:52.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-13T17:26:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:52 smithi070 ceph-mon[34502]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 KiB/s wr, 23 op/s 2024-09-13T17:26:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:52 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3212793265' entity='client.admin' 2024-09-13T17:26:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:52 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:52.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 -- 172.21.15.44:0/771529587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c410c590 msgr2=0x7f08c410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 --2- 172.21.15.44:0/771529587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c410c590 0x7f08c410e980 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009ca0 tx=0x7f08bc02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 -- 172.21.15.44:0/771529587 shutdown_connections 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 --2- 172.21.15.44:0/771529587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c410c590 0x7f08c410e980 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 --2- 172.21.15.44:0/771529587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c40ff390 0x7f08c410c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 --2- 172.21.15.44:0/771529587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c40fea70 0x7f08c40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.480+0000 7f08cb861640 1 -- 172.21.15.44:0/771529587 >> 172.21.15.44:0/771529587 conn(0x7f08c40fa890 msgr2=0x7f08c40fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 -- 172.21.15.44:0/771529587 shutdown_connections 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 -- 172.21.15.44:0/771529587 wait complete. 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 Processor -- start 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 -- start start 2024-09-13T17:26:52.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 0x7f08c4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c40ff390 0x7f08c41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c410c590 0x7f08c419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c419d2c0 con 0x7f08c40fea70 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c419d430 con 0x7f08c40ff390 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08cb861640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c419d5f0 con 0x7f08c410c590 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08c95fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 0x7f08c4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08c95fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 0x7f08c4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54498/0 (socket says 172.21.15.44:54498) 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.481+0000 7f08c95fd640 1 -- 172.21.15.44:0/3938605469 learned_addr learned my addr 172.21.15.44:0/3938605469 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c9dfe640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c410c590 0x7f08c419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c8dfc640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c40ff390 0x7f08c41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c95fd640 1 -- 172.21.15.44:0/3938605469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c410c590 msgr2=0x7f08c419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c95fd640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c410c590 0x7f08c419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c95fd640 1 -- 172.21.15.44:0/3938605469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c40ff390 msgr2=0x7f08c41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c95fd640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c40ff390 0x7f08c41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c95fd640 1 -- 172.21.15.44:0/3938605469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc009900 con 0x7f08c40fea70 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c9dfe640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c410c590 0x7f08c419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08c95fd640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 0x7f08c4198390 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08c000d770 tx=0x7f08c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0009c30 con 0x7f08c40fea70 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c000de80 con 0x7f08c40fea70 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c419d8d0 con 0x7f08c40fea70 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004d40 con 0x7f08c40fea70 2024-09-13T17:26:52.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.482+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c41a5150 con 0x7f08c40fea70 2024-09-13T17:26:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.483+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c4068e10 con 0x7f08c40fea70 2024-09-13T17:26:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.484+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f08c0004ea0 con 0x7f08c40fea70 2024-09-13T17:26:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.485+0000 7f08ba7fc640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f08b00a13a0 0x7f08b00a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:26:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.485+0000 7f08c8dfc640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f08b00a13a0 0x7f08b00a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:26:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.485+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f08c00ccd50 con 0x7f08c40fea70 2024-09-13T17:26:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.486+0000 7f08c8dfc640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f08b00a13a0 0x7f08b00a3860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08c4199930 tx=0x7f08b4005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:26:52.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.489+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f08c00d0030 con 0x7f08c40fea70 2024-09-13T17:26:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:52 smithi100 ceph-mon[35055]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 KiB/s wr, 23 op/s 2024-09-13T17:26:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:52 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3212793265' entity='client.admin' 2024-09-13T17:26:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:26:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:26:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:26:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:52 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:26:52.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.684+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f08c41996e0 con 0x7f08b00a13a0 2024-09-13T17:26:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.691+0000 7f08ba7fc640 1 -- 172.21.15.44:0/3938605469 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f08c41996e0 con 0x7f08b00a13a0 2024-09-13T17:26:52.692 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1m 6m3h 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f08b00a13a0 msgr2=0x7f08b00a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f08b00a13a0 0x7f08b00a3860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08c4199930 tx=0x7f08b4005f90 comp rx=0 tx=0).stop 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 msgr2=0x7f08c4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 0x7f08c4198390 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08c000d770 tx=0x7f08c000dc30 comp rx=0 tx=0).stop 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 shutdown_connections 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f08b00a13a0 0x7f08b00a3860 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c410c590 0x7f08c419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c40ff390 0x7f08c41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 --2- 172.21.15.44:0/3938605469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08c40fea70 0x7f08c4198390 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:26:52.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 >> 172.21.15.44:0/3938605469 conn(0x7f08c40fa890 msgr2=0x7f08c40fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:26:52.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 shutdown_connections 2024-09-13T17:26:52.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:26:52.693+0000 7f08cb861640 1 -- 172.21.15.44:0/3938605469 wait complete. 2024-09-13T17:26:52.704 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-13T17:26:52.731 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-13T17:26:52.744 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-13T17:26:52.782 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:26:52.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-13T17:26:52.785 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T17:26:52.819 INFO:tasks.workunit:Pulling workunits from ref 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:26:52.819 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T17:26:52.819 INFO:tasks.workunit:timeout=3h 2024-09-13T17:26:52.819 INFO:tasks.workunit:cleanup=True 2024-09-13T17:26:52.820 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:52.836 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:52.836 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T17:26:52.837 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-13T17:26:52.837 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:26:52.837 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-13T17:26:52.837 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 17:24:47.844219603 +0000 2024-09-13T17:26:52.837 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 17:24:47.844219603 +0000 2024-09-13T17:26:52.838 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 17:26:41.976478799 +0000 2024-09-13T17:26:52.838 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-09-13T17:26:52.838 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-13T17:26:52.838 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-13T17:26:52.908 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:26:52.965 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T17:26:53.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:53 smithi044 ceph-mon[31175]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:53.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:53 smithi044 ceph-mon[31175]: from='client.15723 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:53 smithi070 ceph-mon[34502]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:53.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:53 smithi070 ceph-mon[34502]: from='client.15723 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:53.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:53 smithi100 ceph-mon[35055]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:53.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:53 smithi100 ceph-mon[35055]: from='client.15723 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:54.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:54 smithi044 ceph-mon[31175]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:54.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:54 smithi044 ceph-mon[31175]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 911 B/s wr, 18 op/s 2024-09-13T17:26:54.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:54 smithi044 ceph-mon[31175]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:54.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:54 smithi100 ceph-mon[35055]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:54.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:54 smithi100 ceph-mon[35055]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 911 B/s wr, 18 op/s 2024-09-13T17:26:54.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:54 smithi100 ceph-mon[35055]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:54 smithi070 ceph-mon[34502]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:54 smithi070 ceph-mon[34502]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 911 B/s wr, 18 op/s 2024-09-13T17:26:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:54 smithi070 ceph-mon[34502]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:26:56.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:56 smithi044 ceph-mon[31175]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-09-13T17:26:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:56 smithi070 ceph-mon[34502]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-09-13T17:26:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:56 smithi100 ceph-mon[35055]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-09-13T17:26:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:26:58 smithi044 ceph-mon[31175]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-13T17:26:58.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:26:58 smithi070 ceph-mon[34502]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-13T17:26:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:26:58 smithi100 ceph-mon[35055]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-13T17:27:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:00 smithi044 ceph-mon[31175]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-13T17:27:00.357 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:27:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:27:00.053+0000 7fb65ed48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:27:00.357 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:27:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:27:00.053+0000 7fb65ed48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:27:00.357 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:27:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:27:00.053+0000 7fb65ed48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:27:00.357 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:27:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:27:00.053+0000 7fb65ed48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:27:00.357 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:27:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:27:00.053+0000 7fb65ed48640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:27:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:00 smithi070 ceph-mon[34502]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-13T17:27:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:00 smithi100 ceph-mon[35055]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-13T17:27:02.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:02 smithi044 ceph-mon[31175]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-13T17:27:02.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:02 smithi044 ceph-mon[31175]: mgrmap e25: x(active, since 18s), standbys: y, z 2024-09-13T17:27:02.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:02 smithi070 ceph-mon[34502]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-13T17:27:02.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:02 smithi070 ceph-mon[34502]: mgrmap e25: x(active, since 18s), standbys: y, z 2024-09-13T17:27:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:02 smithi100 ceph-mon[35055]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-13T17:27:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:02 smithi100 ceph-mon[35055]: mgrmap e25: x(active, since 18s), standbys: y, z 2024-09-13T17:27:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:04 smithi044 ceph-mon[31175]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-09-13T17:27:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:04 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:27:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:04 smithi070 ceph-mon[34502]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-09-13T17:27:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:04 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:27:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:04 smithi100 ceph-mon[35055]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-09-13T17:27:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:04 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:27:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 -- 172.21.15.44:0/2843860378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700ff5d0 msgr2=0x7f8170103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 --2- 172.21.15.44:0/2843860378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700ff5d0 0x7f8170103a90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8160009960 tx=0x7f816002a4f0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 -- 172.21.15.44:0/2843860378 shutdown_connections 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 --2- 172.21.15.44:0/2843860378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81701040e0 0x7f8170104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 --2- 172.21.15.44:0/2843860378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700ff5d0 0x7f8170103a90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 --2- 172.21.15.44:0/2843860378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81700fecb0 0x7f81700ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 -- 172.21.15.44:0/2843860378 >> 172.21.15.44:0/2843860378 conn(0x7f81700fa8d0 msgr2=0x7f81700fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 -- 172.21.15.44:0/2843860378 shutdown_connections 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.924+0000 7f8177cc9640 1 -- 172.21.15.44:0/2843860378 wait complete. 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 Processor -- start 2024-09-13T17:27:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 -- start start 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 0x7f817019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81700ff5d0 0x7f817019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701040e0 0x7f81701a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81701a0600 con 0x7f81700fecb0 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81701a0770 con 0x7f81701040e0 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8177cc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81701a0930 con 0x7f81700ff5d0 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8175a65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 0x7f817019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8175a65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 0x7f817019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42072/0 (socket says 172.21.15.44:42072) 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8175a65640 1 -- 172.21.15.44:0/3100143134 learned_addr learned my addr 172.21.15.44:0/3100143134 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8175264640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81700ff5d0 0x7f817019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.925+0000 7f8176266640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701040e0 0x7f81701a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8175a65640 1 -- 172.21.15.44:0/3100143134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81700ff5d0 msgr2=0x7f817019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8175a65640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81700ff5d0 0x7f817019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8175a65640 1 -- 172.21.15.44:0/3100143134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701040e0 msgr2=0x7f81701a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8175a65640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701040e0 0x7f81701a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8175a65640 1 -- 172.21.15.44:0/3100143134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160009560 con 0x7f81700fecb0 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8176266640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701040e0 0x7f81701a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8175a65640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 0x7f817019c780 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9f0 tx=0x7f816c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8166ffd640 1 -- 172.21.15.44:0/3100143134 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c009700 con 0x7f81700fecb0 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81701a0c10 con 0x7f81700fecb0 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81701a9570 con 0x7f81700fecb0 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8166ffd640 1 -- 172.21.15.44:0/3100143134 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f816c00cd80 con 0x7f81700fecb0 2024-09-13T17:27:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.926+0000 7f8166ffd640 1 -- 172.21.15.44:0/3100143134 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0187e0 con 0x7f81700fecb0 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.928+0000 7f8166ffd640 1 -- 172.21.15.44:0/3100143134 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f816c010430 con 0x7f81700fecb0 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 Processor -- start 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 -- start start 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8df710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8ec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e84ac30 con 0x55be6e8ea4f0 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e8483c0 con 0x55be6e8dfc50 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f817a9eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e843cb0 con 0x55be6e8df330 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f8166ffd640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f815c0a13b0 0x7f815c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f8175264640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8ec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f8175264640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8ec8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42076/0 (socket says 172.21.15.44:42076) 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.929+0000 7f8175264640 1 -- 172.21.15.44:0/2795093658 learned_addr learned my addr 172.21.15.44:0/2795093658 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175a65640 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8176266640 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8176266640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f815c0a13b0 0x7f815c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8166ffd640 1 -- 172.21.15.44:0/3100143134 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f816c014070 con 0x7f81700fecb0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175264640 1 -- 172.21.15.44:0/2795093658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 msgr2=0x55be6e8df710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175264640 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8df710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175264640 1 -- 172.21.15.44:0/2795093658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 msgr2=0x55be6e8e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175264640 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175264640 1 -- 172.21.15.44:0/2795093658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6e7ea910 con 0x55be6e8ea4f0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8175264640 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8ec8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f816002aa10 tx=0x7f8160031530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eea66fca2b3cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f8176266640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f815c0a13b0 0x7f815c0a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8168013b30 tx=0x7f8168005c60 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f815bfff640 1 -- 172.21.15.44:0/2795093658 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160005270 con 0x55be6e8ea4f0 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f815bfff640 1 -- 172.21.15.44:0/2795093658 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8160031dc0 con 0x55be6e8ea4f0 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.930+0000 7f815bfff640 1 -- 172.21.15.44:0/2795093658 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160003c10 con 0x55be6e8ea4f0 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 -- 172.21.15.44:0/2795093658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 msgr2=0x55be6e8ec8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8ec8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f816002aa10 tx=0x7f8160031530 comp rx=0 tx=0).stop 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 -- 172.21.15.44:0/2795093658 shutdown_connections 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8ec8e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 --2- 172.21.15.44:0/2795093658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 -- 172.21.15.44:0/2795093658 >> 172.21.15.44:0/2795093658 conn(0x55be6e8db8c0 msgr2=0x55be6e8dbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 -- 172.21.15.44:0/2795093658 shutdown_connections 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.931+0000 7f817a9eb740 1 -- 172.21.15.44:0/2795093658 wait complete. 2024-09-13T17:27:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 -- start start 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 Processor -- start 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8e9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e801e80 con 0x55be6e8ea4f0 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e801b20 con 0x55be6e8dfc50 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e7c7260 con 0x55be6e8df330 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f817a9eb740 10 client.0 _ll_register_callbacks cb 0x55be6e7fcdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f8176266640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f8176266640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42092/0 (socket says 172.21.15.44:42092) 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.932+0000 7f8176266640 1 -- 172.21.15.44:0/1089146521 learned_addr learned my addr 172.21.15.44:0/1089146521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8175264640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8175a65640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8e9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8176266640 1 -- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 msgr2=0x55be6e8e9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8176266640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8e9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8176266640 1 -- 172.21.15.44:0/1089146521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 msgr2=0x55be6e8e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8176266640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8176266640 1 -- 172.21.15.44:0/1089146521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170113a30 con 0x55be6e8ea4f0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8175a65640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8e9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f8176266640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e76f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8168010be0 tx=0x7f8168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39ddc7773764a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f81597fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f81597fa640 1 -- 172.21.15.44:0/1089146521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168012e20 con 0x55be6e8ea4f0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f81597fa640 1 -- 172.21.15.44:0/1089146521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8168028460 con 0x55be6e8ea4f0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be6e86c680 con 0x55be6e8ea4f0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f81597fa640 1 -- 172.21.15.44:0/1089146521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8168028660 con 0x55be6e8ea4f0 2024-09-13T17:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f817a9eb740 10 client.15753 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.933+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be6e842330 con 0x55be6e8ea4f0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f81597fa640 1 -- 172.21.15.44:0/1089146521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be6e86c680 con 0x55be6e8ea4f0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f81597fa640 1 -- 172.21.15.44:0/1089146521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f816800e7b0 con 0x55be6e8ea4f0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f817a9eb740 10 client.15753 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f817a9eb740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f817a9eb740 20 client.15753 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e8f9ec0 0x55be6e8fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f817a9eb740 4 client.15753 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be6e8375d0 con 0x55be6e8f9ec0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f8175a65640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e8f9ec0 0x55be6e8fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f8175a65640 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e8f9ec0 0x55be6e8fc2e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67339d2fbe5c7055 server_cookie=9431179af547875b in_seq=0 out_seq=0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.934+0000 7f81597fa640 10 client.15753 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f81597fa640 1 -- 172.21.15.44:0/1089146521 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55be6e8375d0 con 0x55be6e8f9ec0 2024-09-13T17:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f81597fa640 10 client.15753 handle_command_reply: tid=0 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 client.15753 shutdown 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 msgr2=0x55be6e8e76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e76f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8168010be0 tx=0x7f8168004a40 comp rx=0 tx=0).stop 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 shutdown_connections 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e8f9ec0 0x55be6e8fc2e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e76f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f81597fa640 0 client.15753 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8dfc50 0x55be6e8e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1089146521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8e9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 >> 172.21.15.44:0/1089146521 conn(0x55be6e8db8c0 msgr2=0x55be6e8dab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.935+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 shutdown_connections 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.936+0000 7f817a9eb740 1 -- 172.21.15.44:0/1089146521 wait complete. 2024-09-13T17:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.936+0000 7f817a9eb740 20 client.15753 trim_cache size 0 max 16384 2024-09-13T17:27:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.948+0000 7f817a9eb740 1 Processor -- start 2024-09-13T17:27:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.948+0000 7f817a9eb740 1 -- start start 2024-09-13T17:27:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.948+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.948+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f817a9eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e837600 con 0x55be6e8dfc50 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f817a9eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e86c6b0 con 0x55be6e8ea4f0 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f817a9eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e8dc550 con 0x55be6e8df330 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8e99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42094/0 (socket says 172.21.15.44:42094) 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 -- 172.21.15.44:0/1349837935 learned_addr learned my addr 172.21.15.44:0/1349837935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8176266640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175a65640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 -- 172.21.15.44:0/1349837935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 msgr2=0x55be6e8d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 -- 172.21.15.44:0/1349837935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 msgr2=0x55be6e8e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 -- 172.21.15.44:0/1349837935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6e7e4300 con 0x55be6e8dfc50 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175a65640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8176266640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8175264640 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8e99c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55be6e8dc9b0 tx=0x7f8160003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f905ad91e6d9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8164ff9640 1 -- 172.21.15.44:0/1349837935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160031b30 con 0x55be6e8dfc50 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8164ff9640 1 -- 172.21.15.44:0/1349837935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81600035f0 con 0x55be6e8dfc50 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.949+0000 7f8164ff9640 1 -- 172.21.15.44:0/1349837935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816003d450 con 0x55be6e8dfc50 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 -- 172.21.15.44:0/1349837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 msgr2=0x55be6e8e99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8e99c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55be6e8dc9b0 tx=0x7f8160003770 comp rx=0 tx=0).stop 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 -- 172.21.15.44:0/1349837935 shutdown_connections 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8e99c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1349837935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 -- 172.21.15.44:0/1349837935 >> 172.21.15.44:0/1349837935 conn(0x55be6e8db8c0 msgr2=0x55be6e8d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 -- 172.21.15.44:0/1349837935 shutdown_connections 2024-09-13T17:27:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 -- 172.21.15.44:0/1349837935 wait complete. 2024-09-13T17:27:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 -- start start 2024-09-13T17:27:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.950+0000 7f817a9eb740 1 Processor -- start 2024-09-13T17:27:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e8f0500 con 0x55be6e8dfc50 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e8f0670 con 0x55be6e8ea4f0 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6e9503d0 con 0x55be6e8df330 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 10 client.0 _ll_register_callbacks cb 0x7f8160009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42098/0 (socket says 172.21.15.44:42098) 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 -- 172.21.15.44:0/1835511362 learned_addr learned my addr 172.21.15.44:0/1835511362 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8176266640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175a65640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 -- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 msgr2=0x55be6e8f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 -- 172.21.15.44:0/1835511362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 msgr2=0x55be6e8efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 -- 172.21.15.44:0/1835511362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6e7ea910 con 0x55be6e8dfc50 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f8175264640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8ef8c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f816002aa10 tx=0x7f8160031be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a770cacf20e3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f81597fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f81597fa640 1 -- 172.21.15.44:0/1835511362 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81600035f0 con 0x55be6e8dfc50 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f81597fa640 1 -- 172.21.15.44:0/1835511362 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8160005280 con 0x55be6e8dfc50 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.951+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be6e537300 con 0x55be6e8dfc50 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f81597fa640 1 -- 172.21.15.44:0/1835511362 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8160005440 con 0x55be6e8dfc50 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 10 client.15759 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be6e8098e0 con 0x55be6e8dfc50 2024-09-13T17:27:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f81597fa640 1 -- 172.21.15.44:0/1835511362 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be6e537300 con 0x55be6e8dfc50 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f81597fa640 1 -- 172.21.15.44:0/1835511362 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f816004e480 con 0x55be6e8dfc50 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 20 client.15759 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e9515a0 0x55be6e9539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 4 client.15759 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be6e548600 con 0x55be6e9515a0 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.952+0000 7f8175a65640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e9515a0 0x55be6e9539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.953+0000 7f8175a65640 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e9515a0 0x55be6e9539c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d516a4682eb06ac6 server_cookie=9bc27b5a8b5962f9 in_seq=0 out_seq=0 2024-09-13T17:27:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.953+0000 7f81597fa640 10 client.15759 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f81597fa640 1 -- 172.21.15.44:0/1835511362 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be6e548600 con 0x55be6e9515a0 2024-09-13T17:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f81597fa640 10 client.15759 handle_command_reply: tid=0 2024-09-13T17:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 client.15759 shutdown 2024-09-13T17:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 msgr2=0x55be6e8ef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8ef8c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f816002aa10 tx=0x7f8160031be0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 shutdown_connections 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55be6e9515a0 0x55be6e9539c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be6e8ea4f0 0x55be6e8efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f81597fa640 0 client.15759 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be6e8dfc50 0x55be6e8ef8c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 --2- 172.21.15.44:0/1835511362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be6e8df330 0x55be6e8f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 >> 172.21.15.44:0/1835511362 conn(0x55be6e8db8c0 msgr2=0x55be6e8e0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 shutdown_connections 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 1 -- 172.21.15.44:0/1835511362 wait complete. 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.964+0000 7f817a9eb740 20 client.15759 trim_cache size 0 max 16384 2024-09-13T17:27:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:27:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f815c0a13b0 msgr2=0x7f815c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f815c0a13b0 0x7f815c0a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8168013b30 tx=0x7f8168005c60 comp rx=0 tx=0).stop 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 msgr2=0x7f817019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 0x7f817019c780 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9f0 tx=0x7f816c00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 shutdown_connections 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f815c0a13b0 0x7f815c0a3870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81700ff5d0 0x7f817019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701040e0 0x7f81701a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 --2- 172.21.15.44:0/3100143134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81700fecb0 0x7f817019c780 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 >> 172.21.15.44:0/3100143134 conn(0x7f81700fa8d0 msgr2=0x7f8170101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 shutdown_connections 2024-09-13T17:27:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:05.965+0000 7f8177cc9640 1 -- 172.21.15.44:0/3100143134 wait complete. 2024-09-13T17:27:06.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:06 smithi044 ceph-mon[31175]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-09-13T17:27:06.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:06 smithi070 ceph-mon[34502]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-09-13T17:27:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:06 smithi100 ceph-mon[35055]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-09-13T17:27:08.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:08 smithi044 ceph-mon[31175]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-13T17:27:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:08 smithi070 ceph-mon[34502]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-13T17:27:08.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:08 smithi100 ceph-mon[35055]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-13T17:27:10.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:10 smithi044 ceph-mon[31175]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-13T17:27:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:10 smithi070 ceph-mon[34502]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-13T17:27:10.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:10 smithi100 ceph-mon[35055]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-13T17:27:12.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:12 smithi044 ceph-mon[31175]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-13T17:27:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:12 smithi070 ceph-mon[34502]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-13T17:27:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:12 smithi100 ceph-mon[35055]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-13T17:27:14.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:14 smithi044 ceph-mon[31175]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-13T17:27:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:14 smithi070 ceph-mon[34502]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-13T17:27:14.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:14 smithi100 ceph-mon[35055]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-13T17:27:16.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:16 smithi044 ceph-mon[31175]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-13T17:27:16.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:16 smithi070 ceph-mon[34502]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-13T17:27:16.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:16 smithi100 ceph-mon[35055]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-13T17:27:18.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:18 smithi070 ceph-mon[34502]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-13T17:27:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:18 smithi100 ceph-mon[35055]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-13T17:27:18.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:18 smithi044 ceph-mon[31175]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 -- 172.21.15.44:0/1817187299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a481008f0 msgr2=0x7f0a48104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 --2- 172.21.15.44:0/1817187299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a481008f0 0x7f0a48104930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009930 tx=0x7f0a3002a420 comp rx=0 tx=0).stop 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 -- 172.21.15.44:0/1817187299 shutdown_connections 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 --2- 172.21.15.44:0/1817187299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a48105090 0x7f0a4810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 --2- 172.21.15.44:0/1817187299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a481008f0 0x7f0a48104930 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 --2- 172.21.15.44:0/1817187299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a480fff40 0x7f0a48100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 -- 172.21.15.44:0/1817187299 >> 172.21.15.44:0/1817187299 conn(0x7f0a480fb9d0 msgr2=0x7f0a480fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 -- 172.21.15.44:0/1817187299 shutdown_connections 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.059+0000 7f0a4d448640 1 -- 172.21.15.44:0/1817187299 wait complete. 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 Processor -- start 2024-09-13T17:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 -- start start 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 0x7f0a480727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a48105090 0x7f0a48072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a48073230 0x7f0a481a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481a49d0 con 0x7f0a480fff40 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481a4b40 con 0x7f0a48073230 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a4d448640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481a4d00 con 0x7f0a48105090 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a46ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 0x7f0a480727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a46ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 0x7f0a480727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33470/0 (socket says 172.21.15.44:33470) 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a46ffd640 1 -- 172.21.15.44:0/34328068 learned_addr learned my addr 172.21.15.44:0/34328068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a477fe640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a48073230 0x7f0a481a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.060+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a48105090 0x7f0a48072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a46ffd640 1 -- 172.21.15.44:0/34328068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a48105090 msgr2=0x7f0a48072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a48105090 0x7f0a48072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a46ffd640 1 -- 172.21.15.44:0/34328068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a48073230 msgr2=0x7f0a481a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a48073230 0x7f0a481a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a46ffd640 1 -- 172.21.15.44:0/34328068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30009560 con 0x7f0a480fff40 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 0x7f0a480727b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800d770 tx=0x7f0a3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a44ff9640 1 -- 172.21.15.44:0/34328068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38009c30 con 0x7f0a480fff40 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a44ff9640 1 -- 172.21.15.44:0/34328068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a3800de80 con 0x7f0a480fff40 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a481a4fe0 con 0x7f0a480fff40 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a44ff9640 1 -- 172.21.15.44:0/34328068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38004d40 con 0x7f0a480fff40 2024-09-13T17:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.061+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a481a53b0 con 0x7f0a480fff40 2024-09-13T17:27:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.063+0000 7f0a50249740 1 Processor -- start 2024-09-13T17:27:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 -- start start 2024-09-13T17:27:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561406025c30 con 0x5614060ba330 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614060233c0 con 0x5614060c54f0 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a50249740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140601ecb0 con 0x5614060bac50 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a44ff9640 1 -- 172.21.15.44:0/34328068 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f0a38004ea0 con 0x7f0a480fff40 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060ba710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33480/0 (socket says 172.21.15.44:33480) 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 -- 172.21.15.44:0/1023808606 learned_addr learned my addr 172.21.15.44:0/1023808606 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 -- 172.21.15.44:0/1023808606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 msgr2=0x5614060c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 -- 172.21.15.44:0/1023808606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 msgr2=0x5614060c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.064+0000 7f0a477fe640 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a477fe640 1 -- 172.21.15.44:0/1023808606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561405fc5910 con 0x5614060ba330 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a477fe640 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060ba710 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009f70 tx=0x7f0a40010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d1cf6663c97630 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a44ff9640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0a340a13e0 0x7f0a340a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a3d7fa640 1 -- 172.21.15.44:0/1023808606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004d20 con 0x5614060ba330 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a3d7fa640 1 -- 172.21.15.44:0/1023808606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a40004e80 con 0x5614060ba330 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a3d7fa640 1 -- 172.21.15.44:0/1023808606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012850 con 0x5614060ba330 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0a340a13e0 0x7f0a340a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.065+0000 7f0a44ff9640 1 -- 172.21.15.44:0/34328068 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0a380ccbe0 con 0x7f0a480fff40 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 -- 172.21.15.44:0/1023808606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 msgr2=0x5614060ba710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a46ffd640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0a340a13e0 0x7f0a340a38a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a380c9e30 tx=0x7f0a380b9040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060ba710 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009f70 tx=0x7f0a40010800 comp rx=0 tx=0).stop 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 -- 172.21.15.44:0/1023808606 shutdown_connections 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 --2- 172.21.15.44:0/1023808606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060ba710 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 -- 172.21.15.44:0/1023808606 >> 172.21.15.44:0/1023808606 conn(0x5614060b68c0 msgr2=0x5614060b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 -- 172.21.15.44:0/1023808606 shutdown_connections 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 -- 172.21.15.44:0/1023808606 wait complete. 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 -- start start 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.066+0000 7f0a50249740 1 Processor -- start 2024-09-13T17:27:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561405fdce80 con 0x5614060bac50 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561405fdcb20 con 0x5614060c54f0 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561405fa2260 con 0x5614060ba330 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 10 client.0 _ll_register_callbacks cb 0x561405fd7df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060c2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33490/0 (socket says 172.21.15.44:33490) 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2295758592 learned_addr learned my addr 172.21.15.44:0/2295758592 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 msgr2=0x5614060c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2295758592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 msgr2=0x5614060c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2295758592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a40009bd0 con 0x5614060bac50 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060c2170 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a48069580 tx=0x7f0a30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1908d7e6c30a3ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2295758592 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30031b60 con 0x5614060bac50 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2295758592 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0a30031cc0 con 0x5614060bac50 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.067+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614060125d0 con 0x5614060bac50 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2295758592 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a30031e80 con 0x5614060bac50 2024-09-13T17:27:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 10 client.15771 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56140601d330 con 0x5614060bac50 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2295758592 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5614060125d0 con 0x5614060bac50 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2295758592 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a3003d730 con 0x5614060bac50 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 10 client.15771 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 20 client.15771 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5614060d4dd0 0x5614060d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 4 client.15771 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561405d12300 con 0x5614060d4dd0 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.068+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5614060d4dd0 0x5614060d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.069+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5614060d4dd0 0x5614060d71f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996fba8086be1393 server_cookie=18a119968ea9c3eb in_seq=0 out_seq=0 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.069+0000 7f0a2e7fc640 10 client.15771 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.069+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2295758592 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561405d12300 con 0x5614060d4dd0 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.069+0000 7f0a2e7fc640 10 client.15771 handle_command_reply: tid=0 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.069+0000 7f0a50249740 1 client.15771 shutdown 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.069+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 msgr2=0x5614060c2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060c2170 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a48069580 tx=0x7f0a30004270 comp rx=0 tx=0).stop 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 shutdown_connections 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5614060d4dd0 0x5614060d71f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060c2170 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 --2- 172.21.15.44:0/2295758592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a2e7fc640 0 client.15771 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 >> 172.21.15.44:0/2295758592 conn(0x5614060b68c0 msgr2=0x5614060b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 shutdown_connections 2024-09-13T17:27:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 1 -- 172.21.15.44:0/2295758592 wait complete. 2024-09-13T17:27:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.070+0000 7f0a50249740 20 client.15771 trim_cache size 0 max 16384 2024-09-13T17:27:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.083+0000 7f0a50249740 1 Processor -- start 2024-09-13T17:27:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.083+0000 7f0a50249740 1 -- start start 2024-09-13T17:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.083+0000 7f0a50249740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a50249740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a50249740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a50249740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614060c06a0 con 0x5614060bac50 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a50249740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614060d7520 con 0x5614060c54f0 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a50249740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614060dc370 con 0x5614060ba330 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060bd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33502/0 (socket says 172.21.15.44:33502) 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2172612051 learned_addr learned my addr 172.21.15.44:0/2172612051 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2172612051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 msgr2=0x5614060b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2172612051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 msgr2=0x5614060bd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060bd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 -- 172.21.15.44:0/2172612051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561405fbf300 con 0x5614060bac50 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060bd410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a3002aa10 tx=0x7f0a30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78baf8379311eaa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3dffb640 1 -- 172.21.15.44:0/2172612051 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3003d640 con 0x5614060bac50 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3dffb640 1 -- 172.21.15.44:0/2172612051 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a30005850 con 0x5614060bac50 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.084+0000 7f0a3dffb640 1 -- 172.21.15.44:0/2172612051 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30003850 con 0x5614060bac50 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 -- 172.21.15.44:0/2172612051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 msgr2=0x5614060bd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060bd410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a3002aa10 tx=0x7f0a30000bc0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 -- 172.21.15.44:0/2172612051 shutdown_connections 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060bac50 0x5614060bd410 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 --2- 172.21.15.44:0/2172612051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060ba330 0x5614060b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 -- 172.21.15.44:0/2172612051 >> 172.21.15.44:0/2172612051 conn(0x5614060b68c0 msgr2=0x5614060b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 -- 172.21.15.44:0/2172612051 shutdown_connections 2024-09-13T17:27:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.085+0000 7f0a50249740 1 -- 172.21.15.44:0/2172612051 wait complete. 2024-09-13T17:27:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 -- start start 2024-09-13T17:27:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 Processor -- start 2024-09-13T17:27:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140612b2d0 con 0x5614060ba330 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140612b410 con 0x5614060c54f0 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140612b550 con 0x5614060bac50 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a50249740 10 client.0 _ll_register_callbacks cb 0x7f0a40009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33508/0 (socket says 172.21.15.44:33508) 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 -- 172.21.15.44:0/2672787660 learned_addr learned my addr 172.21.15.44:0/2672787660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 -- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 msgr2=0x5614060c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 -- 172.21.15.44:0/2672787660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 msgr2=0x5614060c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 -- 172.21.15.44:0/2672787660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56140601d330 con 0x5614060ba330 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a477fe640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060c8b10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0a40010d00 tx=0x7f0a30031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cec76693d1ddc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.086+0000 7f0a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2672787660 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4002b630 con 0x5614060ba330 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2672787660 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0a4002b790 con 0x5614060ba330 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561406047680 con 0x5614060ba330 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2672787660 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a40003970 con 0x5614060ba330 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 10 client.15777 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561405fe48e0 con 0x5614060ba330 2024-09-13T17:27:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2672787660 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561406047680 con 0x5614060ba330 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2672787660 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a4002bd20 con 0x5614060ba330 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 10 client.15777 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 20 client.15777 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56140612cb50 0x56140612ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 4 client.15777 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.087+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561405d23600 con 0x56140612cb50 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.088+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56140612cb50 0x56140612ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.088+0000 7f0a3ffff640 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56140612cb50 0x56140612ef70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cfbbf5133e6d95 server_cookie=1b4d4fc4742af9f5 in_seq=0 out_seq=0 2024-09-13T17:27:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.088+0000 7f0a2e7fc640 10 client.15777 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a2e7fc640 1 -- 172.21.15.44:0/2672787660 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561405d23600 con 0x56140612cb50 2024-09-13T17:27:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a2e7fc640 10 client.15777 handle_command_reply: tid=0 2024-09-13T17:27:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 client.15777 shutdown 2024-09-13T17:27:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 msgr2=0x5614060c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060c8b10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0a40010d00 tx=0x7f0a30031640 comp rx=0 tx=0).stop 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 shutdown_connections 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56140612cb50 0x56140612ef70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a2e7fc640 0 client.15777 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614060bac50 0x5614060c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614060c54f0 0x5614060c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 --2- 172.21.15.44:0/2672787660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614060ba330 0x5614060c8b10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.091+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 >> 172.21.15.44:0/2672787660 conn(0x5614060b68c0 msgr2=0x5614060b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 shutdown_connections 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a50249740 1 -- 172.21.15.44:0/2672787660 wait complete. 2024-09-13T17:27:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a50249740 20 client.15777 trim_cache size 0 max 16384 2024-09-13T17:27:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:27:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:27:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:27:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0a340a13e0 msgr2=0x7f0a340a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0a340a13e0 0x7f0a340a38a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a380c9e30 tx=0x7f0a380b9040 comp rx=0 tx=0).stop 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 msgr2=0x7f0a480727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 0x7f0a480727b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800d770 tx=0x7f0a3800dc30 comp rx=0 tx=0).stop 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 shutdown_connections 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0a340a13e0 0x7f0a340a38a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a48105090 0x7f0a48072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a48073230 0x7f0a481a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 --2- 172.21.15.44:0/34328068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a480fff40 0x7f0a480727b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 >> 172.21.15.44:0/34328068 conn(0x7f0a480fb9d0 msgr2=0x7f0a480fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 shutdown_connections 2024-09-13T17:27:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:19.092+0000 7f0a4d448640 1 -- 172.21.15.44:0/34328068 wait complete. 2024-09-13T17:27:20.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:20 smithi070 ceph-mon[34502]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-13T17:27:20.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:20 smithi100 ceph-mon[35055]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-13T17:27:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:20 smithi044 ceph-mon[31175]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-13T17:27:20.623 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:27:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 -- 172.21.15.44:0/18877475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6240ff810 msgr2=0x7fa624103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 --2- 172.21.15.44:0/18877475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6240ff810 0x7fa624103850 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa618009930 tx=0x7fa61802a420 comp rx=0 tx=0).stop 2024-09-13T17:27:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 -- 172.21.15.44:0/18877475 shutdown_connections 2024-09-13T17:27:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 --2- 172.21.15.44:0/18877475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103fb0 0x7fa62410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 --2- 172.21.15.44:0/18877475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6240ff810 0x7fa624103850 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 --2- 172.21.15.44:0/18877475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6240fee60 0x7fa6240ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 -- 172.21.15.44:0/18877475 >> 172.21.15.44:0/18877475 conn(0x7fa624075c60 msgr2=0x7fa6240fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 -- 172.21.15.44:0/18877475 shutdown_connections 2024-09-13T17:27:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.745+0000 7fa629472640 1 -- 172.21.15.44:0/18877475 wait complete. 2024-09-13T17:27:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.746+0000 7fa629472640 1 Processor -- start 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.746+0000 7fa629472640 1 -- start start 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa629472640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6240fee60 0x7fa6241970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa629472640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa624103fb0 0x7fa624197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa629472640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 0x7fa6241a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa629472640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241a27e0 con 0x7fa624197b40 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa629472640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241a2950 con 0x7fa624103fb0 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa629472640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241a2b10 con 0x7fa6240fee60 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 0x7fa6241a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6240fee60 0x7fa6241970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 0x7fa6241a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48680/0 (socket says 172.21.15.44:48680) 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 -- 172.21.15.44:0/1162284943 learned_addr learned my addr 172.21.15.44:0/1162284943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6227fc640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa624103fb0 0x7fa624197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 -- 172.21.15.44:0/1162284943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6240fee60 msgr2=0x7fa6241970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6240fee60 0x7fa6241970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 -- 172.21.15.44:0/1162284943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa624103fb0 msgr2=0x7fa624197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa624103fb0 0x7fa624197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.747+0000 7fa6237fe640 1 -- 172.21.15.44:0/1162284943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa618009560 con 0x7fa624197b40 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.748+0000 7fa6237fe640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 0x7fa6241a2140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00ea60 tx=0x7fa61c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.748+0000 7fa60bfff640 1 -- 172.21.15.44:0/1162284943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00ce40 con 0x7fa624197b40 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.748+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6241a2df0 con 0x7fa624197b40 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.748+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6241a32d0 con 0x7fa624197b40 2024-09-13T17:27:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.748+0000 7fa60bfff640 1 -- 172.21.15.44:0/1162284943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61c005020 con 0x7fa624197b40 2024-09-13T17:27:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.748+0000 7fa60bfff640 1 -- 172.21.15.44:0/1162284943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c010690 con 0x7fa624197b40 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.751+0000 7fa62c273740 1 Processor -- start 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.751+0000 7fa60bfff640 1 -- 172.21.15.44:0/1162284943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fa61c004a40 con 0x7fa624197b40 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.751+0000 7fa62c273740 1 -- start start 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.751+0000 7fa62c273740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cb41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.751+0000 7fa62c273740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa62c273740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cc13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa62c273740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97c1ef20 con 0x559e97cbefd0 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa62c273740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97c1c6b0 con 0x559e97cb3e10 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa62c273740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97c17fa0 con 0x559e97cb4730 2024-09-13T17:27:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cc13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cc13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48696/0 (socket says 172.21.15.44:48696) 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 -- 172.21.15.44:0/674627036 learned_addr learned my addr 172.21.15.44:0/674627036 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa622ffd640 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cb41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 -- 172.21.15.44:0/674627036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 msgr2=0x559e97cbea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 -- 172.21.15.44:0/674627036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 msgr2=0x559e97cb41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cb41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 -- 172.21.15.44:0/674627036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e97bd2840 con 0x559e97cbefd0 2024-09-13T17:27:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa60bfff640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa60c0a13b0 0x7fa60c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa6227fc640 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cc13c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa618005ec0 tx=0x7fa618005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1c4e61d04b80ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.752+0000 7fa622ffd640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa60c0a13b0 0x7fa60c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa60bfff640 1 -- 172.21.15.44:0/1162284943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa61c014070 con 0x7fa624197b40 2024-09-13T17:27:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa6097fa640 1 -- 172.21.15.44:0/674627036 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618003770 con 0x559e97cbefd0 2024-09-13T17:27:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa6097fa640 1 -- 172.21.15.44:0/674627036 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa618003d60 con 0x559e97cbefd0 2024-09-13T17:27:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa6097fa640 1 -- 172.21.15.44:0/674627036 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61803db70 con 0x559e97cbefd0 2024-09-13T17:27:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa622ffd640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa60c0a13b0 0x7fa60c0a3870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x559e97cb11e0 tx=0x7fa61400b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 -- 172.21.15.44:0/674627036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 msgr2=0x559e97cc13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cc13c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa618005ec0 tx=0x7fa618005fd0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 -- 172.21.15.44:0/674627036 shutdown_connections 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cc13c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 --2- 172.21.15.44:0/674627036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cb41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.753+0000 7fa62c273740 1 -- 172.21.15.44:0/674627036 >> 172.21.15.44:0/674627036 conn(0x559e97cb0380 msgr2=0x559e97cb0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 -- 172.21.15.44:0/674627036 shutdown_connections 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 -- 172.21.15.44:0/674627036 wait complete. 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 -- start start 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 Processor -- start 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cbe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cbc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97bb92a0 con 0x559e97cbefd0 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e978f0430 con 0x559e97cb3e10 2024-09-13T17:27:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97b9b680 con 0x559e97cb4730 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa62c273740 10 client.0 _ll_register_callbacks cb 0x559e97c11610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cbc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cbc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48704/0 (socket says 172.21.15.44:48704) 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.754+0000 7fa6237fe640 1 -- 172.21.15.44:0/831182769 learned_addr learned my addr 172.21.15.44:0/831182769 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa622ffd640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cbe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6227fc640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6237fe640 1 -- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 msgr2=0x559e97cbbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6237fe640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6237fe640 1 -- 172.21.15.44:0/831182769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 msgr2=0x559e97cbe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6237fe640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cbe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6237fe640 1 -- 172.21.15.44:0/831182769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa618045040 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa622ffd640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cbe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa6237fe640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cbc250 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa61c099eb0 tx=0x7fa61c09a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a301194a2e44e1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa606ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa606ff5640 1 -- 172.21.15.44:0/831182769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c09a610 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa606ff5640 1 -- 172.21.15.44:0/831182769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa61c0996a0 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e9790b4e0 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa606ff5640 1 -- 172.21.15.44:0/831182769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa61c099860 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa62c273740 10 client.15789 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.755+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e97c16620 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa606ff5640 1 -- 172.21.15.44:0/831182769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61c07f080 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa606ff5640 1 -- 172.21.15.44:0/831182769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa61c01f060 con 0x559e97cbefd0 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa62c273740 10 client.15789 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa62c273740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa62c273740 20 client.15789 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa62c273740 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97ccee50 0x559e97cd1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa62c273740 4 client.15789 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e9791c7e0 con 0x559e97ccee50 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.756+0000 7fa622ffd640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97ccee50 0x559e97cd1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa606ff5640 10 client.15789 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa622ffd640 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97ccee50 0x559e97cd1270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565f51ba3e616e9b server_cookie=c889022113b4deba in_seq=0 out_seq=0 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa606ff5640 1 -- 172.21.15.44:0/831182769 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559e9791c7e0 con 0x559e97ccee50 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa606ff5640 10 client.15789 handle_command_reply: tid=0 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 client.15789 shutdown 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 msgr2=0x559e97cbc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cbc250 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa61c099eb0 tx=0x7fa61c09a1b0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 shutdown_connections 2024-09-13T17:27:20.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97ccee50 0x559e97cd1270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cbbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cbefd0 0x559e97cbc250 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 --2- 172.21.15.44:0/831182769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cb3e10 0x559e97cbe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.757+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 >> 172.21.15.44:0/831182769 conn(0x559e97cb0380 msgr2=0x559e97cb6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.758+0000 7fa606ff5640 0 client.15789 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.758+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 shutdown_connections 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.758+0000 7fa62c273740 1 -- 172.21.15.44:0/831182769 wait complete. 2024-09-13T17:27:20.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.758+0000 7fa62c273740 20 client.15789 trim_cache size 0 max 16384 2024-09-13T17:27:20.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 Processor -- start 2024-09-13T17:27:20.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 -- start start 2024-09-13T17:27:20.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 0x559e97cd5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cbefd0 0x559e97cb0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97ccb580 con 0x559e97cb3e10 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97ccace0 con 0x559e97cbefd0 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa62c273740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97cb12e0 con 0x559e97cb4730 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 0x559e97cd5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 0x559e97cd5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48706/0 (socket says 172.21.15.44:48706) 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa622ffd640 1 -- 172.21.15.44:0/3066872471 learned_addr learned my addr 172.21.15.44:0/3066872471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa6237fe640 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cbefd0 0x559e97cb0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa622ffd640 1 -- 172.21.15.44:0/3066872471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 msgr2=0x559e97cb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.772+0000 7fa622ffd640 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa622ffd640 1 -- 172.21.15.44:0/3066872471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cbefd0 msgr2=0x559e97cb0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa622ffd640 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cbefd0 0x559e97cb0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa622ffd640 1 -- 172.21.15.44:0/3066872471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e97bb8790 con 0x559e97cb3e10 2024-09-13T17:27:20.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa622ffd640 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 0x559e97cd5bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa614005fd0 tx=0x7fa614016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8166da3a0969130b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa609ffb640 1 -- 172.21.15.44:0/3066872471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614030070 con 0x559e97cb3e10 2024-09-13T17:27:20.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa609ffb640 1 -- 172.21.15.44:0/3066872471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa614026040 con 0x559e97cb3e10 2024-09-13T17:27:20.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.773+0000 7fa609ffb640 1 -- 172.21.15.44:0/3066872471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614024040 con 0x559e97cb3e10 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 -- 172.21.15.44:0/3066872471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 msgr2=0x559e97cd5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 0x559e97cd5bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa614005fd0 tx=0x7fa614016000 comp rx=0 tx=0).stop 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 -- 172.21.15.44:0/3066872471 shutdown_connections 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cb4730 0x559e97cb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97cbefd0 0x559e97cb0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 --2- 172.21.15.44:0/3066872471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb3e10 0x559e97cd5bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 -- 172.21.15.44:0/3066872471 >> 172.21.15.44:0/3066872471 conn(0x559e97cb0380 msgr2=0x559e97cab1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 -- 172.21.15.44:0/3066872471 shutdown_connections 2024-09-13T17:27:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 -- 172.21.15.44:0/3066872471 wait complete. 2024-09-13T17:27:20.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.774+0000 7fa62c273740 1 -- start start 2024-09-13T17:27:20.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 Processor -- start 2024-09-13T17:27:20.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 0x559e97cc1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cbefd0 0x559e97cc24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97d22350 0x559e97d24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 0x559e97cc1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97cc2c40 con 0x559e97cb4730 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97d24c80 con 0x559e97d22350 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e97d24e40 con 0x559e97cbefd0 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 0x559e97cc1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48722/0 (socket says 172.21.15.44:48722) 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 -- 172.21.15.44:0/149941726 learned_addr learned my addr 172.21.15.44:0/149941726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa62c273740 10 client.0 _ll_register_callbacks cb 0x7fa61c005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa6227fc640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cbefd0 0x559e97cc24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa6237fe640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97d22350 0x559e97d24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 -- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cbefd0 msgr2=0x559e97cc24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cbefd0 0x559e97cc24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 -- 172.21.15.44:0/149941726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97d22350 msgr2=0x559e97d24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97d22350 0x559e97d24740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa622ffd640 1 -- 172.21.15.44:0/149941726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61400bdd0 con 0x559e97cb4730 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.775+0000 7fa6237fe640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97d22350 0x559e97d24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa622ffd640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 0x559e97cc1f90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa614018040 tx=0x7fa6140056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e632097f255679f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa606ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa606ff5640 1 -- 172.21.15.44:0/149941726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614024070 con 0x559e97cb4730 2024-09-13T17:27:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa606ff5640 1 -- 172.21.15.44:0/149941726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa6140200c0 con 0x559e97cb4730 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa618003d60 con 0x559e97cb4730 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa606ff5640 1 -- 172.21.15.44:0/149941726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa614030070 con 0x559e97cb4730 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa62c273740 10 client.15795 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e97bddea0 con 0x559e97cb4730 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.776+0000 7fa606ff5640 1 -- 172.21.15.44:0/149941726 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61402b8c0 con 0x559e97cb4730 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa606ff5640 1 -- 172.21.15.44:0/149941726 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa61403d070 con 0x559e97cb4730 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa62c273740 10 client.15795 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa62c273740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa62c273740 20 client.15795 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa62c273740 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97d265b0 0x559e97d289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa62c273740 4 client.15795 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fa618003d60 con 0x559e97d265b0 2024-09-13T17:27:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.777+0000 7fa6227fc640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97d265b0 0x559e97d289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa606ff5640 10 client.15795 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa6227fc640 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97d265b0 0x559e97d289d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1345cc4cc9b8caae server_cookie=aa44de7b75c43e0b in_seq=0 out_seq=0 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa606ff5640 1 -- 172.21.15.44:0/149941726 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fa618003d60 con 0x559e97d265b0 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa606ff5640 10 client.15795 handle_command_reply: tid=0 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 client.15795 shutdown 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 msgr2=0x559e97cc1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 0x559e97cc1f90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa614018040 tx=0x7fa6140056d0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 shutdown_connections 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559e97d265b0 0x559e97d289d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e97cbefd0 0x559e97cc24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e97d22350 0x559e97d24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa606ff5640 0 client.15795 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 --2- 172.21.15.44:0/149941726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e97cb4730 0x559e97cc1f90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 >> 172.21.15.44:0/149941726 conn(0x559e97cb0380 msgr2=0x559e97cd5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 shutdown_connections 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.778+0000 7fa62c273740 1 -- 172.21.15.44:0/149941726 wait complete. 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa62c273740 20 client.15795 trim_cache size 0 max 16384 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:27:20.788 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa60c0a13b0 msgr2=0x7fa60c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa60c0a13b0 0x7fa60c0a3870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x559e97cb11e0 tx=0x7fa61400b410 comp rx=0 tx=0).stop 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 msgr2=0x7fa6241a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 0x7fa6241a2140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00ea60 tx=0x7fa61c00ef20 comp rx=0 tx=0).stop 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 shutdown_connections 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa60c0a13b0 0x7fa60c0a3870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa624197b40 0x7fa6241a2140 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa624103fb0 0x7fa624197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 --2- 172.21.15.44:0/1162284943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6240fee60 0x7fa6241970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 >> 172.21.15.44:0/1162284943 conn(0x7fa624075c60 msgr2=0x7fa624102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.779+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 shutdown_connections 2024-09-13T17:27:20.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.780+0000 7fa629472640 1 -- 172.21.15.44:0/1162284943 wait complete. 2024-09-13T17:27:20.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:27:20.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.920+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1502032735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d80ff830 msgr2=0x7f94d8103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.920+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1502032735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d80ff830 0x7f94d8103870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009930 tx=0x7f94cc02a420 comp rx=0 tx=0).stop 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1502032735 shutdown_connections 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1502032735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d8103fd0 0x7f94d810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1502032735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d80ff830 0x7f94d8103870 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1502032735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94d80fee80 0x7f94d80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1502032735 >> 172.21.15.44:0/1502032735 conn(0x7f94d8075c60 msgr2=0x7f94d80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:20.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1502032735 shutdown_connections 2024-09-13T17:27:20.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1502032735 wait complete. 2024-09-13T17:27:20.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 Processor -- start 2024-09-13T17:27:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.921+0000 7f94dd6aa640 1 -- start start 2024-09-13T17:27:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94dd6aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d80fee80 0x7f94d81961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94dd6aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 0x7f94d8196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94dd6aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94d819ab20 0x7f94d819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94dd6aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d819b510 con 0x7f94d8103fd0 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94dd6aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d819b680 con 0x7f94d80fee80 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94dd6aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d819b7f0 con 0x7f94d819ab20 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 0x7f94d8196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 0x7f94d8196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48738/0 (socket says 172.21.15.44:48738) 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 -- 172.21.15.44:0/1423911338 learned_addr learned my addr 172.21.15.44:0/1423911338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d6ffd640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d80fee80 0x7f94d81961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d77fe640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94d819ab20 0x7f94d819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 -- 172.21.15.44:0/1423911338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94d819ab20 msgr2=0x7f94d819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94d819ab20 0x7f94d819afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 -- 172.21.15.44:0/1423911338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d80fee80 msgr2=0x7f94d81961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d80fee80 0x7f94d81961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d67fc640 1 -- 172.21.15.44:0/1423911338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc009560 con 0x7f94d8103fd0 2024-09-13T17:27:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.922+0000 7f94d6ffd640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d80fee80 0x7f94d81961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.923+0000 7f94d67fc640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 0x7f94d8196730 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f94cc005ab0 tx=0x7f94cc032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.923+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc032e10 con 0x7f94d8103fd0 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.923+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d81a2dd0 con 0x7f94d8103fd0 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.923+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94cc0046c0 con 0x7f94d8103fd0 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.923+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003850 con 0x7f94d8103fd0 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.923+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d81a31f0 con 0x7f94d8103fd0 2024-09-13T17:27:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.924+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d8100350 con 0x7f94d8103fd0 2024-09-13T17:27:20.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.926+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f94cc004830 con 0x7f94d8103fd0 2024-09-13T17:27:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.927+0000 7f94bffff640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94c00a1430 0x7f94c00a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.927+0000 7f94d6ffd640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94c00a1430 0x7f94c00a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.928+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f94cc0ee190 con 0x7f94d8103fd0 2024-09-13T17:27:20.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.928+0000 7f94d6ffd640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94c00a1430 0x7f94c00a38f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94c8005fd0 tx=0x7f94c800b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:20.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:20.931+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f94cc0b7130 con 0x7f94d8103fd0 2024-09-13T17:27:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.213+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94d80ff260 con 0x7f94d8103fd0 2024-09-13T17:27:21.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.214+0000 7f94bffff640 1 -- 172.21.15.44:0/1423911338 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f94cc040090 con 0x7f94d8103fd0 2024-09-13T17:27:21.216 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:27:21.216 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:27:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:27:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94c00a1430 msgr2=0x7f94c00a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94c00a1430 0x7f94c00a38f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94c8005fd0 tx=0x7f94c800b390 comp rx=0 tx=0).stop 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 msgr2=0x7f94d8196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 0x7f94d8196730 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f94cc005ab0 tx=0x7f94cc032c20 comp rx=0 tx=0).stop 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 shutdown_connections 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94c00a1430 0x7f94c00a38f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94d819ab20 0x7f94d819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94d8103fd0 0x7f94d8196730 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 --2- 172.21.15.44:0/1423911338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94d80fee80 0x7f94d81961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.217+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 >> 172.21.15.44:0/1423911338 conn(0x7f94d8075c60 msgr2=0x7f94d8102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.218+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 shutdown_connections 2024-09-13T17:27:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.218+0000 7f94dd6aa640 1 -- 172.21.15.44:0/1423911338 wait complete. 2024-09-13T17:27:21.234 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:27:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 -- 172.21.15.44:0/3775558898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80ff810 msgr2=0x7fcea8103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 --2- 172.21.15.44:0/3775558898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80ff810 0x7fcea8103850 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce94009930 tx=0x7fce9402a420 comp rx=0 tx=0).stop 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 -- 172.21.15.44:0/3775558898 shutdown_connections 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 --2- 172.21.15.44:0/3775558898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8103fb0 0x7fcea810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 --2- 172.21.15.44:0/3775558898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80ff810 0x7fcea8103850 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 --2- 172.21.15.44:0/3775558898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea80fee60 0x7fcea80ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 -- 172.21.15.44:0/3775558898 >> 172.21.15.44:0/3775558898 conn(0x7fcea8075c60 msgr2=0x7fcea80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 -- 172.21.15.44:0/3775558898 shutdown_connections 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.342+0000 7fceaed0f640 1 -- 172.21.15.44:0/3775558898 wait complete. 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 Processor -- start 2024-09-13T17:27:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 -- start start 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80fee60 0x7fcea80fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea8103fb0 0x7fcea80fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 0x7fcea81a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea81a4a80 con 0x7fcea80fbb20 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea81a4bf0 con 0x7fcea80fee60 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.343+0000 7fceaed0f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea81a4db0 con 0x7fcea8103fb0 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 0x7fcea81a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 0x7fcea81a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48758/0 (socket says 172.21.15.44:48758) 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 -- 172.21.15.44:0/168371209 learned_addr learned my addr 172.21.15.44:0/168371209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcea7fff640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea8103fb0 0x7fcea80fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fceacaab640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80fee60 0x7fcea80fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 -- 172.21.15.44:0/168371209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea8103fb0 msgr2=0x7fcea80fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea8103fb0 0x7fcea80fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 -- 172.21.15.44:0/168371209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80fee60 msgr2=0x7fcea80fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80fee60 0x7fcea80fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 -- 172.21.15.44:0/168371209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009840 con 0x7fcea80fbb20 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcead2ac640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 0x7fcea81a4430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcea000ec70 tx=0x7fcea000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcea5ffb640 1 -- 172.21.15.44:0/168371209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0003b70 con 0x7fcea80fbb20 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fcea5ffb640 1 -- 172.21.15.44:0/168371209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcea000ee80 con 0x7fcea80fbb20 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.344+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce94009560 con 0x7fcea80fbb20 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.345+0000 7fcea5ffb640 1 -- 172.21.15.44:0/168371209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea00106d0 con 0x7fcea80fbb20 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.345+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea81a53e0 con 0x7fcea80fbb20 2024-09-13T17:27:21.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 Processor -- start 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fcea5ffb640 1 -- 172.21.15.44:0/168371209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fcea0004020 con 0x7fcea80fbb20 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 -- start start 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f139ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13a5950 0x55e9f13a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f1304cc0 con 0x55e9f139b0b0 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f1302450 con 0x55e9f139a790 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.348+0000 7fceb1a30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f12fdd40 con 0x55e9f13a5950 2024-09-13T17:27:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48760/0 (socket says 172.21.15.44:48760) 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 -- 172.21.15.44:0/2142439529 learned_addr learned my addr 172.21.15.44:0/2142439529 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fcea7fff640 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13a5950 0x55e9f13a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fcea7fff640 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f139ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fcea5ffb640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fce980a1500 0x7fce980a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 -- 172.21.15.44:0/2142439529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13a5950 msgr2=0x55e9f13a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13a5950 0x55e9f13a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 -- 172.21.15.44:0/2142439529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 msgr2=0x55e9f139ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f139ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fcead2ac640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fce980a1500 0x7fce980a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fceacaab640 1 -- 172.21.15.44:0/2142439529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f12c5620 con 0x55e9f139b0b0 2024-09-13T17:27:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.349+0000 7fcea5ffb640 1 -- 172.21.15.44:0/168371209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcea0014070 con 0x7fcea80fbb20 2024-09-13T17:27:21.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.350+0000 7fceacaab640 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a5410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009d20 tx=0x7fce9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f675fdaba18d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.350+0000 7fce92ffd640 1 -- 172.21.15.44:0/2142439529 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c009e10 con 0x55e9f139b0b0 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.350+0000 7fcead2ac640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fce980a1500 0x7fce980a39c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cad00 tx=0x7fcea0014d30 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.350+0000 7fce92ffd640 1 -- 172.21.15.44:0/2142439529 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce9c0198a0 con 0x55e9f139b0b0 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.350+0000 7fce92ffd640 1 -- 172.21.15.44:0/2142439529 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c018880 con 0x55e9f139b0b0 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 -- 172.21.15.44:0/2142439529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 msgr2=0x55e9f13a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a5410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009d20 tx=0x7fce9c011810 comp rx=0 tx=0).stop 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 -- 172.21.15.44:0/2142439529 shutdown_connections 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13a5950 0x55e9f13a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a5410 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2142439529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f139ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 -- 172.21.15.44:0/2142439529 >> 172.21.15.44:0/2142439529 conn(0x55e9f1396d00 msgr2=0x55e9f13970e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 -- 172.21.15.44:0/2142439529 shutdown_connections 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 -- 172.21.15.44:0/2142439529 wait complete. 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.351+0000 7fceb1a30740 1 -- start start 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 Processor -- start 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f12bc180 con 0x55e9f139b0b0 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f0fd6430 con 0x55e9f13a5950 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f12815e0 con 0x55e9f139a790 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceb1a30740 10 client.0 _ll_register_callbacks cb 0x55e9f12edaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceacaab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceacaab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48766/0 (socket says 172.21.15.44:48766) 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fceacaab640 1 -- 172.21.15.44:0/2275878058 learned_addr learned my addr 172.21.15.44:0/2275878058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fcea7fff640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.352+0000 7fcea7fff640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceacaab640 1 -- 172.21.15.44:0/2275878058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 msgr2=0x55e9f13a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceacaab640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceacaab640 1 -- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 msgr2=0x55e9f13a27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceacaab640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13a27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceacaab640 1 -- 172.21.15.44:0/2275878058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c009950 con 0x55e9f139b0b0 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceacaab640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a2280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009710 tx=0x7fce9c019c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd9609ce947fb41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fce87fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:21.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fce87fff640 1 -- 172.21.15.44:0/2275878058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c009e80 con 0x55e9f139b0b0 2024-09-13T17:27:21.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9f1327b80 con 0x55e9f139b0b0 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fce87fff640 1 -- 172.21.15.44:0/2275878058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fce9c0052e0 con 0x55e9f139b0b0 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fce87fff640 1 -- 172.21.15.44:0/2275878058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce9c0054a0 con 0x55e9f139b0b0 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceb1a30740 10 client.15813 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.353+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e9f12fc3c0 con 0x55e9f139b0b0 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fce87fff640 1 -- 172.21.15.44:0/2275878058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e9f1327b80 con 0x55e9f139b0b0 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fce87fff640 1 -- 172.21.15.44:0/2275878058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce9c018650 con 0x55e9f139b0b0 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fceb1a30740 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fceb1a30740 10 client.15813 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fceb1a30740 20 client.15813 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f13b5210 0x55e9f13b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fceb1a30740 4 client.15813 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9f12f01f0 con 0x55e9f13b5210 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.354+0000 7fcea7fff640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f13b5210 0x55e9f13b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fcea7fff640 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f13b5210 0x55e9f13b7630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f946927fab61aad5 server_cookie=2ff6a57393f6a262 in_seq=0 out_seq=0 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fce87fff640 10 client.15813 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fce87fff640 1 -- 172.21.15.44:0/2275878058 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e9f12f01f0 con 0x55e9f13b5210 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fce87fff640 10 client.15813 handle_command_reply: tid=0 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 client.15813 shutdown 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 msgr2=0x55e9f13a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a2280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009710 tx=0x7fce9c019c40 comp rx=0 tx=0).stop 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 shutdown_connections 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f13b5210 0x55e9f13b7630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fce87fff640 0 client.15813 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f139b0b0 0x55e9f13a2280 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 --2- 172.21.15.44:0/2275878058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.355+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 >> 172.21.15.44:0/2275878058 conn(0x55e9f1396d00 msgr2=0x55e9f139ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.356+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 shutdown_connections 2024-09-13T17:27:21.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.356+0000 7fceb1a30740 1 -- 172.21.15.44:0/2275878058 wait complete. 2024-09-13T17:27:21.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.356+0000 7fceb1a30740 20 client.15813 trim_cache size 0 max 16384 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 Processor -- start 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 -- start start 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13977f0 0x55e9f13acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f12f01f0 con 0x55e9f13977f0 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f13913b0 con 0x55e9f13a5950 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.378+0000 7fceb1a30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f13a3160 con 0x55e9f139a790 2024-09-13T17:27:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fcea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13977f0 0x55e9f13acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:45696/0 (socket says 172.21.15.44:45696) 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 -- 172.21.15.44:0/3679116257 learned_addr learned my addr 172.21.15.44:0/3679116257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fcea7fff640 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 -- 172.21.15.44:0/3679116257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 msgr2=0x55e9f13966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 -- 172.21.15.44:0/3679116257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13977f0 msgr2=0x55e9f13acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13977f0 0x55e9f13acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 -- 172.21.15.44:0/3679116257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f129e770 con 0x55e9f13a5950 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fceacaab640 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13ac610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce9c018a60 tx=0x7fce9c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66193a2fec1312d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.379+0000 7fce937fe640 1 -- 172.21.15.44:0/3679116257 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0187c0 con 0x55e9f13a5950 2024-09-13T17:27:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fce937fe640 1 -- 172.21.15.44:0/3679116257 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce9c005d30 con 0x55e9f13a5950 2024-09-13T17:27:21.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fce937fe640 1 -- 172.21.15.44:0/3679116257 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0038e0 con 0x55e9f13a5950 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 -- 172.21.15.44:0/3679116257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 msgr2=0x55e9f13ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13ac610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce9c018a60 tx=0x7fce9c005f90 comp rx=0 tx=0).stop 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 -- 172.21.15.44:0/3679116257 shutdown_connections 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13977f0 0x55e9f13acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f13a5950 0x55e9f13ac610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 --2- 172.21.15.44:0/3679116257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f139a790 0x55e9f13966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.380+0000 7fceb1a30740 1 -- 172.21.15.44:0/3679116257 >> 172.21.15.44:0/3679116257 conn(0x55e9f1396d00 msgr2=0x55e9f13970e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.381+0000 7fceb1a30740 1 -- 172.21.15.44:0/3679116257 shutdown_connections 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.381+0000 7fceb1a30740 1 -- 172.21.15.44:0/3679116257 wait complete. 2024-09-13T17:27:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.381+0000 7fceb1a30740 1 -- start start 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.381+0000 7fceb1a30740 1 Processor -- start 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13977f0 0x55e9f13ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f13acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 0x55e9f13a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f13a94d0 con 0x55e9f13a5950 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f13a9640 con 0x55e9f139a790 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f13a9800 con 0x55e9f13977f0 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceb1a30740 10 client.0 _ll_register_callbacks cb 0x7fce9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 0x55e9f13a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 0x55e9f13a8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48774/0 (socket says 172.21.15.44:48774) 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 -- 172.21.15.44:0/1495090582 learned_addr learned my addr 172.21.15.44:0/1495090582 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceacaab640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f13acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13977f0 0x55e9f13ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 -- 172.21.15.44:0/1495090582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13977f0 msgr2=0x55e9f13ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13977f0 0x55e9f13ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 -- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 msgr2=0x55e9f13acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f13acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 -- 172.21.15.44:0/1495090582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9f12fc3c0 con 0x55e9f13a5950 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fceacaab640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f13acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fcea7fff640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 0x55e9f13a8dd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce94009340 tx=0x7fce9403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ea45b663685696 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.382+0000 7fce87fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fce87fff640 1 -- 172.21.15.44:0/1495090582 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94003f70 con 0x55e9f13a5950 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fce87fff640 1 -- 172.21.15.44:0/1495090582 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fce9404b3b0 con 0x55e9f13a5950 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce9c0198a0 con 0x55e9f13a5950 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fce87fff640 1 -- 172.21.15.44:0/1495090582 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce9404b570 con 0x55e9f13a5950 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 10 client.15819 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e9f12c3e80 con 0x55e9f13a5950 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fce87fff640 1 -- 172.21.15.44:0/1495090582 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce94051070 con 0x55e9f13a5950 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fce87fff640 1 -- 172.21.15.44:0/1495090582 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce94055030 con 0x55e9f13a5950 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 10 client.15819 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 10 client.15819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 20 client.15819 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f140ca00 0x55e9f140ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 4 client.15819 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.383+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fce9c0198a0 con 0x55e9f140ca00 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.384+0000 7fceacaab640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f140ca00 0x55e9f140ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.384+0000 7fce87fff640 10 client.15819 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.384+0000 7fceacaab640 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f140ca00 0x55e9f140ee20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8ce74db117ca30f server_cookie=b9c1d537e22615eb in_seq=0 out_seq=0 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.384+0000 7fce87fff640 1 -- 172.21.15.44:0/1495090582 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fce9c0198a0 con 0x55e9f140ca00 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.384+0000 7fce87fff640 10 client.15819 handle_command_reply: tid=0 2024-09-13T17:27:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.384+0000 7fceb1a30740 1 client.15819 shutdown 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 msgr2=0x55e9f13a8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 0x55e9f13a8dd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce94009340 tx=0x7fce9403f040 comp rx=0 tx=0).stop 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 shutdown_connections 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e9f140ca00 0x55e9f140ee20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9f13a5950 0x55e9f13a8dd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fce87fff640 0 client.15819 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e9f139a790 0x55e9f13acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 --2- 172.21.15.44:0/1495090582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9f13977f0 0x55e9f13ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 >> 172.21.15.44:0/1495090582 conn(0x55e9f1396d00 msgr2=0x55e9f13a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 shutdown_connections 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.385+0000 7fceb1a30740 1 -- 172.21.15.44:0/1495090582 wait complete. 2024-09-13T17:27:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceb1a30740 20 client.15819 trim_cache size 0 max 16384 2024-09-13T17:27:21.393 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:27:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fce980a1500 msgr2=0x7fce980a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceaed0f640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fce980a1500 0x7fce980a39c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cad00 tx=0x7fcea0014d30 comp rx=0 tx=0).stop 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 msgr2=0x7fcea81a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceaed0f640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 0x7fcea81a4430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcea000ec70 tx=0x7fcea000c6a0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 shutdown_connections 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.386+0000 7fceaed0f640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fce980a1500 0x7fce980a39c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.387+0000 7fceaed0f640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea8103fb0 0x7fcea80fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.387+0000 7fceaed0f640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80fbb20 0x7fcea81a4430 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.387+0000 7fceaed0f640 1 --2- 172.21.15.44:0/168371209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80fee60 0x7fcea80fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.387+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 >> 172.21.15.44:0/168371209 conn(0x7fcea8075c60 msgr2=0x7fcea8067740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.387+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 shutdown_connections 2024-09-13T17:27:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.387+0000 7fceaed0f640 1 -- 172.21.15.44:0/168371209 wait complete. 2024-09-13T17:27:21.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 -- 172.21.15.44:0/2886098732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b580feac0 msgr2=0x7f2b580feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/2886098732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b580feac0 0x7f2b580feea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009900 tx=0x7f2b5402a410 comp rx=0 tx=0).stop 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 -- 172.21.15.44:0/2886098732 shutdown_connections 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/2886098732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5810c670 0x7f2b5810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/2886098732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b580ff470 0x7f2b5810c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/2886098732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b580feac0 0x7f2b580feea0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.523+0000 7f2b5fca3640 1 -- 172.21.15.44:0/2886098732 >> 172.21.15.44:0/2886098732 conn(0x7f2b58075c60 msgr2=0x7f2b580fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.524+0000 7f2b5fca3640 1 -- 172.21.15.44:0/2886098732 shutdown_connections 2024-09-13T17:27:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.524+0000 7f2b5fca3640 1 -- 172.21.15.44:0/2886098732 wait complete. 2024-09-13T17:27:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.524+0000 7f2b5fca3640 1 Processor -- start 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.524+0000 7f2b5fca3640 1 -- start start 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5fca3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b580feac0 0x7f2b581983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5fca3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ff470 0x7f2b58198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5fca3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 0x7f2b5819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5fca3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5819d2f0 con 0x7f2b5810c670 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5fca3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5819d460 con 0x7f2b580feac0 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5fca3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5819d620 con 0x7f2b580ff470 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 0x7f2b5819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 0x7f2b5819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48800/0 (socket says 172.21.15.44:48800) 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 -- 172.21.15.44:0/1601818630 learned_addr learned my addr 172.21.15.44:0/1601818630 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ff470 0x7f2b58198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b580feac0 0x7f2b581983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 -- 172.21.15.44:0/1601818630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ff470 msgr2=0x7f2b58198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ff470 0x7f2b58198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 -- 172.21.15.44:0/1601818630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b580feac0 msgr2=0x7f2b581983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b580feac0 0x7f2b581983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 -- 172.21.15.44:0/1601818630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54009560 con 0x7f2b5810c670 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.525+0000 7f2b5e240640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 0x7f2b5819ccf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000d9d0 tx=0x7f2b5000de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.526+0000 7f2b4effd640 1 -- 172.21.15.44:0/1601818630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000bb70 con 0x7f2b5810c670 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.526+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5819d900 con 0x7f2b5810c670 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.526+0000 7f2b4effd640 1 -- 172.21.15.44:0/1601818630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b50003e00 con 0x7f2b5810c670 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.526+0000 7f2b4effd640 1 -- 172.21.15.44:0/1601818630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50004800 con 0x7f2b5810c670 2024-09-13T17:27:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.526+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b581a5180 con 0x7f2b5810c670 2024-09-13T17:27:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.529+0000 7f2b629c9740 1 Processor -- start 2024-09-13T17:27:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.529+0000 7f2b4effd640 1 -- 172.21.15.44:0/1601818630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f2b50003970 con 0x7f2b5810c670 2024-09-13T17:27:21.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.529+0000 7f2b629c9740 1 -- start start 2024-09-13T17:27:21.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 0x561bdfadbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfae6950 0x561bdfae8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b629c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfa45cc0 con 0x561bdfadc0b0 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b629c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfa43450 con 0x561bdfae6950 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b629c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfa3ed40 con 0x561bdfadb790 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5da3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 0x561bdfadbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 0x561bdfadbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:42448/0 (socket says 172.21.15.44:42448) 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1018248478 learned_addr learned my addr 172.21.15.44:0/1018248478 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfae6950 0x561bdfae8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b4effd640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2b440a13b0 0x7f2b440a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1018248478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfae6950 msgr2=0x561bdfae8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfae6950 0x561bdfae8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1018248478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 msgr2=0x561bdfae6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.530+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1018248478 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bdfa06620 con 0x561bdfadb790 2024-09-13T17:27:21.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b4effd640 1 -- 172.21.15.44:0/1601818630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2b500cca00 con 0x7f2b5810c670 2024-09-13T17:27:21.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b5e240640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2b440a13b0 0x7f2b440a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 0x561bdfadbb70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2b480127c0 tx=0x7f2b4800b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a86718bb6f93c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b5e240640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2b440a13b0 0x7f2b440a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b500c9c50 tx=0x7f2b50069530 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b43fff640 1 -- 172.21.15.44:0/1018248478 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48012e00 con 0x561bdfadb790 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b43fff640 1 -- 172.21.15.44:0/1018248478 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b48010c00 con 0x561bdfadb790 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.531+0000 7f2b43fff640 1 -- 172.21.15.44:0/1018248478 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801e870 con 0x561bdfadb790 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 -- 172.21.15.44:0/1018248478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 msgr2=0x561bdfadbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 0x561bdfadbb70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2b480127c0 tx=0x7f2b4800b7c0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 -- 172.21.15.44:0/1018248478 shutdown_connections 2024-09-13T17:27:21.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfae6950 0x561bdfae8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1018248478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadb790 0x561bdfadbb70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 -- 172.21.15.44:0/1018248478 >> 172.21.15.44:0/1018248478 conn(0x561bdfad7d00 msgr2=0x561bdfad80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 -- 172.21.15.44:0/1018248478 shutdown_connections 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 -- 172.21.15.44:0/1018248478 wait complete. 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 -- start start 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.532+0000 7f2b629c9740 1 Processor -- start 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfae5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfae3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdf9fd180 con 0x561bdfadc0b0 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdf717430 con 0x561bdfadb790 2024-09-13T17:27:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdf9c25e0 con 0x561bdfae6950 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b5da3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b629c9740 10 client.0 _ll_register_callbacks cb 0x561bdfa2eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b5da3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48810/0 (socket says 172.21.15.44:48810) 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b5da3f640 1 -- 172.21.15.44:0/3731766060 learned_addr learned my addr 172.21.15.44:0/3731766060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfae5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.533+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfae3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5d23e640 1 -- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 msgr2=0x561bdfae3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfae3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5d23e640 1 -- 172.21.15.44:0/3731766060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 msgr2=0x561bdfae3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5d23e640 1 -- 172.21.15.44:0/3731766060 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4801a040 con 0x561bdfadb790 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfae5f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b48012790 tx=0x7f2b48014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afc213d31e62cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b417fa640 1 -- 172.21.15.44:0/3731766060 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48010a40 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b417fa640 1 -- 172.21.15.44:0/3731766060 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2b4801e610 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bdfa68b80 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b417fa640 1 -- 172.21.15.44:0/3731766060 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b4801e7d0 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.534+0000 7f2b629c9740 10 client.25421 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561bdfa3d3c0 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b417fa640 1 -- 172.21.15.44:0/3731766060 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b48037080 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b417fa640 1 -- 172.21.15.44:0/3731766060 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b4802c7b0 con 0x561bdfadb790 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 10 client.25421 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 10 client.25421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 20 client.25421 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfaf61c0 0x561bdfaf85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 4 client.25421 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:21.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bdfa311f0 con 0x561bdfaf61c0 2024-09-13T17:27:21.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.535+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfaf61c0 0x561bdfaf85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.536+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfaf61c0 0x561bdfaf85b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3f932d3eb0ea79d server_cookie=de2775396867c37c in_seq=0 out_seq=0 2024-09-13T17:27:21.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.536+0000 7f2b417fa640 10 client.25421 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.536+0000 7f2b417fa640 1 -- 172.21.15.44:0/3731766060 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561bdfa311f0 con 0x561bdfaf61c0 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.536+0000 7f2b417fa640 10 client.25421 handle_command_reply: tid=0 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.536+0000 7f2b629c9740 1 client.25421 shutdown 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 msgr2=0x561bdfae5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfae5f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b48012790 tx=0x7f2b48014040 comp rx=0 tx=0).stop 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 shutdown_connections 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfaf61c0 0x561bdfaf85b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfae3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b417fa640 0 client.25421 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:21.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadc0b0 0x561bdfae3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 --2- 172.21.15.44:0/3731766060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfae5f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 >> 172.21.15.44:0/3731766060 conn(0x561bdfad7d00 msgr2=0x561bdfadda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 shutdown_connections 2024-09-13T17:27:21.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 1 -- 172.21.15.44:0/3731766060 wait complete. 2024-09-13T17:27:21.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.537+0000 7f2b629c9740 20 client.25421 trim_cache size 0 max 16384 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 Processor -- start 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 -- start start 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 0x561bdfae1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadc0b0 0x561bdfae2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfade200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfae3aa0 con 0x561bdfadb790 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfae1750 con 0x561bdfadc0b0 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b629c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfaf88e0 con 0x561bdfae6950 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b5d23e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 0x561bdfae1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b5d23e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 0x561bdfae1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48820/0 (socket says 172.21.15.44:48820) 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.548+0000 7f2b5d23e640 1 -- 172.21.15.44:0/2218575118 learned_addr learned my addr 172.21.15.44:0/2218575118 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b5d23e640 1 -- 172.21.15.44:0/2218575118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 msgr2=0x561bdfade200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfade200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b5d23e640 1 -- 172.21.15.44:0/2218575118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadc0b0 msgr2=0x561bdfae2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadc0b0 0x561bdfae2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b5d23e640 1 -- 172.21.15.44:0/2218575118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bdf9df770 con 0x561bdfadb790 2024-09-13T17:27:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 0x561bdfae1ad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b48012fd0 tx=0x7f2b4802ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c8006a9025fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b4cff9640 1 -- 172.21.15.44:0/2218575118 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48010a40 con 0x561bdfadb790 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b4cff9640 1 -- 172.21.15.44:0/2218575118 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561bdfa68b80 con 0x561bdfadb790 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.549+0000 7f2b4cff9640 1 -- 172.21.15.44:0/2218575118 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48005240 con 0x561bdfadb790 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 -- 172.21.15.44:0/2218575118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 msgr2=0x561bdfae1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 0x561bdfae1ad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b48012fd0 tx=0x7f2b4802ce80 comp rx=0 tx=0).stop 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 -- 172.21.15.44:0/2218575118 shutdown_connections 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfae6950 0x561bdfade200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadc0b0 0x561bdfae2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 --2- 172.21.15.44:0/2218575118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfadb790 0x561bdfae1ad0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 -- 172.21.15.44:0/2218575118 >> 172.21.15.44:0/2218575118 conn(0x561bdfad7d00 msgr2=0x561bdfad80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 -- 172.21.15.44:0/2218575118 shutdown_connections 2024-09-13T17:27:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.550+0000 7f2b629c9740 1 -- 172.21.15.44:0/2218575118 wait complete. 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.551+0000 7f2b629c9740 1 -- start start 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.551+0000 7f2b629c9740 1 Processor -- start 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.551+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfaed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.551+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadc0b0 0x561bdfaebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b629c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 0x561bdfaec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b629c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfaec9b0 con 0x561bdfae6950 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b629c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfaecb20 con 0x561bdfadb790 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b629c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdfb4c670 con 0x561bdfadc0b0 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 0x561bdfaec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 0x561bdfaec2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48834/0 (socket says 172.21.15.44:48834) 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1204816938 learned_addr learned my addr 172.21.15.44:0/1204816938 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfaed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b629c9740 10 client.0 _ll_register_callbacks cb 0x7f2b4800bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadc0b0 0x561bdfaebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadc0b0 msgr2=0x561bdfaebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadc0b0 0x561bdfaebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1204816938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 msgr2=0x561bdfaed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfaed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 -- 172.21.15.44:0/1204816938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bdfa3d3c0 con 0x561bdfae6950 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b5d23e640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 0x561bdfaec2b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x561bdfaeb5c0 tx=0x7f2b48001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16e5614bff56adb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.552+0000 7f2b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b417fa640 1 -- 172.21.15.44:0/1204816938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48012e00 con 0x561bdfae6950 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b417fa640 1 -- 172.21.15.44:0/1204816938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2b4802cc40 con 0x561bdfae6950 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b48010c00 con 0x561bdfae6950 2024-09-13T17:27:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b417fa640 1 -- 172.21.15.44:0/1204816938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b480051e0 con 0x561bdfae6950 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b629c9740 10 client.15837 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561bdfa04e80 con 0x561bdfae6950 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b417fa640 1 -- 172.21.15.44:0/1204816938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b48010c00 con 0x561bdfae6950 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.553+0000 7f2b417fa640 1 -- 172.21.15.44:0/1204816938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b480039e0 con 0x561bdfae6950 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b629c9740 10 client.15837 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b629c9740 10 client.15837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b629c9740 20 client.15837 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfb4d740 0x561bdfb4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b629c9740 4 client.15837 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561bdf7324e0 con 0x561bdfb4d740 2024-09-13T17:27:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfb4d740 0x561bdfb4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b417fa640 10 client.15837 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.554+0000 7f2b5da3f640 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfb4d740 0x561bdfb4fb60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11c6ee346f660c90 server_cookie=b9c2404c19108759 in_seq=0 out_seq=0 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b417fa640 1 -- 172.21.15.44:0/1204816938 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561bdf7324e0 con 0x561bdfb4d740 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b417fa640 10 client.15837 handle_command_reply: tid=0 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 client.15837 shutdown 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 msgr2=0x561bdfaec2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 0x561bdfaec2b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x561bdfaeb5c0 tx=0x7f2b48001e50 comp rx=0 tx=0).stop 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 shutdown_connections 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561bdfb4d740 0x561bdfb4fb60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdfadc0b0 0x561bdfaebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdfae6950 0x561bdfaec2b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b417fa640 0 client.15837 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 --2- 172.21.15.44:0/1204816938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdfadb790 0x561bdfaed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 >> 172.21.15.44:0/1204816938 conn(0x561bdfad7d00 msgr2=0x561bdfae19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 shutdown_connections 2024-09-13T17:27:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 1 -- 172.21.15.44:0/1204816938 wait complete. 2024-09-13T17:27:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.555+0000 7f2b629c9740 20 client.15837 trim_cache size 0 max 16384 2024-09-13T17:27:21.562 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:27:21.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2b440a13b0 msgr2=0x7f2b440a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2b440a13b0 0x7f2b440a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b500c9c50 tx=0x7f2b50069530 comp rx=0 tx=0).stop 2024-09-13T17:27:21.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 msgr2=0x7f2b5819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 0x7f2b5819ccf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000d9d0 tx=0x7f2b5000de90 comp rx=0 tx=0).stop 2024-09-13T17:27:21.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 shutdown_connections 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2b440a13b0 0x7f2b440a3870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ff470 0x7f2b58198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b5810c670 0x7f2b5819ccf0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 --2- 172.21.15.44:0/1601818630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b580feac0 0x7f2b581983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 >> 172.21.15.44:0/1601818630 conn(0x7f2b58075c60 msgr2=0x7f2b580fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.556+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 shutdown_connections 2024-09-13T17:27:21.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.557+0000 7f2b5fca3640 1 -- 172.21.15.44:0/1601818630 wait complete. 2024-09-13T17:27:21.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:27:21.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 -- 172.21.15.44:0/2855925555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0ff810 msgr2=0x7f115c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 --2- 172.21.15.44:0/2855925555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0ff810 0x7f115c103850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1148009930 tx=0x7f114802a420 comp rx=0 tx=0).stop 2024-09-13T17:27:21.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 -- 172.21.15.44:0/2855925555 shutdown_connections 2024-09-13T17:27:21.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 --2- 172.21.15.44:0/2855925555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f115c103fb0 0x7f115c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 --2- 172.21.15.44:0/2855925555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0ff810 0x7f115c103850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 --2- 172.21.15.44:0/2855925555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c0fee60 0x7f115c0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 -- 172.21.15.44:0/2855925555 >> 172.21.15.44:0/2855925555 conn(0x7f115c075c60 msgr2=0x7f115c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 -- 172.21.15.44:0/2855925555 shutdown_connections 2024-09-13T17:27:21.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.677+0000 7f11630a3640 1 -- 172.21.15.44:0/2855925555 wait complete. 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 Processor -- start 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 -- start start 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 0x7f115c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f115c103fb0 0x7f115c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c19ab10 0x7f115c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c19b500 con 0x7f115c0fee60 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c19b670 con 0x7f115c103fb0 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f11630a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c19b7e0 con 0x7f115c19ab10 2024-09-13T17:27:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f1160e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 0x7f115c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f1160e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 0x7f115c1961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48854/0 (socket says 172.21.15.44:48854) 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f1160e3f640 1 -- 172.21.15.44:0/3424701413 learned_addr learned my addr 172.21.15.44:0/3424701413 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f115bfff640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f115c103fb0 0x7f115c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.678+0000 7f1161640640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c19ab10 0x7f115c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1160e3f640 1 -- 172.21.15.44:0/3424701413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c19ab10 msgr2=0x7f115c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1160e3f640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c19ab10 0x7f115c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1160e3f640 1 -- 172.21.15.44:0/3424701413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f115c103fb0 msgr2=0x7f115c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1160e3f640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f115c103fb0 0x7f115c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1160e3f640 1 -- 172.21.15.44:0/3424701413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148009560 con 0x7f115c0fee60 2024-09-13T17:27:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1161640640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c19ab10 0x7f115c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1160e3f640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 0x7f115c1961e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f115000d950 tx=0x7f115000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1159ffb640 1 -- 172.21.15.44:0/3424701413 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150009c30 con 0x7f115c0fee60 2024-09-13T17:27:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1159ffb640 1 -- 172.21.15.44:0/3424701413 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1150009d90 con 0x7f115c0fee60 2024-09-13T17:27:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c1a2dc0 con 0x7f115c0fee60 2024-09-13T17:27:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f1159ffb640 1 -- 172.21.15.44:0/3424701413 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000f620 con 0x7f115c0fee60 2024-09-13T17:27:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.679+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1a3230 con 0x7f115c0fee60 2024-09-13T17:27:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.681+0000 7f1159ffb640 1 -- 172.21.15.44:0/3424701413 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f11500109f0 con 0x7f115c0fee60 2024-09-13T17:27:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1159ffb640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f114c0a1430 0x7f114c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 Processor -- start 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 -- start start 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1159ffb640 1 -- 172.21.15.44:0/3424701413 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f11500cce80 con 0x7f115c0fee60 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f115bfff640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f114c0a1430 0x7f114c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80ef5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f04820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f07150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80e5fcc0 con 0x562b80ef5690 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80e5d450 con 0x562b80f04d60 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1165dc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80e58d40 con 0x562b80ef5fb0 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1160e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80ef5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1160e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80ef5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48860/0 (socket says 172.21.15.44:48860) 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.683+0000 7f1160e3f640 1 -- 172.21.15.44:0/2903352944 learned_addr learned my addr 172.21.15.44:0/2903352944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f115bfff640 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f04820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1161640640 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f07150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f115bfff640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f114c0a1430 0x7f114c0a38f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1148005ab0 tx=0x7f1148005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1160e3f640 1 -- 172.21.15.44:0/2903352944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 msgr2=0x562b80f04820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f04820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1160e3f640 1 -- 172.21.15.44:0/2903352944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 msgr2=0x562b80f07150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f07150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1160e3f640 1 -- 172.21.15.44:0/2903352944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b80e20620 con 0x562b80ef5690 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1161640640 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f07150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80ef5a70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1150005fd0 tx=0x7f115009a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa88960d31a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1146ffd640 1 -- 172.21.15.44:0/2903352944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500a1eb0 con 0x562b80ef5690 2024-09-13T17:27:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1146ffd640 1 -- 172.21.15.44:0/2903352944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1150096140 con 0x562b80ef5690 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.684+0000 7f1146ffd640 1 -- 172.21.15.44:0/2903352944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150095a80 con 0x562b80ef5690 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 -- 172.21.15.44:0/2903352944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 msgr2=0x562b80ef5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80ef5a70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1150005fd0 tx=0x7f115009a130 comp rx=0 tx=0).stop 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 -- 172.21.15.44:0/2903352944 shutdown_connections 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f04820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f07150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2903352944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80ef5a70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 -- 172.21.15.44:0/2903352944 >> 172.21.15.44:0/2903352944 conn(0x562b80ef1d00 msgr2=0x562b80ef20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 -- 172.21.15.44:0/2903352944 shutdown_connections 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 -- 172.21.15.44:0/2903352944 wait complete. 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 -- start start 2024-09-13T17:27:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.685+0000 7f1165dc9740 1 Processor -- start 2024-09-13T17:27:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80f04410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80e17180 con 0x562b80ef5690 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80b31430 con 0x562b80f04d60 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80ddc5e0 con 0x562b80ef5fb0 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1165dc9740 10 client.0 _ll_register_callbacks cb 0x562b80e48aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80f04410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80f04410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48866/0 (socket says 172.21.15.44:48866) 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 -- 172.21.15.44:0/2498353133 learned_addr learned my addr 172.21.15.44:0/2498353133 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f115bfff640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1161640640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 -- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 msgr2=0x562b80f017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 -- 172.21.15.44:0/2498353133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 msgr2=0x562b80f01ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f01ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.686+0000 7f1160e3f640 1 -- 172.21.15.44:0/2498353133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1a3230 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80f04410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1150095f30 tx=0x7f11500a1d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985666ab3f85d7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f113bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f113bfff640 1 -- 172.21.15.44:0/2498353133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500a1eb0 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b80e82b80 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f113bfff640 1 -- 172.21.15.44:0/2498353133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f11500b2360 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f113bfff640 1 -- 172.21.15.44:0/2498353133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150095820 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f1165dc9740 10 client.15849 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b80e573c0 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.687+0000 7f113bfff640 1 -- 172.21.15.44:0/2498353133 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562b80e82b80 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f113bfff640 1 -- 172.21.15.44:0/2498353133 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11500ca580 con 0x562b80ef5690 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f1165dc9740 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f1165dc9740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f1165dc9740 20 client.15849 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f142c0 0x562b80f166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f1165dc9740 4 client.15849 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.688+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b80e4b1f0 con 0x562b80f142c0 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.689+0000 7f115bfff640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f142c0 0x562b80f166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.689+0000 7f113bfff640 10 client.15849 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.689+0000 7f115bfff640 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f142c0 0x562b80f166b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63a74d916b399d6 server_cookie=f7aabf4ff7450d4e in_seq=0 out_seq=0 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f113bfff640 1 -- 172.21.15.44:0/2498353133 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562b80e4b1f0 con 0x562b80f142c0 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f113bfff640 10 client.15849 handle_command_reply: tid=0 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 client.15849 shutdown 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 msgr2=0x562b80f04410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80f04410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1150095f30 tx=0x7f11500a1d50 comp rx=0 tx=0).stop 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 shutdown_connections 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f142c0 0x562b80f166b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80f04d60 0x562b80f01ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f113bfff640 0 client.15849 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2498353133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80ef5690 0x562b80f04410 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 >> 172.21.15.44:0/2498353133 conn(0x562b80ef1d00 msgr2=0x562b80ef7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 shutdown_connections 2024-09-13T17:27:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 1 -- 172.21.15.44:0/2498353133 wait complete. 2024-09-13T17:27:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.690+0000 7f1165dc9740 20 client.15849 trim_cache size 0 max 16384 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.701+0000 7f1165dc9740 1 Processor -- start 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.701+0000 7f1165dc9740 1 -- start start 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80ef16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80efced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f0b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1165dc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80efd410 con 0x562b80f04d60 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1165dc9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80ef6740 con 0x562b80ef5690 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1165dc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80ef2e90 con 0x562b80ef5fb0 2024-09-13T17:27:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1161640640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f0b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1161640640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f0b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48874/0 (socket says 172.21.15.44:48874) 2024-09-13T17:27:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1161640640 1 -- 172.21.15.44:0/1992329126 learned_addr learned my addr 172.21.15.44:0/1992329126 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80ef16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f115bfff640 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80efced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 -- 172.21.15.44:0/1992329126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 msgr2=0x562b80efced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80efced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 -- 172.21.15.44:0/1992329126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 msgr2=0x562b80f0b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f0b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 -- 172.21.15.44:0/1992329126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b80df9770 con 0x562b80ef5690 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1161640640 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f0b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.702+0000 7f1160e3f640 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80ef16a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11500ca160 tx=0x7f115009a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44664d9a1f498946 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f11477fe640 1 -- 172.21.15.44:0/1992329126 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150098a60 con 0x562b80ef5690 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f11477fe640 1 -- 172.21.15.44:0/1992329126 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11500b2360 con 0x562b80ef5690 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f11477fe640 1 -- 172.21.15.44:0/1992329126 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500ca380 con 0x562b80ef5690 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 -- 172.21.15.44:0/1992329126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 msgr2=0x562b80ef16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80ef16a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11500ca160 tx=0x7f115009a260 comp rx=0 tx=0).stop 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 -- 172.21.15.44:0/1992329126 shutdown_connections 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80efced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f0b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 --2- 172.21.15.44:0/1992329126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80ef16a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 -- 172.21.15.44:0/1992329126 >> 172.21.15.44:0/1992329126 conn(0x562b80ef1d00 msgr2=0x562b80efc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 -- 172.21.15.44:0/1992329126 shutdown_connections 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.703+0000 7f1165dc9740 1 -- 172.21.15.44:0/1992329126 wait complete. 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 -- start start 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 Processor -- start 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80f0ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f0bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80f08600 con 0x562b80f04d60 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80f08770 con 0x562b80ef5690 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.704+0000 7f1165dc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80f08930 con 0x562b80ef5fb0 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1165dc9740 10 client.0 _ll_register_callbacks cb 0x7f1150009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1161640640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1161640640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f07f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48882/0 (socket says 172.21.15.44:48882) 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1161640640 1 -- 172.21.15.44:0/2527171699 learned_addr learned my addr 172.21.15.44:0/2527171699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f115bfff640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f0bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80f0ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 -- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 msgr2=0x562b80f0bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f0bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 -- 172.21.15.44:0/2527171699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 msgr2=0x562b80f07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f07f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 -- 172.21.15.44:0/2527171699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1154009c40 con 0x562b80ef5690 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1161640640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f1160e3f640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80f0ba10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1150009610 tx=0x7f1150000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9369d457c46d1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f113bfff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.705+0000 7f113bfff640 1 -- 172.21.15.44:0/2527171699 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500a1eb0 con 0x562b80ef5690 2024-09-13T17:27:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f113bfff640 1 -- 172.21.15.44:0/2527171699 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1150095820 con 0x562b80ef5690 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1150096140 con 0x562b80ef5690 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f113bfff640 1 -- 172.21.15.44:0/2527171699 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150095a20 con 0x562b80ef5690 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 10 client.25442 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b80e573c0 con 0x562b80ef5690 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f113bfff640 1 -- 172.21.15.44:0/2527171699 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1150096140 con 0x562b80ef5690 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f113bfff640 1 -- 172.21.15.44:0/2527171699 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1150010460 con 0x562b80ef5690 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 10 client.25442 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 10 client.25442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 20 client.25442 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f6b5b0 0x562b80f6d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 4 client.25442 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:27:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b80b4c4e0 con 0x562b80f6b5b0 2024-09-13T17:27:21.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.706+0000 7f115bfff640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f6b5b0 0x562b80f6d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f115bfff640 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f6b5b0 0x562b80f6d9a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9451a59aa75a51d7 server_cookie=398b95888fded148 in_seq=0 out_seq=0 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f113bfff640 10 client.25442 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f113bfff640 1 -- 172.21.15.44:0/2527171699 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b80b4c4e0 con 0x562b80f6b5b0 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f113bfff640 10 client.25442 handle_command_reply: tid=0 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 client.25442 shutdown 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 msgr2=0x562b80f0ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80f0ba10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1150009610 tx=0x7f1150000bc0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 shutdown_connections 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b80f6b5b0 0x562b80f6d9a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b80ef5fb0 0x562b80f0bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b80f04d60 0x562b80f07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f113bfff640 0 client.25442 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 --2- 172.21.15.44:0/2527171699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b80ef5690 0x562b80f0ba10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.707+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 >> 172.21.15.44:0/2527171699 conn(0x562b80ef1d00 msgr2=0x562b80ef71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 shutdown_connections 2024-09-13T17:27:21.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f1165dc9740 1 -- 172.21.15.44:0/2527171699 wait complete. 2024-09-13T17:27:21.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f1165dc9740 20 client.25442 trim_cache size 0 max 16384 2024-09-13T17:27:21.715 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f114c0a1430 msgr2=0x7f114c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f11630a3640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f114c0a1430 0x7f114c0a38f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1148005ab0 tx=0x7f1148005a20 comp rx=0 tx=0).stop 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 msgr2=0x7f115c1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.708+0000 7f11630a3640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 0x7f115c1961e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f115000d950 tx=0x7f115000de10 comp rx=0 tx=0).stop 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 shutdown_connections 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f114c0a1430 0x7f114c0a38f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f115c19ab10 0x7f115c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f115c103fb0 0x7f115c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 --2- 172.21.15.44:0/3424701413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c0fee60 0x7f115c1961e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 >> 172.21.15.44:0/3424701413 conn(0x7f115c075c60 msgr2=0x7f115c0677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:21.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 shutdown_connections 2024-09-13T17:27:21.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:21.709+0000 7f11630a3640 1 -- 172.21.15.44:0/3424701413 wait complete. 2024-09-13T17:27:22.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:22 smithi070 ceph-mon[34502]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-13T17:27:22.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:22 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1423911338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:27:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:22 smithi100 ceph-mon[35055]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-13T17:27:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:22 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1423911338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:27:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:22 smithi044 ceph-mon[31175]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-13T17:27:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:22 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1423911338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:27:22.726 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:27:22.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 -- 172.21.15.44:0/847813601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c0ff810 msgr2=0x7f866c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 --2- 172.21.15.44:0/847813601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c0ff810 0x7f866c103850 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8654009930 tx=0x7f865402a420 comp rx=0 tx=0).stop 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 -- 172.21.15.44:0/847813601 shutdown_connections 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 --2- 172.21.15.44:0/847813601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f866c103fb0 0x7f866c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 --2- 172.21.15.44:0/847813601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c0ff810 0x7f866c103850 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 --2- 172.21.15.44:0/847813601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fee60 0x7f866c0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.868+0000 7f8671fd4640 1 -- 172.21.15.44:0/847813601 >> 172.21.15.44:0/847813601 conn(0x7f866c075c60 msgr2=0x7f866c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.869+0000 7f8671fd4640 1 -- 172.21.15.44:0/847813601 shutdown_connections 2024-09-13T17:27:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.869+0000 7f8671fd4640 1 -- 172.21.15.44:0/847813601 wait complete. 2024-09-13T17:27:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.869+0000 7f8671fd4640 1 Processor -- start 2024-09-13T17:27:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.869+0000 7f8671fd4640 1 -- start start 2024-09-13T17:27:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8671fd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f866c0fee60 0x7f866c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8671fd4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 0x7f866c197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8671fd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c197a70 0x7f866c1a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8671fd4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c1a2710 con 0x7f866c103fb0 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8671fd4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c1a2880 con 0x7f866c0fee60 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8671fd4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c1a2a40 con 0x7f866c197a70 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 0x7f866c197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 0x7f866c197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48912/0 (socket says 172.21.15.44:48912) 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 -- 172.21.15.44:0/2983565657 learned_addr learned my addr 172.21.15.44:0/2983565657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f866bfff640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c197a70 0x7f866c1a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f866b7fe640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f866c0fee60 0x7f866c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 -- 172.21.15.44:0/2983565657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c197a70 msgr2=0x7f866c1a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c197a70 0x7f866c1a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 -- 172.21.15.44:0/2983565657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f866c0fee60 msgr2=0x7f866c196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f866c0fee60 0x7f866c196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.870+0000 7f8663fff640 1 -- 172.21.15.44:0/2983565657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009560 con 0x7f866c103fb0 2024-09-13T17:27:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.871+0000 7f8663fff640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 0x7f866c197530 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8654005ab0 tx=0x7f8654032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.871+0000 7f86697fa640 1 -- 172.21.15.44:0/2983565657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540044d0 con 0x7f866c103fb0 2024-09-13T17:27:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.871+0000 7f86697fa640 1 -- 172.21.15.44:0/2983565657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8654032e10 con 0x7f866c103fb0 2024-09-13T17:27:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.871+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c1a2cc0 con 0x7f866c103fb0 2024-09-13T17:27:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.871+0000 7f86697fa640 1 -- 172.21.15.44:0/2983565657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654003850 con 0x7f866c103fb0 2024-09-13T17:27:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.871+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c1a31c0 con 0x7f866c103fb0 2024-09-13T17:27:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 Processor -- start 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 -- start start 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f86697fa640 1 -- 172.21.15.44:0/2983565657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f8654004850 con 0x7f866c103fb0 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedbeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedcbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbed29c60 con 0x55cdbedbf0e0 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbed273f0 con 0x55cdbedbe7c0 2024-09-13T17:27:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.874+0000 7f8674dd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbed22ce0 con 0x55cdbedc9980 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48920/0 (socket says 172.21.15.44:48920) 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedbeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 -- 172.21.15.44:0/3291890367 learned_addr learned my addr 172.21.15.44:0/3291890367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f8663fff640 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedcbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 -- 172.21.15.44:0/3291890367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 msgr2=0x55cdbedcbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedcbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 -- 172.21.15.44:0/3291890367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 msgr2=0x55cdbedbeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedbeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 -- 172.21.15.44:0/3291890367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbecde970 con 0x55cdbedbf0e0 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f8663fff640 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedcbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f866b7fe640 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc9440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f865c00bcd0 tx=0x7f865c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff53c48fabd8d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.875+0000 7f86697fa640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f86580a5840 0x7f86580a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f8661ffb640 1 -- 172.21.15.44:0/3291890367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c005220 con 0x55cdbedbf0e0 2024-09-13T17:27:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f86697fa640 1 -- 172.21.15.44:0/2983565657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f86540ee0b0 con 0x7f866c103fb0 2024-09-13T17:27:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f8661ffb640 1 -- 172.21.15.44:0/3291890367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f865c015070 con 0x55cdbedbf0e0 2024-09-13T17:27:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f8661ffb640 1 -- 172.21.15.44:0/3291890367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c011440 con 0x55cdbedbf0e0 2024-09-13T17:27:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f866bfff640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f86580a5840 0x7f86580a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f8674dd3740 1 -- 172.21.15.44:0/3291890367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 msgr2=0x55cdbedc9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f8674dd3740 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc9440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f865c00bcd0 tx=0x7f865c0107a0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.876+0000 7f866bfff640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f86580a5840 0x7f86580a7d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f8664009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 -- 172.21.15.44:0/3291890367 shutdown_connections 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedcbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc9440 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 --2- 172.21.15.44:0/3291890367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedbeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 -- 172.21.15.44:0/3291890367 >> 172.21.15.44:0/3291890367 conn(0x55cdbedbad50 msgr2=0x55cdbedbb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 -- 172.21.15.44:0/3291890367 shutdown_connections 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 -- 172.21.15.44:0/3291890367 wait complete. 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 -- start start 2024-09-13T17:27:22.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.877+0000 7f8674dd3740 1 Processor -- start 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 0x55cdbedc6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbece0f80 con 0x55cdbedc9980 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbece0c20 con 0x55cdbedbe7c0 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbecadab0 con 0x55cdbedbf0e0 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8674dd3740 10 client.0 _ll_register_callbacks cb 0x55cdbed1c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 0x55cdbedc6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 0x55cdbedc6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48924/0 (socket says 172.21.15.44:48924) 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866bfff640 1 -- 172.21.15.44:0/2742453143 learned_addr learned my addr 172.21.15.44:0/2742453143 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866b7fe640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f8663fff640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866b7fe640 1 -- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbf0e0 msgr2=0x55cdbedc66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866b7fe640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866b7fe640 1 -- 172.21.15.44:0/2742453143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 msgr2=0x55cdbedc6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866b7fe640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 0x55cdbedc6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866b7fe640 1 -- 172.21.15.44:0/2742453143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c00b900 con 0x55cdbedbe7c0 2024-09-13T17:27:22.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.878+0000 7f866bfff640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 0x55cdbedc6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f866b7fe640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc9420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cab2fcbe2f3a137 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f864affd640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f864affd640 1 -- 172.21.15.44:0/2742453143 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0048c0 con 0x55cdbedbe7c0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdbea16590 con 0x55cdbedbe7c0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f864affd640 1 -- 172.21.15.44:0/2742453143 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f865c017070 con 0x55cdbedbe7c0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f864affd640 1 -- 172.21.15.44:0/2742453143 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865c004a20 con 0x55cdbedbe7c0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f8674dd3740 10 client.25454 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cdbed21360 con 0x55cdbedbe7c0 2024-09-13T17:27:22.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f864affd640 1 -- 172.21.15.44:0/2742453143 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cdbea16590 con 0x55cdbedbe7c0 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f864affd640 1 -- 172.21.15.44:0/2742453143 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f865c011cc0 con 0x55cdbedbe7c0 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.879+0000 7f8674dd3740 10 client.25454 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8674dd3740 10 client.25454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8674dd3740 20 client.25454 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8674dd3740 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbedd9680 0x55cdbeddba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8674dd3740 4 client.25454 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdbea27860 con 0x55cdbedd9680 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8663fff640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbedd9680 0x55cdbeddba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f864affd640 10 client.25454 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.880+0000 7f8663fff640 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbedd9680 0x55cdbeddba70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693541fcaa2afca6 server_cookie=f08fe6f2accaa27d in_seq=0 out_seq=0 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f864affd640 1 -- 172.21.15.44:0/2742453143 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cdbea27860 con 0x55cdbedd9680 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f864affd640 10 client.25454 handle_command_reply: tid=0 2024-09-13T17:27:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 client.25454 shutdown 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 msgr2=0x55cdbedc9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc9420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c004e90 comp rx=0 tx=0).stop 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 shutdown_connections 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbedd9680 0x55cdbeddba70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbf0e0 0x55cdbedc66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f864affd640 0 client.25454 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedc9980 0x55cdbedc6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 --2- 172.21.15.44:0/2742453143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc9420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.881+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 >> 172.21.15.44:0/2742453143 conn(0x55cdbedbad50 msgr2=0x55cdbedc01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.882+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 shutdown_connections 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.882+0000 7f8674dd3740 1 -- 172.21.15.44:0/2742453143 wait complete. 2024-09-13T17:27:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.882+0000 7f8674dd3740 20 client.25454 trim_cache size 0 max 16384 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 Processor -- start 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 -- start start 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbf0e0 0x55cdbedd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedd0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbedacef0 con 0x55cdbedbe7c0 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbedbba40 con 0x55cdbedbf0e0 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8674dd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbedc4bf0 con 0x55cdbedc9980 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48932/0 (socket says 172.21.15.44:48932) 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 -- 172.21.15.44:0/354857310 learned_addr learned my addr 172.21.15.44:0/354857310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f8663fff640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbf0e0 0x55cdbedd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866bfff640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedd0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 -- 172.21.15.44:0/354857310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 msgr2=0x55cdbedd0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedd0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 -- 172.21.15.44:0/354857310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbf0e0 msgr2=0x55cdbedd0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbf0e0 0x55cdbedd0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.896+0000 7f866b7fe640 1 -- 172.21.15.44:0/354857310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbecc31c0 con 0x55cdbedbe7c0 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.897+0000 7f8663fff640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbf0e0 0x55cdbedd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.897+0000 7f866bfff640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedd0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.897+0000 7f866b7fe640 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc8f30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f865c019800 tx=0x7f865c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9ee35f7247fa31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.897+0000 7f86627fc640 1 -- 172.21.15.44:0/354857310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c005890 con 0x55cdbedbe7c0 2024-09-13T17:27:22.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.897+0000 7f86627fc640 1 -- 172.21.15.44:0/354857310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f865c020040 con 0x55cdbedbe7c0 2024-09-13T17:27:22.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.897+0000 7f86627fc640 1 -- 172.21.15.44:0/354857310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c019330 con 0x55cdbedbe7c0 2024-09-13T17:27:22.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 -- 172.21.15.44:0/354857310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 msgr2=0x55cdbedc8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc8f30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f865c019800 tx=0x7f865c00bfd0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 -- 172.21.15.44:0/354857310 shutdown_connections 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedc9980 0x55cdbedd0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedbf0e0 0x55cdbedd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 --2- 172.21.15.44:0/354857310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbe7c0 0x55cdbedc8f30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 -- 172.21.15.44:0/354857310 >> 172.21.15.44:0/354857310 conn(0x55cdbedbad50 msgr2=0x55cdbede0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 -- 172.21.15.44:0/354857310 shutdown_connections 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 -- 172.21.15.44:0/354857310 wait complete. 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.898+0000 7f8674dd3740 1 -- start start 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 Processor -- start 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbe7c0 0x55cdbedced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedcf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8663fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedcf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8663fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedcf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48948/0 (socket says 172.21.15.44:48948) 2024-09-13T17:27:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbe7c0 0x55cdbedced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedc9980 0x55cdbedd00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8663fff640 1 -- 172.21.15.44:0/1169524204 learned_addr learned my addr 172.21.15.44:0/1169524204 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbedd0680 con 0x55cdbedbf0e0 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbedd07f0 con 0x55cdbedc9980 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbedd09b0 con 0x55cdbedbe7c0 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.899+0000 7f8674dd3740 10 client.0 _ll_register_callbacks cb 0x7f8664009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8663fff640 1 -- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbe7c0 msgr2=0x55cdbedced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8663fff640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbe7c0 0x55cdbedced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f866bfff640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedc9980 0x55cdbedd00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8663fff640 1 -- 172.21.15.44:0/1169524204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedc9980 msgr2=0x55cdbedd00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8663fff640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedc9980 0x55cdbedd00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8663fff640 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdbed21360 con 0x55cdbedbf0e0 2024-09-13T17:27:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f866b7fe640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbe7c0 0x55cdbedced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f865c015070 con 0x55cdbedbf0e0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8663fff640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedcf270 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f86540b78a0 tx=0x7f865402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d1fdf765aa6b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f864affd640 10 client.15873 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f864affd640 1 -- 172.21.15.44:0/1169524204 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540bae30 con 0x55cdbedbf0e0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f864affd640 1 -- 172.21.15.44:0/1169524204 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f86540bebb0 con 0x55cdbedbf0e0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f864affd640 1 -- 172.21.15.44:0/1169524204 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86540c3200 con 0x55cdbedbf0e0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8674dd3740 10 client.15873 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.900+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55cdbece8990 con 0x55cdbedbf0e0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f864affd640 1 -- 172.21.15.44:0/1169524204 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f865404d070 con 0x55cdbedbf0e0 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f8674dd3740 10 client.15873 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f8674dd3740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f8674dd3740 20 client.15873 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f8674dd3740 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbee30900 0x55cdbee32d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f8674dd3740 4 client.15873 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cdbea7aeb0 con 0x55cdbee30900 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f866b7fe640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbee30900 0x55cdbee32d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f866b7fe640 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbee30900 0x55cdbee32d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffb4ebdf7f0b170 server_cookie=2a3688079b5fd744 in_seq=0 out_seq=0 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.901+0000 7f864affd640 10 client.15873 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.902+0000 7f864affd640 1 -- 172.21.15.44:0/1169524204 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cdbea7aeb0 con 0x55cdbee30900 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.902+0000 7f864affd640 10 client.15873 handle_command_reply: tid=0 2024-09-13T17:27:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.902+0000 7f8674dd3740 1 client.15873 shutdown 2024-09-13T17:27:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 msgr2=0x55cdbedcf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedcf270 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f86540b78a0 tx=0x7f865402a420 comp rx=0 tx=0).stop 2024-09-13T17:27:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 shutdown_connections 2024-09-13T17:27:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cdbee30900 0x55cdbee32d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cdbedc9980 0x55cdbedd00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cdbedbf0e0 0x55cdbedcf270 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 --2- 172.21.15.44:0/1169524204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdbedbe7c0 0x55cdbedced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 >> 172.21.15.44:0/1169524204 conn(0x55cdbedbad50 msgr2=0x55cdbedc1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f864affd640 0 client.15873 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 shutdown_connections 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 1 -- 172.21.15.44:0/1169524204 wait complete. 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.903+0000 7f8674dd3740 20 client.15873 trim_cache size 0 max 16384 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:27:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:27:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "c4c07fd1-768d-4aef-b69e-19c56cf8f205", 2024-09-13T17:27:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:27:22.911 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:27:22.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f86580a5840 msgr2=0x7f86580a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f86580a5840 0x7f86580a7d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f8664009500 comp rx=0 tx=0).stop 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 msgr2=0x7f866c197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 0x7f866c197530 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8654005ab0 tx=0x7f8654032c20 comp rx=0 tx=0).stop 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 shutdown_connections 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f86580a5840 0x7f86580a7d00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c197a70 0x7f866c1a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f866c103fb0 0x7f866c197530 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 --2- 172.21.15.44:0/2983565657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f866c0fee60 0x7f866c196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 >> 172.21.15.44:0/2983565657 conn(0x7f866c075c60 msgr2=0x7f866c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 shutdown_connections 2024-09-13T17:27:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:22.904+0000 7f8671fd4640 1 -- 172.21.15.44:0/2983565657 wait complete. 2024-09-13T17:27:22.917 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:27:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/1929085361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc41008f0 msgr2=0x7f1dc4104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/1929085361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc41008f0 0x7f1dc4104930 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009930 tx=0x7f1db402a420 comp rx=0 tx=0).stop 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/1929085361 shutdown_connections 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/1929085361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4105090 0x7f1dc410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/1929085361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc41008f0 0x7f1dc4104930 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/1929085361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dc40fff40 0x7f1dc4100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/1929085361 >> 172.21.15.44:0/1929085361 conn(0x7f1dc40fb9f0 msgr2=0x7f1dc40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/1929085361 shutdown_connections 2024-09-13T17:27:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.021+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/1929085361 wait complete. 2024-09-13T17:27:23.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.022+0000 7f1dcc6b1640 1 Processor -- start 2024-09-13T17:27:23.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.022+0000 7f1dcc6b1640 1 -- start start 2024-09-13T17:27:23.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.022+0000 7f1dcc6b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc40fff40 0x7f1dc40727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.022+0000 7f1dcc6b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 0x7f1dc4072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.022+0000 7f1dcc6b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dc4073230 0x7f1dc41a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.022+0000 7f1dc9c4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 0x7f1dc4072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dcc6b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc41a49a0 con 0x7f1dc4105090 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dcc6b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc41a4b10 con 0x7f1dc4073230 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dcc6b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc41a4cd0 con 0x7f1dc40fff40 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dc9c4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 0x7f1dc4072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48960/0 (socket says 172.21.15.44:48960) 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/871979630 learned_addr learned my addr 172.21.15.44:0/871979630 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc40fff40 0x7f1dc40727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dc4073230 0x7f1dc41a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 -- 172.21.15.44:0/871979630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dc4073230 msgr2=0x7f1dc41a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dc4073230 0x7f1dc41a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 -- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 msgr2=0x7f1dc4072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 0x7f1dc4072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 -- 172.21.15.44:0/871979630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009490 con 0x7f1dc40fff40 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 0x7f1dc4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dca44d640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc40fff40 0x7f1dc40727b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e9f0 tx=0x7f1dc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.023+0000 7f1dbb7fe640 1 -- 172.21.15.44:0/871979630 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000cd80 con 0x7f1dc40fff40 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.024+0000 7f1dbb7fe640 1 -- 172.21.15.44:0/871979630 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dc00044b0 con 0x7f1dc40fff40 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.024+0000 7f1dbb7fe640 1 -- 172.21.15.44:0/871979630 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0010430 con 0x7f1dc40fff40 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.024+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db4009560 con 0x7f1dc40fff40 2024-09-13T17:27:23.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.024+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc41a51f0 con 0x7f1dc40fff40 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.026+0000 7f1dcf3d5740 1 Processor -- start 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.026+0000 7f1dcf3d5740 1 -- start start 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1291f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af129730 0x55b9af133a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af133fd0 0x55b9af1363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af093f20 con 0x55b9af128e10 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af0916b0 con 0x55b9af133fd0 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af08cfa0 con 0x55b9af129730 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1291f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1291f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48972/0 (socket says 172.21.15.44:48972) 2024-09-13T17:27:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 -- 172.21.15.44:0/83375505 learned_addr learned my addr 172.21.15.44:0/83375505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af133fd0 0x55b9af1363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dbb7fe640 1 -- 172.21.15.44:0/871979630 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f1dc0003ef0 con 0x7f1dc40fff40 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 -- 172.21.15.44:0/83375505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af129730 msgr2=0x55b9af133a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af129730 0x55b9af133a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 -- 172.21.15.44:0/83375505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af133fd0 msgr2=0x55b9af1363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af133fd0 0x55b9af1363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dcac4e640 1 -- 172.21.15.44:0/83375505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9af047840 con 0x55b9af128e10 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.027+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af133fd0 0x55b9af1363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.028+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1291f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009f50 tx=0x7f1dbc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca4d86802651b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.028+0000 7f1db8ff9640 1 -- 172.21.15.44:0/83375505 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc004d20 con 0x55b9af128e10 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.028+0000 7f1db8ff9640 1 -- 172.21.15.44:0/83375505 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dbc004e80 con 0x55b9af128e10 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.028+0000 7f1db8ff9640 1 -- 172.21.15.44:0/83375505 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc005020 con 0x55b9af128e10 2024-09-13T17:27:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.028+0000 7f1dbb7fe640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1db00a13e0 0x7f1db00a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dca44d640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1db00a13e0 0x7f1db00a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dbb7fe640 1 -- 172.21.15.44:0/871979630 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1dc0014070 con 0x7f1dc40fff40 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/83375505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 msgr2=0x55b9af1291f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1291f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009f50 tx=0x7f1dbc0118d0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/83375505 shutdown_connections 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af129730 0x55b9af133a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af133fd0 0x55b9af1363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/83375505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1291f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/83375505 >> 172.21.15.44:0/83375505 conn(0x55b9af125380 msgr2=0x55b9af125760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dca44d640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1db00a13e0 0x7f1db00a38a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00ca910 tx=0x7f1dc00cd740 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/83375505 shutdown_connections 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.029+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/83375505 wait complete. 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 -- start start 2024-09-13T17:27:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 Processor -- start 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af130c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af133fd0 0x55b9af1311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af02e2a0 con 0x55b9af128e10 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9aed65430 con 0x55b9af129730 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af010680 con 0x55b9af133fd0 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcf3d5740 10 client.0 _ll_register_callbacks cb 0x55b9af086610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1339d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48984/0 (socket says 172.21.15.44:48984) 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.030+0000 7f1dcac4e640 1 -- 172.21.15.44:0/2095491661 learned_addr learned my addr 172.21.15.44:0/2095491661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af133fd0 0x55b9af1311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 -- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af133fd0 msgr2=0x55b9af1311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af133fd0 0x55b9af1311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 -- 172.21.15.44:0/2095491661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 msgr2=0x55b9af130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 -- 172.21.15.44:0/2095491661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc009c00 con 0x55b9af128e10 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1339d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005c40 tx=0x7f1dbc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e045976a70b8ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dadffb640 1 -- 172.21.15.44:0/2095491661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc005200 con 0x55b9af128e10 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dadffb640 1 -- 172.21.15.44:0/2095491661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1dbc005360 con 0x55b9af128e10 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.031+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9aed917e0 con 0x55b9af128e10 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dadffb640 1 -- 172.21.15.44:0/2095491661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dbc005520 con 0x55b9af128e10 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 10 client.15885 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b9af08b620 con 0x55b9af128e10 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dadffb640 1 -- 172.21.15.44:0/2095491661 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b9aed917e0 con 0x55b9af128e10 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dadffb640 1 -- 172.21.15.44:0/2095491661 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1dbc028ab0 con 0x55b9af128e10 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 10 client.15885 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 20 client.15885 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af146030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 4 client.15885 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.032+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9aede4e30 con 0x55b9af143c40 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.033+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.033+0000 7f1dadffb640 10 client.15885 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.033+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af146030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46aad33ad7794d7e server_cookie=b997fe2ced034b92 in_seq=0 out_seq=0 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dadffb640 1 -- 172.21.15.44:0/2095491661 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b9aede4e30 con 0x55b9af143c40 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dadffb640 10 client.15885 handle_command_reply: tid=0 2024-09-13T17:27:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 client.15885 shutdown 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 msgr2=0x55b9af1339d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1339d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005c40 tx=0x7f1dbc005bd0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 shutdown_connections 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af146030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af133fd0 0x55b9af1311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dadffb640 0 client.15885 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/2095491661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af128e10 0x55b9af1339d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 >> 172.21.15.44:0/2095491661 conn(0x55b9af125380 msgr2=0x55b9af12b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 shutdown_connections 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/2095491661 wait complete. 2024-09-13T17:27:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.034+0000 7f1dcf3d5740 20 client.15885 trim_cache size 0 max 16384 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 Processor -- start 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 -- start start 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af1334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af12f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af12f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af140b00 con 0x55b9af133fd0 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af128b90 con 0x55b9af129730 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.063+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af12ff30 con 0x55b9af128e10 2024-09-13T17:27:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af12f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af12f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48990/0 (socket says 172.21.15.44:48990) 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dcac4e640 1 -- 172.21.15.44:0/286213208 learned_addr learned my addr 172.21.15.44:0/286213208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af12f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af1334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/286213208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 msgr2=0x55b9af1334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af1334e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/286213208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 msgr2=0x55b9af12f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af12f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/286213208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9af02d790 con 0x55b9af129730 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af12f4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4003a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3ab34026b3edef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.064+0000 7f1db97fa640 1 -- 172.21.15.44:0/286213208 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4042040 con 0x55b9af129730 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1db97fa640 1 -- 172.21.15.44:0/286213208 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1db40467c0 con 0x55b9af129730 2024-09-13T17:27:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1db97fa640 1 -- 172.21.15.44:0/286213208 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4046a90 con 0x55b9af129730 2024-09-13T17:27:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/286213208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 msgr2=0x55b9af12f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af12f4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4003a60 comp rx=0 tx=0).stop 2024-09-13T17:27:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/286213208 shutdown_connections 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af12f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af12f4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.065+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/286213208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af1334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/286213208 >> 172.21.15.44:0/286213208 conn(0x55b9af125380 msgr2=0x55b9af12be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/286213208 shutdown_connections 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/286213208 wait complete. 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- start start 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 Processor -- start 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af139400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af1995f0 con 0x55b9af133fd0 2024-09-13T17:27:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af199730 con 0x55b9af129730 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9af199870 con 0x55b9af128e10 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcf3d5740 10 client.0 _ll_register_callbacks cb 0x7f1dbc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.066+0000 7f1dcac4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49004/0 (socket says 172.21.15.44:49004) 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dcac4e640 1 -- 172.21.15.44:0/1580770060 learned_addr learned my addr 172.21.15.44:0/1580770060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 msgr2=0x55b9af138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af138ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/1580770060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 msgr2=0x55b9af1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af1990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 -- 172.21.15.44:0/1580770060 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db40037d0 con 0x55b9af129730 2024-09-13T17:27:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dc9c4c640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af139400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b9af119c80 tx=0x7f1db40035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77aad0390023b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dadffb640 1 -- 172.21.15.44:0/1580770060 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4042040 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9aed804e0 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.067+0000 7f1dadffb640 1 -- 172.21.15.44:0/1580770060 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1db4003de0 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dadffb640 1 -- 172.21.15.44:0/1580770060 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1db4046330 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 10 client.25472 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b9af052ea0 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dadffb640 1 -- 172.21.15.44:0/1580770060 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b9aed804e0 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dadffb640 1 -- 172.21.15.44:0/1580770060 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1db404f410 con 0x55b9af129730 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 10 client.25472 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 20 client.25472 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 4 client.25472 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.068+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b9af19d520 con 0x55b9af143c40 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcac4e640 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af19cd30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbea5ed64049fb73 server_cookie=51e2b418adb1f32c in_seq=0 out_seq=0 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dadffb640 10 client.25472 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dadffb640 1 -- 172.21.15.44:0/1580770060 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b9af19d520 con 0x55b9af143c40 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dadffb640 10 client.25472 handle_command_reply: tid=0 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcf3d5740 1 client.25472 shutdown 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 msgr2=0x55b9af139400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af139400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b9af119c80 tx=0x7f1db40035f0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 shutdown_connections 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.069+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b9af143c40 0x55b9af19cd30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b9af133fd0 0x55b9af1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9af129730 0x55b9af139400 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dadffb640 0 client.25472 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 1 --2- 172.21.15.44:0/1580770060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9af128e10 0x55b9af138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 >> 172.21.15.44:0/1580770060 conn(0x55b9af125380 msgr2=0x55b9af12f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 shutdown_connections 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 1 -- 172.21.15.44:0/1580770060 wait complete. 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcf3d5740 20 client.25472 trim_cache size 0 max 16384 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:27:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "c4c07fd1-768d-4aef-b69e-19c56cf8f205": { 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "c4c07fd1-768d-4aef-b69e-19c56cf8f205", 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:27:23.078 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1db00a13e0 msgr2=0x7f1db00a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1db00a13e0 0x7f1db00a38a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00ca910 tx=0x7f1dc00cd740 comp rx=0 tx=0).stop 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc40fff40 msgr2=0x7f1dc40727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc40fff40 0x7f1dc40727b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e9f0 tx=0x7f1dc000eeb0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 shutdown_connections 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1db00a13e0 0x7f1db00a38a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dc4073230 0x7f1dc41a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4105090 0x7f1dc4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 --2- 172.21.15.44:0/871979630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc40fff40 0x7f1dc40727b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.070+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 >> 172.21.15.44:0/871979630 conn(0x7f1dc40fb9f0 msgr2=0x7f1dc40fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.071+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 shutdown_connections 2024-09-13T17:27:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:23.071+0000 7f1dcc6b1640 1 -- 172.21.15.44:0/871979630 wait complete. 2024-09-13T17:27:23.083 INFO:tasks.cephfs.filesystem:scrub status for tag:c4c07fd1-768d-4aef-b69e-19c56cf8f205 - {'path': '/', 'tag': 'c4c07fd1-768d-4aef-b69e-19c56cf8f205', 'options': 'recursive,force'} 2024-09-13T17:27:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:24 smithi070 ceph-mon[34502]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:27:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:24 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:27:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:24 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:24 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:27:24.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:24 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:24.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:24 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:27:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:24 smithi100 ceph-mon[35055]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:27:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:24 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:27:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:24 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:24 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:27:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:24 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:24.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:24 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:27:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:24 smithi044 ceph-mon[31175]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:27:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:24 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:27:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:24 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:24 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:27:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:24 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:24 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:27:26.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:26 smithi070 ceph-mon[34502]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T17:27:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:26 smithi100 ceph-mon[35055]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T17:27:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:26 smithi044 ceph-mon[31175]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T17:27:28.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:28 smithi070 ceph-mon[34502]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:28 smithi100 ceph-mon[35055]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:28 smithi044 ceph-mon[31175]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:30 smithi070 ceph-mon[34502]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:30 smithi100 ceph-mon[35055]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:30.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:30 smithi044 ceph-mon[31175]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 -- 172.21.15.44:0/3645947840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840ff970 msgr2=0x7f158410ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 --2- 172.21.15.44:0/3645947840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840ff970 0x7f158410ba40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f157c009cd0 tx=0x7f157c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 -- 172.21.15.44:0/3645947840 shutdown_connections 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 --2- 172.21.15.44:0/3645947840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15840feff0 0x7f15840ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 --2- 172.21.15.44:0/3645947840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840ff970 0x7f158410ba40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 --2- 172.21.15.44:0/3645947840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1584104c30 0x7f1584105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 -- 172.21.15.44:0/3645947840 >> 172.21.15.44:0/3645947840 conn(0x7f1584074940 msgr2=0x7f1584074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 -- 172.21.15.44:0/3645947840 shutdown_connections 2024-09-13T17:27:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.226+0000 7f158b75e640 1 -- 172.21.15.44:0/3645947840 wait complete. 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.227+0000 7f158b75e640 1 Processor -- start 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.227+0000 7f158b75e640 1 -- start start 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.227+0000 7f158b75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 0x7f1584194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.227+0000 7f158b75e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15840ff970 0x7f1584195300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 0x7f1584194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 0x7f1584194dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55042/0 (socket says 172.21.15.44:55042) 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f158b75e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1584104c30 0x7f1584195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 -- 172.21.15.44:0/2180566971 learned_addr learned my addr 172.21.15.44:0/2180566971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841a4500 con 0x7f15840feff0 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841a4670 con 0x7f15840ff970 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841a4830 con 0x7f1584104c30 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1584104c30 0x7f1584195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f1588cf9640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15840ff970 0x7f1584195300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 -- 172.21.15.44:0/2180566971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1584104c30 msgr2=0x7f1584195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1584104c30 0x7f1584195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 -- 172.21.15.44:0/2180566971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15840ff970 msgr2=0x7f1584195300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15840ff970 0x7f1584195300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 -- 172.21.15.44:0/2180566971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c009900 con 0x7f15840feff0 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f1588cf9640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15840ff970 0x7f1584195300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.228+0000 7f15894fa640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 0x7f1584194dc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f158000d950 tx=0x7f158000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.229+0000 7f157a7fc640 1 -- 172.21.15.44:0/2180566971 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580009a30 con 0x7f15840feff0 2024-09-13T17:27:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.229+0000 7f157a7fc640 1 -- 172.21.15.44:0/2180566971 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15800044b0 con 0x7f15840feff0 2024-09-13T17:27:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.229+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15841a4b10 con 0x7f15840feff0 2024-09-13T17:27:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.229+0000 7f157a7fc640 1 -- 172.21.15.44:0/2180566971 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800047a0 con 0x7f15840feff0 2024-09-13T17:27:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.229+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15841a4fe0 con 0x7f15840feff0 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.232+0000 7f158e47f740 1 Processor -- start 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 -- start start 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f157a7fc640 1 -- 172.21.15.44:0/2180566971 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f1580004020 con 0x7f15840feff0 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e862710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e86cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e86f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e7cdc30 con 0x55861e862330 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e7cb3c0 con 0x55861e86d4f0 2024-09-13T17:27:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f158e47f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e7c6cb0 con 0x55861e862c50 2024-09-13T17:27:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f1588cf9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e86f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f1589cfb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e862710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f1588cf9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e86f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54670/0 (socket says 172.21.15.44:54670) 2024-09-13T17:27:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f1588cf9640 1 -- 172.21.15.44:0/3035673724 learned_addr learned my addr 172.21.15.44:0/3035673724 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.233+0000 7f15894fa640 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e86cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1588cf9640 1 -- 172.21.15.44:0/3035673724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 msgr2=0x55861e86cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1588cf9640 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e86cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1588cf9640 1 -- 172.21.15.44:0/3035673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 msgr2=0x55861e862710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1588cf9640 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e862710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1588cf9640 1 -- 172.21.15.44:0/3035673724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55861e76d910 con 0x55861e86d4f0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1589cfb640 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e862710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f1588cf9640 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e86f8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f157400bb40 tx=0x7f157400d340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaacbfa43c041442 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f157a7fc640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f15700a1430 0x7f15700a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.234+0000 7f156f7fe640 1 -- 172.21.15.44:0/3035673724 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574017420 con 0x55861e86d4f0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.235+0000 7f156f7fe640 1 -- 172.21.15.44:0/3035673724 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1574017a10 con 0x55861e86d4f0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.235+0000 7f157a7fc640 1 -- 172.21.15.44:0/2180566971 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f15800ccbd0 con 0x7f15840feff0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.235+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f15700a1430 0x7f15700a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.235+0000 7f156f7fe640 1 -- 172.21.15.44:0/3035673724 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157400f9b0 con 0x55861e86d4f0 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.235+0000 7f158e47f740 1 -- 172.21.15.44:0/3035673724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 msgr2=0x55861e86f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.235+0000 7f158e47f740 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e86f8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f157400bb40 tx=0x7f157400d340 comp rx=0 tx=0).stop 2024-09-13T17:27:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f15700a1430 0x7f15700a38f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f157c004af0 tx=0x7f157c005d80 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 -- 172.21.15.44:0/3035673724 shutdown_connections 2024-09-13T17:27:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e86cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e86f8e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 --2- 172.21.15.44:0/3035673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e862710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 -- 172.21.15.44:0/3035673724 >> 172.21.15.44:0/3035673724 conn(0x55861e85e8c0 msgr2=0x55861e85eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 -- 172.21.15.44:0/3035673724 shutdown_connections 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 -- 172.21.15.44:0/3035673724 wait complete. 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 -- start start 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.236+0000 7f158e47f740 1 Processor -- start 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862330 0x55861e86cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e869fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 0x55861e86a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e784e80 con 0x55861e86d4f0 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e784b20 con 0x55861e862330 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 0x55861e86a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 0x55861e86a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55064/0 (socket says 172.21.15.44:55064) 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 -- 172.21.15.44:0/437851842 learned_addr learned my addr 172.21.15.44:0/437851842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1588cf9640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e869fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e74a260 con 0x55861e862c50 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f15894fa640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862330 0x55861e86cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f158e47f740 10 client.0 _ll_register_callbacks cb 0x55861e77fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 -- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 msgr2=0x55861e869fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e869fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 -- 172.21.15.44:0/437851842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862330 msgr2=0x55861e86cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862330 0x55861e86cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.237+0000 7f1589cfb640 1 -- 172.21.15.44:0/437851842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157400b770 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f1589cfb640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 0x55861e86a4f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f157c035000 tx=0x7f157c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c350c0a8c82f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f156cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f156cff9640 1 -- 172.21.15.44:0/437851842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c041070 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f156cff9640 1 -- 172.21.15.44:0/437851842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f157c047440 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55861e7ef680 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f156cff9640 1 -- 172.21.15.44:0/437851842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f157c058680 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f158e47f740 10 client.15903 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.238+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55861e7c5330 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f156cff9640 1 -- 172.21.15.44:0/437851842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f157c058c70 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f156cff9640 1 -- 172.21.15.44:0/437851842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f157c0478c0 con 0x55861e86d4f0 2024-09-13T17:27:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f158e47f740 10 client.15903 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f158e47f740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f158e47f740 20 client.15903 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f158e47f740 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e87f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f158e47f740 4 client.15903 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.239+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55861e7ef680 con 0x55861e87ce70 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.240+0000 7f15894fa640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e87f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.240+0000 7f156cff9640 10 client.15903 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.240+0000 7f15894fa640 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e87f260 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f13227da62fca3 server_cookie=529124cf23cbec20 in_seq=0 out_seq=0 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.241+0000 7f156cff9640 1 -- 172.21.15.44:0/437851842 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55861e7ef680 con 0x55861e87ce70 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.241+0000 7f156cff9640 10 client.15903 handle_command_reply: tid=0 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.241+0000 7f158e47f740 1 client.15903 shutdown 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.241+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 msgr2=0x55861e86a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.241+0000 7f158e47f740 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 0x55861e86a4f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f157c035000 tx=0x7f157c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.241+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 shutdown_connections 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e87f260 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e869fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f156cff9640 0 client.15903 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e86d4f0 0x55861e86a4f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 --2- 172.21.15.44:0/437851842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862330 0x55861e86cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 >> 172.21.15.44:0/437851842 conn(0x55861e85e8c0 msgr2=0x55861e85db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 shutdown_connections 2024-09-13T17:27:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 1 -- 172.21.15.44:0/437851842 wait complete. 2024-09-13T17:27:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.242+0000 7f158e47f740 20 client.15903 trim_cache size 0 max 16384 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.267+0000 7f158e47f740 1 Processor -- start 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.267+0000 7f158e47f740 1 -- start start 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f158e47f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e868860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f158e47f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862c50 0x55861e868da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f158e47f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e86d4f0 0x55861e867210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f15894fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e868860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f15894fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e868860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55070/0 (socket says 172.21.15.44:55070) 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f15894fa640 1 -- 172.21.15.44:0/2894058245 learned_addr learned my addr 172.21.15.44:0/2894058245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e85f340 con 0x55861e862330 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e8692e0 con 0x55861e862c50 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e86aaf0 con 0x55861e86d4f0 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f1588cf9640 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862c50 0x55861e868da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e86d4f0 0x55861e867210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f1589cfb640 1 -- 172.21.15.44:0/2894058245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862c50 msgr2=0x55861e868da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862c50 0x55861e868da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.268+0000 7f1589cfb640 1 -- 172.21.15.44:0/2894058245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 msgr2=0x55861e868860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e868860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f1589cfb640 1 -- 172.21.15.44:0/2894058245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55861e767300 con 0x55861e86d4f0 2024-09-13T17:27:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f15894fa640 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e868860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f1589cfb640 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e86d4f0 0x55861e867210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f157c02ae00 tx=0x7f157c02b020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c835395b1bbf8328 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f156ffff640 1 -- 172.21.15.44:0/2894058245 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0305c0 con 0x55861e86d4f0 2024-09-13T17:27:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f156ffff640 1 -- 172.21.15.44:0/2894058245 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f157c0478d0 con 0x55861e86d4f0 2024-09-13T17:27:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.269+0000 7f156ffff640 1 -- 172.21.15.44:0/2894058245 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c048b20 con 0x55861e86d4f0 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e86d4f0 msgr2=0x55861e867210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e86d4f0 0x55861e867210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f157c02ae00 tx=0x7f157c02b020 comp rx=0 tx=0).stop 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 shutdown_connections 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e86d4f0 0x55861e867210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e862c50 0x55861e868da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 --2- 172.21.15.44:0/2894058245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e868860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 >> 172.21.15.44:0/2894058245 conn(0x55861e85e8c0 msgr2=0x55861e865310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 shutdown_connections 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.270+0000 7f158e47f740 1 -- 172.21.15.44:0/2894058245 wait complete. 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.271+0000 7f158e47f740 1 -- start start 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.271+0000 7f158e47f740 1 Processor -- start 2024-09-13T17:27:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.271+0000 7f158e47f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e8740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.271+0000 7f15894fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e8740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e8740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55082/0 (socket says 172.21.15.44:55082) 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 -- 172.21.15.44:0/1374001864 learned_addr learned my addr 172.21.15.44:0/1374001864 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e874600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e8726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e872d80 con 0x55861e862330 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e872ef0 con 0x55861e86d4f0 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861e8730b0 con 0x55861e862c50 2024-09-13T17:27:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f1588cf9640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e874600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f1589cfb640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e8726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 -- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 msgr2=0x55861e874600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e874600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 -- 172.21.15.44:0/1374001864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 msgr2=0x55861e8726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e8726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.272+0000 7f15894fa640 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15841a4fe0 con 0x55861e862330 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f158e47f740 10 client.0 _ll_register_callbacks cb 0x7f157400b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1574017a10 con 0x55861e862330 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f15894fa640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e8740c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1580099480 tx=0x7f15800d0240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3611935d9f7e2c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f156cff9640 10 client.15909 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f156cff9640 1 -- 172.21.15.44:0/1374001864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800990e0 con 0x55861e862330 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f156cff9640 1 -- 172.21.15.44:0/1374001864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f15800d1040 con 0x55861e862330 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f156cff9640 1 -- 172.21.15.44:0/1374001864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1580009de0 con 0x55861e862330 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f158e47f740 10 client.15909 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.273+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55861e7c5330 con 0x55861e862330 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f156cff9640 1 -- 172.21.15.44:0/1374001864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15800b90b0 con 0x55861e862330 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f158e47f740 10 client.15909 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f158e47f740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f158e47f740 20 client.15909 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e8d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f158e47f740 4 client.15909 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:27:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55861e7ba5d0 con 0x55861e87ce70 2024-09-13T17:27:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f1588cf9640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e8d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f156cff9640 10 client.15909 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.274+0000 7f1588cf9640 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e8d63c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce87ede9c20ca655 server_cookie=ab04469b9f60b80f in_seq=0 out_seq=0 2024-09-13T17:27:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f156cff9640 1 -- 172.21.15.44:0/1374001864 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55861e7ba5d0 con 0x55861e87ce70 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f156cff9640 10 client.15909 handle_command_reply: tid=0 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 client.15909 shutdown 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 msgr2=0x55861e8740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e8740c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1580099480 tx=0x7f15800d0240 comp rx=0 tx=0).stop 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 shutdown_connections 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55861e87ce70 0x55861e8d63c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55861e862c50 0x55861e874600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f156cff9640 0 client.15909 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.278+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55861e86d4f0 0x55861e8726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.279+0000 7f158e47f740 1 --2- 172.21.15.44:0/1374001864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55861e862330 0x55861e8740c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.279+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 >> 172.21.15.44:0/1374001864 conn(0x55861e85e8c0 msgr2=0x55861e868820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.279+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 shutdown_connections 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.279+0000 7f158e47f740 1 -- 172.21.15.44:0/1374001864 wait complete. 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.279+0000 7f158e47f740 20 client.15909 trim_cache size 0 max 16384 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:27:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:27:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f15700a1430 msgr2=0x7f15700a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f15700a1430 0x7f15700a38f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f157c004af0 tx=0x7f157c005d80 comp rx=0 tx=0).stop 2024-09-13T17:27:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 msgr2=0x7f1584194dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 0x7f1584194dc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f158000d950 tx=0x7f158000de10 comp rx=0 tx=0).stop 2024-09-13T17:27:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 shutdown_connections 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f15700a1430 0x7f15700a38f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1584104c30 0x7f1584195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15840ff970 0x7f1584195300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 --2- 172.21.15.44:0/2180566971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15840feff0 0x7f1584194dc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 >> 172.21.15.44:0/2180566971 conn(0x7f1584074940 msgr2=0x7f15840fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 shutdown_connections 2024-09-13T17:27:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:32.280+0000 7f158b75e640 1 -- 172.21.15.44:0/2180566971 wait complete. 2024-09-13T17:27:32.409 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:32 smithi044 ceph-mon[31175]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T17:27:32.448 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:32 smithi070 ceph-mon[34502]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T17:27:32.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:32 smithi100 ceph-mon[35055]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T17:27:34.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:34 smithi070 ceph-mon[34502]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:34 smithi100 ceph-mon[35055]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:34 smithi044 ceph-mon[31175]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-13T17:27:36.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:36 smithi070 ceph-mon[34502]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-13T17:27:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:36 smithi100 ceph-mon[35055]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-13T17:27:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:36 smithi044 ceph-mon[31175]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-13T17:27:38.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:38 smithi070 ceph-mon[34502]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:38.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:38 smithi100 ceph-mon[35055]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:38.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:38 smithi044 ceph-mon[31175]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:40.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:40 smithi070 ceph-mon[34502]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:40.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:40 smithi100 ceph-mon[35055]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:40 smithi044 ceph-mon[31175]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:42.446 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:42 smithi070 ceph-mon[34502]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:42.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:42 smithi100 ceph-mon[35055]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:42.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:42 smithi044 ceph-mon[31175]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:44 smithi070 ceph-mon[34502]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:44.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:44 smithi100 ceph-mon[35055]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:44.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:44 smithi044 ceph-mon[31175]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 -- 172.21.15.44:0/415715020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0feff0 msgr2=0x7f025c0ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 --2- 172.21.15.44:0/415715020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0feff0 0x7f025c0ff430 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0250009930 tx=0x7f025002a420 comp rx=0 tx=0).stop 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 -- 172.21.15.44:0/415715020 shutdown_connections 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 --2- 172.21.15.44:0/415715020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0ff970 0x7f025c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 --2- 172.21.15.44:0/415715020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0feff0 0x7f025c0ff430 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 --2- 172.21.15.44:0/415715020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c104c30 0x7f025c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 -- 172.21.15.44:0/415715020 >> 172.21.15.44:0/415715020 conn(0x7f025c074940 msgr2=0x7f025c074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 -- 172.21.15.44:0/415715020 shutdown_connections 2024-09-13T17:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.418+0000 7f02624f0640 1 -- 172.21.15.44:0/415715020 wait complete. 2024-09-13T17:27:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.419+0000 7f02624f0640 1 Processor -- start 2024-09-13T17:27:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.419+0000 7f02624f0640 1 -- start start 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f02624f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0feff0 0x7f025c195040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f02624f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 0x7f025c195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f02624f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c104c30 0x7f025c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 0x7f025c195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 0x7f025c195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59664/0 (socket says 172.21.15.44:59664) 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f025b7fe640 1 -- 172.21.15.44:0/251693760 learned_addr learned my addr 172.21.15.44:0/251693760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c199f70 con 0x7f025c0ff970 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c19a0e0 con 0x7f025c104c30 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c19a2a0 con 0x7f025c0feff0 2024-09-13T17:27:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f025bfff640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0feff0 0x7f025c195040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.420+0000 7f0260a8d640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c104c30 0x7f025c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025bfff640 1 -- 172.21.15.44:0/251693760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c104c30 msgr2=0x7f025c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025bfff640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c104c30 0x7f025c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025bfff640 1 -- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 msgr2=0x7f025c195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025bfff640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 0x7f025c195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025bfff640 1 -- 172.21.15.44:0/251693760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250009560 con 0x7f025c0feff0 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025b7fe640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 0x7f025c195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f025bfff640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0feff0 0x7f025c195040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d6d0 tx=0x7f024c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f02597fa640 1 -- 172.21.15.44:0/251693760 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c005220 con 0x7f025c0feff0 2024-09-13T17:27:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c19a520 con 0x7f025c0feff0 2024-09-13T17:27:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f02597fa640 1 -- 172.21.15.44:0/251693760 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f024c009cb0 con 0x7f025c0feff0 2024-09-13T17:27:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f02597fa640 1 -- 172.21.15.44:0/251693760 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c004c60 con 0x7f025c0feff0 2024-09-13T17:27:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.421+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c067a10 con 0x7f025c0feff0 2024-09-13T17:27:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.424+0000 7f02597fa640 1 -- 172.21.15.44:0/251693760 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f024c009820 con 0x7f025c0feff0 2024-09-13T17:27:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 Processor -- start 2024-09-13T17:27:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 -- start start 2024-09-13T17:27:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba7bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eb9d5c30 con 0x55e2eba6aaf0 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eb9d33c0 con 0x55e2eba79900 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0265215740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eb9cecb0 con 0x55e2eba6a1d0 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.425+0000 7f0260a8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f0260a8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba6a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:36514/0 (socket says 172.21.15.44:36514) 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f0260a8d640 1 -- 172.21.15.44:0/1489510463 learned_addr learned my addr 172.21.15.44:0/1489510463 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba7bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025bfff640 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f02597fa640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f02440a13b0 0x7f02440a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 -- 172.21.15.44:0/1489510463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 msgr2=0x55e2eba6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 -- 172.21.15.44:0/1489510463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 msgr2=0x55e2eba793c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba793c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 -- 172.21.15.44:0/1489510463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2eb975910 con 0x55e2eba79900 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025bfff640 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f02597fa640 1 -- 172.21.15.44:0/251693760 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f024c0ccd70 con 0x7f025c0feff0 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f0260a8d640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f02440a13b0 0x7f02440a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.426+0000 7f025b7fe640 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba7bcf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0250009900 tx=0x7f02500040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5036c3113960e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.427+0000 7f02427fc640 1 -- 172.21.15.44:0/1489510463 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250031960 con 0x55e2eba79900 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.427+0000 7f02427fc640 1 -- 172.21.15.44:0/1489510463 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02500035f0 con 0x55e2eba79900 2024-09-13T17:27:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.427+0000 7f02427fc640 1 -- 172.21.15.44:0/1489510463 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025003d6b0 con 0x55e2eba79900 2024-09-13T17:27:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 -- 172.21.15.44:0/1489510463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 msgr2=0x55e2eba7bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0260a8d640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f02440a13b0 0x7f02440a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0254005c40 tx=0x7f0254009e10 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba7bcf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0250009900 tx=0x7f02500040c0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 -- 172.21.15.44:0/1489510463 shutdown_connections 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba7bcf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 --2- 172.21.15.44:0/1489510463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 -- 172.21.15.44:0/1489510463 >> 172.21.15.44:0/1489510463 conn(0x55e2eba668c0 msgr2=0x55e2eba66ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 -- 172.21.15.44:0/1489510463 shutdown_connections 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 -- 172.21.15.44:0/1489510463 wait complete. 2024-09-13T17:27:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.428+0000 7f0265215740 1 -- start start 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.429+0000 7f0265215740 1 Processor -- start 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.429+0000 7f0265215740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.429+0000 7f0265215740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba76530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba76530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba792b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:36522/0 (socket says 172.21.15.44:36522) 2024-09-13T17:27:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba76530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59684/0 (socket says 172.21.15.44:59684) 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 -- 172.21.15.44:0/2235900180 learned_addr learned my addr 172.21.15.44:0/2235900180 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eb98ce80 con 0x55e2eba6aaf0 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eb98cb20 con 0x55e2eba79900 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eb952260 con 0x55e2eba6a1d0 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f0260a8d640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 -- 172.21.15.44:0/2235900180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 msgr2=0x55e2eba76b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba76b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 -- 172.21.15.44:0/2235900180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 msgr2=0x55e2eba76530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.430+0000 7f025bfff640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba76530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f025bfff640 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c10c340 con 0x55e2eba6a1d0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f025b7fe640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba76530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f0265215740 10 client.0 _ll_register_callbacks cb 0x55e2eb987df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2eb9f7680 con 0x55e2eba6a1d0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f025bfff640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba792b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d6a0 tx=0x7f024c099280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f300f4323b40397f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f023b7fe640 10 client.25561 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f023b7fe640 1 -- 172.21.15.44:0/2235900180 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c09a020 con 0x55e2eba6a1d0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f023b7fe640 1 -- 172.21.15.44:0/2235900180 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f024c099920 con 0x55e2eba6a1d0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f023b7fe640 1 -- 172.21.15.44:0/2235900180 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f024c09d750 con 0x55e2eba6a1d0 2024-09-13T17:27:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f0265215740 10 client.25561 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.431+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55e2eb9cd330 con 0x55e2eba6a1d0 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f023b7fe640 1 -- 172.21.15.44:0/2235900180 <== mon.1 v2:172.21.15.100:3300/0 4 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f024c0d0030 con 0x55e2eba6a1d0 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f0265215740 10 client.25561 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f0265215740 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f0265215740 20 client.25561 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2eba890c0 0x55e2eba8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f0265215740 4 client.25561 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2eb9c25d0 con 0x55e2eba890c0 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.432+0000 7f025b7fe640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2eba890c0 0x55e2eba8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.433+0000 7f023b7fe640 10 client.25561 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.433+0000 7f025b7fe640 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2eba890c0 0x55e2eba8b580 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f8aaf9475f1169d server_cookie=63502335b7303c14 in_seq=0 out_seq=0 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.433+0000 7f023b7fe640 1 -- 172.21.15.44:0/2235900180 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e2eb9c25d0 con 0x55e2eba890c0 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.433+0000 7f023b7fe640 10 client.25561 handle_command_reply: tid=0 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.433+0000 7f0265215740 1 client.25561 shutdown 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 msgr2=0x55e2eba792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba792b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d6a0 tx=0x7f024c099280 comp rx=0 tx=0).stop 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 shutdown_connections 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2eba890c0 0x55e2eba8b580 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba79900 0x55e2eba76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba6aaf0 0x55e2eba76530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 --2- 172.21.15.44:0/2235900180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6a1d0 0x55e2eba792b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 >> 172.21.15.44:0/2235900180 conn(0x55e2eba668c0 msgr2=0x55e2eba65b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f023b7fe640 0 client.25561 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 shutdown_connections 2024-09-13T17:27:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 1 -- 172.21.15.44:0/2235900180 wait complete. 2024-09-13T17:27:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.434+0000 7f0265215740 20 client.25561 trim_cache size 0 max 16384 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 Processor -- start 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 -- start start 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba78e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2eba805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eba6b820 con 0x55e2eba79900 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eba907e0 con 0x55e2eba6a1d0 2024-09-13T17:27:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0265215740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eba77aa0 con 0x55e2eba6aaf0 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0260a8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2eba805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0260a8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2eba805c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59700/0 (socket says 172.21.15.44:59700) 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f0260a8d640 1 -- 172.21.15.44:0/2896482068 learned_addr learned my addr 172.21.15.44:0/2896482068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f025b7fe640 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba78e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.459+0000 7f025bfff640 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0260a8d640 1 -- 172.21.15.44:0/2896482068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 msgr2=0x55e2eba78e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0260a8d640 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba78e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0260a8d640 1 -- 172.21.15.44:0/2896482068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 msgr2=0x55e2eba76a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0260a8d640 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba76a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0260a8d640 1 -- 172.21.15.44:0/2896482068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2eb96f300 con 0x55e2eba79900 2024-09-13T17:27:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0260a8d640 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2eba805c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0254014a10 tx=0x7f025401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9ea2e21f35bc67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0242ffd640 1 -- 172.21.15.44:0/2896482068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02540099c0 con 0x55e2eba79900 2024-09-13T17:27:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0242ffd640 1 -- 172.21.15.44:0/2896482068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0254014e90 con 0x55e2eba79900 2024-09-13T17:27:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.460+0000 7f0242ffd640 1 -- 172.21.15.44:0/2896482068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254029a00 con 0x55e2eba79900 2024-09-13T17:27:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.461+0000 7f0265215740 1 -- 172.21.15.44:0/2896482068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 msgr2=0x55e2eba805c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.461+0000 7f0265215740 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2eba805c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0254014a10 tx=0x7f025401d000 comp rx=0 tx=0).stop 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.461+0000 7f0265215740 1 -- 172.21.15.44:0/2896482068 shutdown_connections 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba78e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2eba805c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 --2- 172.21.15.44:0/2896482068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba76a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 -- 172.21.15.44:0/2896482068 >> 172.21.15.44:0/2896482068 conn(0x55e2eba668c0 msgr2=0x55e2eba61a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 -- 172.21.15.44:0/2896482068 shutdown_connections 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 -- 172.21.15.44:0/2896482068 wait complete. 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 -- start start 2024-09-13T17:27:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.462+0000 7f0265215740 1 Processor -- start 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba7c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba7ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2ebade690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2eba7d410 con 0x55e2eba79900 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2ebadebd0 con 0x55e2eba6a1d0 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2ebaded60 con 0x55e2eba6aaf0 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0260a8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2ebade690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba7ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0260a8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2ebade690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59706/0 (socket says 172.21.15.44:59706) 2024-09-13T17:27:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0260a8d640 1 -- 172.21.15.44:0/4153316887 learned_addr learned my addr 172.21.15.44:0/4153316887 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025bfff640 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba7c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 10 client.0 _ll_register_callbacks cb 0x7f0250005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025b7fe640 1 -- 172.21.15.44:0/4153316887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 msgr2=0x55e2eba7c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025b7fe640 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba7c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025b7fe640 1 -- 172.21.15.44:0/4153316887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 msgr2=0x55e2ebade690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025b7fe640 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2ebade690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f025b7fe640 1 -- 172.21.15.44:0/4153316887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254014850 con 0x55e2eba6aaf0 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.463+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02500035f0 con 0x55e2eba6aaf0 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f025b7fe640 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba7ccd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02500057c0 tx=0x7f0250005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d0ef707aec6f08 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f023b7fe640 10 client.25567 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f023b7fe640 1 -- 172.21.15.44:0/4153316887 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025004b040 con 0x55e2eba6aaf0 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f023b7fe640 1 -- 172.21.15.44:0/4153316887 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0250052430 con 0x55e2eba6aaf0 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f023b7fe640 1 -- 172.21.15.44:0/4153316887 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0250052650 con 0x55e2eba6aaf0 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f0265215740 10 client.25567 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.464+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55e2eb9948e0 con 0x55e2eba6aaf0 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f023b7fe640 1 -- 172.21.15.44:0/4153316887 <== mon.1 v2:172.21.15.100:3300/0 4 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02500527b0 con 0x55e2eba6aaf0 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f0265215740 10 client.25567 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f0265215740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f0265215740 20 client.25567 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f0265215740 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2ebae00e0 0x55e2ebae24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f0265215740 4 client.25567 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:27:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2eb6c2300 con 0x55e2ebae00e0 2024-09-13T17:27:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f025bfff640 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2ebae00e0 0x55e2ebae24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f023b7fe640 10 client.25567 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.465+0000 7f025bfff640 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2ebae00e0 0x55e2ebae24d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f6fac1efb87b8f server_cookie=beaa178b273a0197 in_seq=0 out_seq=0 2024-09-13T17:27:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f023b7fe640 1 -- 172.21.15.44:0/4153316887 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2eb6c2300 con 0x55e2ebae00e0 2024-09-13T17:27:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f023b7fe640 10 client.25567 handle_command_reply: tid=0 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 client.25567 shutdown 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 msgr2=0x55e2eba7ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba7ccd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02500057c0 tx=0x7f0250005e00 comp rx=0 tx=0).stop 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 shutdown_connections 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2ebae00e0 0x55e2ebae24d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2eba6aaf0 0x55e2eba7ccd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2eba79900 0x55e2ebade690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 --2- 172.21.15.44:0/4153316887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2eba6a1d0 0x55e2eba7c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 >> 172.21.15.44:0/4153316887 conn(0x55e2eba668c0 msgr2=0x55e2eba74c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f023b7fe640 0 client.25567 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 shutdown_connections 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.469+0000 7f0265215740 1 -- 172.21.15.44:0/4153316887 wait complete. 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f0265215740 20 client.25567 trim_cache size 0 max 16384 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:27:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:27:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f02440a13b0 msgr2=0x7f02440a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f02440a13b0 0x7f02440a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0254005c40 tx=0x7f0254009e10 comp rx=0 tx=0).stop 2024-09-13T17:27:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0feff0 msgr2=0x7f025c195040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0feff0 0x7f025c195040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d6d0 tx=0x7f024c00db90 comp rx=0 tx=0).stop 2024-09-13T17:27:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 shutdown_connections 2024-09-13T17:27:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f02440a13b0 0x7f02440a3870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c104c30 0x7f025c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f025c0ff970 0x7f025c195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 --2- 172.21.15.44:0/251693760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c0feff0 0x7f025c195040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 >> 172.21.15.44:0/251693760 conn(0x7f025c074940 msgr2=0x7f025c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 shutdown_connections 2024-09-13T17:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:45.470+0000 7f02624f0640 1 -- 172.21.15.44:0/251693760 wait complete. 2024-09-13T17:27:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:46 smithi070 ceph-mon[34502]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:46.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:46 smithi100 ceph-mon[35055]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:46.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:46 smithi044 ceph-mon[31175]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:48 smithi070 ceph-mon[34502]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:27:48.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:48 smithi100 ceph-mon[35055]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:27:48.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:48 smithi044 ceph-mon[31175]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:27:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:50 smithi070 ceph-mon[34502]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:50 smithi100 ceph-mon[35055]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:50.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:50 smithi044 ceph-mon[31175]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:27:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:52 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:27:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:27:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:27:52.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:52 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:52.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:27:52.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:52.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:27:52.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:27:52.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:52 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:27:53.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 -- 172.21.15.44:0/799944630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec0fff50 msgr2=0x7fe9ec100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 --2- 172.21.15.44:0/799944630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec100330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009930 tx=0x7fe9dc02a420 comp rx=0 tx=0).stop 2024-09-13T17:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 -- 172.21.15.44:0/799944630 shutdown_connections 2024-09-13T17:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 --2- 172.21.15.44:0/799944630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec1050a0 0x7fe9ec10cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 --2- 172.21.15.44:0/799944630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9ec100900 0x7fe9ec104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 --2- 172.21.15.44:0/799944630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec100330 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 -- 172.21.15.44:0/799944630 >> 172.21.15.44:0/799944630 conn(0x7fe9ec0fb9e0 msgr2=0x7fe9ec0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 -- 172.21.15.44:0/799944630 shutdown_connections 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 -- 172.21.15.44:0/799944630 wait complete. 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 Processor -- start 2024-09-13T17:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.193+0000 7fe9f1444640 1 -- start start 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9f1444640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9f1444640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9ec1050a0 0x7fe9ec072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9f1444640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec073230 0x7fe9ec1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9f1444640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1a4960 con 0x7fe9ec073230 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9f1444640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1a4ad0 con 0x7fe9ec1050a0 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9f1444640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1a4c90 con 0x7fe9ec0fff50 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec073230 0x7fe9ec1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec073230 0x7fe9ec1a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35356/0 (socket says 172.21.15.44:35356) 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/320100792 learned_addr learned my addr 172.21.15.44:0/320100792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9ec1050a0 0x7fe9ec072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 -- 172.21.15.44:0/320100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9ec1050a0 msgr2=0x7fe9ec072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9ec1050a0 0x7fe9ec072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 -- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec073230 msgr2=0x7fe9ec1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec073230 0x7fe9ec1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 -- 172.21.15.44:0/320100792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc009560 con 0x7fe9ec0fff50 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.194+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec0727b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc02aa10 tx=0x7fe9dc032c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.195+0000 7fe9d3fff640 1 -- 172.21.15.44:0/320100792 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc005410 con 0x7fe9ec0fff50 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.195+0000 7fe9d3fff640 1 -- 172.21.15.44:0/320100792 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9dc005570 con 0x7fe9ec0fff50 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.195+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec1a4f10 con 0x7fe9ec0fff50 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.195+0000 7fe9d3fff640 1 -- 172.21.15.44:0/320100792 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0038e0 con 0x7fe9ec0fff50 2024-09-13T17:27:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.195+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec1a5330 con 0x7fe9ec0fff50 2024-09-13T17:27:53.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9d3fff640 1 -- 172.21.15.44:0/320100792 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fe9dc003a60 con 0x7fe9ec0fff50 2024-09-13T17:27:53.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 Processor -- start 2024-09-13T17:27:53.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 -- start start 2024-09-13T17:27:53.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf60730 0x56129cf6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf6afd0 0x56129cf6d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cecaf20 con 0x56129cf5fe10 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cec86b0 con 0x56129cf6afd0 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.197+0000 7fe9f4238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cec3fa0 con 0x56129cf60730 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35366/0 (socket says 172.21.15.44:35366) 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/1161236417 learned_addr learned my addr 172.21.15.44:0/1161236417 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf60730 0x56129cf6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9d3fff640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9d40a13b0 0x7fe9d40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf6afd0 0x56129cf6d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/1161236417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf60730 msgr2=0x56129cf6aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf60730 0x56129cf6aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/1161236417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf6afd0 msgr2=0x56129cf6d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf6afd0 0x56129cf6d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/1161236417 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56129ce7e840 con 0x56129cf5fe10 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf6afd0 0x56129cf6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9d3fff640 1 -- 172.21.15.44:0/320100792 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe9dc0ed950 con 0x7fe9ec0fff50 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9d40a13b0 0x7fe9d40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf601f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4011570 tx=0x7fe9e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370bd2aaf6392231 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.198+0000 7fe9d17fa640 1 -- 172.21.15.44:0/1161236417 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e401b7c0 con 0x56129cf5fe10 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9d17fa640 1 -- 172.21.15.44:0/1161236417 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e401bdb0 con 0x56129cf5fe10 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9d17fa640 1 -- 172.21.15.44:0/1161236417 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400eb40 con 0x56129cf5fe10 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 -- 172.21.15.44:0/1161236417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 msgr2=0x56129cf601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf601f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4011570 tx=0x7fe9e4011a30 comp rx=0 tx=0).stop 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9d40a13b0 0x7fe9d40a3870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4023000 tx=0x7fe9e4024000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 -- 172.21.15.44:0/1161236417 shutdown_connections 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf60730 0x56129cf6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf6afd0 0x56129cf6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1161236417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf601f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 -- 172.21.15.44:0/1161236417 >> 172.21.15.44:0/1161236417 conn(0x56129cf5c380 msgr2=0x56129cf5c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 -- 172.21.15.44:0/1161236417 shutdown_connections 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.199+0000 7fe9f4238740 1 -- 172.21.15.44:0/1161236417 wait complete. 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 -- start start 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 Processor -- start 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf6a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf67b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129ce652a0 con 0x56129cf6afd0 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cb9c430 con 0x56129cf60730 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129ce47680 con 0x56129cf5fe10 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9f4238740 10 client.0 _ll_register_callbacks cb 0x56129cebd610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35368/0 (socket says 172.21.15.44:35368) 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/3174537088 learned_addr learned my addr 172.21.15.44:0/3174537088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf6a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf67b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 msgr2=0x56129cf6a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf6a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/3174537088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 msgr2=0x56129cf67b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf67b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/3174537088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e4009d20 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.200+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf67b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf680b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe9e401b8a0 tx=0x7fe9e40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb89718d6add964 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9ceff5640 1 -- 172.21.15.44:0/3174537088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400f3c0 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9ceff5640 1 -- 172.21.15.44:0/3174537088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe9e400e3b0 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56129cbb74e0 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9ceff5640 1 -- 172.21.15.44:0/3174537088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e400ee20 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 10 client.15939 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56129cec2620 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9ceff5640 1 -- 172.21.15.44:0/3174537088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56129cbb74e0 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9ceff5640 1 -- 172.21.15.44:0/3174537088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9e4024070 con 0x56129cf6afd0 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 10 client.15939 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:53.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 20 client.15939 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cf7ad00 0x56129cf7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 4 client.15939 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.201+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56129cbc87e0 con 0x56129cf7ad00 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.202+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cf7ad00 0x56129cf7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.202+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cf7ad00 0x56129cf7d120 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229f32b078f59f55 server_cookie=1da80cf491357de5 in_seq=0 out_seq=0 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.202+0000 7fe9ceff5640 10 client.15939 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9ceff5640 1 -- 172.21.15.44:0/3174537088 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56129cbc87e0 con 0x56129cf7ad00 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9ceff5640 10 client.15939 handle_command_reply: tid=0 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 client.15939 shutdown 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 msgr2=0x56129cf680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf680b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe9e401b8a0 tx=0x7fe9e40096c0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 shutdown_connections 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cf7ad00 0x56129cf7d120 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf680b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf67b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 --2- 172.21.15.44:0/3174537088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf6a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 >> 172.21.15.44:0/3174537088 conn(0x56129cf5c380 msgr2=0x56129cf5b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9ceff5640 0 client.15939 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 shutdown_connections 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 1 -- 172.21.15.44:0/3174537088 wait complete. 2024-09-13T17:27:53.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.203+0000 7fe9f4238740 20 client.15939 trim_cache size 0 max 16384 2024-09-13T17:27:53.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 Processor -- start 2024-09-13T17:27:53.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 -- start start 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf5bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf6afd0 0x56129cf722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cbb7510 con 0x56129cf5fe10 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cf5d5c0 con 0x56129cf60730 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.217+0000 7fe9f4238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cf686b0 con 0x56129cf6afd0 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf5bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf5bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35382/0 (socket says 172.21.15.44:35382) 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 -- 172.21.15.44:0/1475561590 learned_addr learned my addr 172.21.15.44:0/1475561590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf6afd0 0x56129cf722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 -- 172.21.15.44:0/1475561590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf6afd0 msgr2=0x56129cf722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf6afd0 0x56129cf722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 -- 172.21.15.44:0/1475561590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 msgr2=0x56129cf71d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf71d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 -- 172.21.15.44:0/1475561590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56129ce64790 con 0x56129cf5fe10 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf5bd00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0eac30 tx=0x7fe9dc0efdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f486f62dbf1a66c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9d1ffb640 1 -- 172.21.15.44:0/1475561590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0b9f10 con 0x56129cf5fe10 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9d1ffb640 1 -- 172.21.15.44:0/1475561590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9dc0ba5b0 con 0x56129cf5fe10 2024-09-13T17:27:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.218+0000 7fe9d1ffb640 1 -- 172.21.15.44:0/1475561590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0b6cc0 con 0x56129cf5fe10 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 -- 172.21.15.44:0/1475561590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 msgr2=0x56129cf5bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf5bd00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0eac30 tx=0x7fe9dc0efdd0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 -- 172.21.15.44:0/1475561590 shutdown_connections 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf6afd0 0x56129cf722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf71d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 --2- 172.21.15.44:0/1475561590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf5fe10 0x56129cf5bd00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 -- 172.21.15.44:0/1475561590 >> 172.21.15.44:0/1475561590 conn(0x56129cf5c380 msgr2=0x56129cf81a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 -- 172.21.15.44:0/1475561590 shutdown_connections 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.219+0000 7fe9f4238740 1 -- 172.21.15.44:0/1475561590 wait complete. 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 -- start start 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 Processor -- start 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cf63e80 con 0x56129cf6afd0 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cf63ff0 con 0x56129cf60730 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cf641b0 con 0x56129cf5fe10 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9f4238740 10 client.0 _ll_register_callbacks cb 0x7fe9e4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35398/0 (socket says 172.21.15.44:35398) 2024-09-13T17:27:53.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/2832331163 learned_addr learned my addr 172.21.15.44:0/2832331163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9ea7fc640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 msgr2=0x56129cf722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/2832331163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 msgr2=0x56129cf650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 -- 172.21.15.44:0/2832331163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc0f4040 con 0x56129cf6afd0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9eb7fe640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf657e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4038fd0 tx=0x7fe9e4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1201d78425d2f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.220+0000 7fe9ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9ceff5640 1 -- 172.21.15.44:0/2832331163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400f3c0 con 0x56129cf6afd0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9ceff5640 1 -- 172.21.15.44:0/2832331163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe9e400e3b0 con 0x56129cf6afd0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e401bdb0 con 0x56129cf6afd0 2024-09-13T17:27:53.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9ceff5640 1 -- 172.21.15.44:0/2832331163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e400ebd0 con 0x56129cf6afd0 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 10 client.15945 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56129ce89ea0 con 0x56129cf6afd0 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9ceff5640 1 -- 172.21.15.44:0/2832331163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e401bdb0 con 0x56129cf6afd0 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9ceff5640 1 -- 172.21.15.44:0/2832331163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9e402f500 con 0x56129cf6afd0 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 10 client.15945 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 20 client.15945 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cfd1960 0x56129cfd3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 4 client.15945 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.221+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56129cc1be30 con 0x56129cfd1960 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cfd1960 0x56129cfd3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9ceff5640 10 client.15945 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:53.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9eaffd640 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cfd1960 0x56129cfd3d50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20fa180582966737 server_cookie=f60f0618d07a5cc2 in_seq=0 out_seq=0 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9ceff5640 1 -- 172.21.15.44:0/2832331163 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56129cc1be30 con 0x56129cfd1960 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9ceff5640 10 client.15945 handle_command_reply: tid=0 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 client.15945 shutdown 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 msgr2=0x56129cf657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf657e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4038fd0 tx=0x7fe9e4009500 comp rx=0 tx=0).stop 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 shutdown_connections 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56129cfd1960 0x56129cfd3d50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56129cf6afd0 0x56129cf657e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56129cf60730 0x56129cf650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 --2- 172.21.15.44:0/2832331163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56129cf5fe10 0x56129cf722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9ceff5640 0 client.15945 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 >> 172.21.15.44:0/2832331163 conn(0x56129cf5c380 msgr2=0x56129cf62eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 shutdown_connections 2024-09-13T17:27:53.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.222+0000 7fe9f4238740 1 -- 172.21.15.44:0/2832331163 wait complete. 2024-09-13T17:27:53.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f4238740 20 client.15945 trim_cache size 0 max 16384 2024-09-13T17:27:53.231 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:27:53.231 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:27:53.231 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:27:53.231 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9d40a13b0 msgr2=0x7fe9d40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9d40a13b0 0x7fe9d40a3870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4023000 tx=0x7fe9e4024000 comp rx=0 tx=0).stop 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0fff50 msgr2=0x7fe9ec0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec0727b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc02aa10 tx=0x7fe9dc032c20 comp rx=0 tx=0).stop 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 shutdown_connections 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9d40a13b0 0x7fe9d40a3870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ec073230 0x7fe9ec1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9ec1050a0 0x7fe9ec072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 --2- 172.21.15.44:0/320100792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0fff50 0x7fe9ec0727b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 >> 172.21.15.44:0/320100792 conn(0x7fe9ec0fb9e0 msgr2=0x7fe9ec0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 shutdown_connections 2024-09-13T17:27:53.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.223+0000 7fe9f1444640 1 -- 172.21.15.44:0/320100792 wait complete. 2024-09-13T17:27:53.235 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:27:53.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:27:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 -- 172.21.15.44:0/2714724562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc0ff830 msgr2=0x7f10fc103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 --2- 172.21.15.44:0/2714724562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc0ff830 0x7f10fc103870 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009930 tx=0x7f10ec02a420 comp rx=0 tx=0).stop 2024-09-13T17:27:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 -- 172.21.15.44:0/2714724562 shutdown_connections 2024-09-13T17:27:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 --2- 172.21.15.44:0/2714724562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10fc103fd0 0x7f10fc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 --2- 172.21.15.44:0/2714724562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc0ff830 0x7f10fc103870 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 --2- 172.21.15.44:0/2714724562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10fc0fee80 0x7f10fc0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 -- 172.21.15.44:0/2714724562 >> 172.21.15.44:0/2714724562 conn(0x7f10fc075c60 msgr2=0x7f10fc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 -- 172.21.15.44:0/2714724562 shutdown_connections 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 -- 172.21.15.44:0/2714724562 wait complete. 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 Processor -- start 2024-09-13T17:27:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.339+0000 7f1103e12640 1 -- start start 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1103e12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10fc0fee80 0x7f10fc197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1103e12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 0x7f10fc197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1103e12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10fc197ab0 0x7f10fc1a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1103e12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc1a2750 con 0x7f10fc103fd0 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1103e12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc1a28c0 con 0x7f10fc197ab0 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1103e12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc1a2a80 con 0x7f10fc0fee80 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 0x7f10fc197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 0x7f10fc197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35416/0 (socket says 172.21.15.44:35416) 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 -- 172.21.15.44:0/3784523210 learned_addr learned my addr 172.21.15.44:0/3784523210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11023af640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10fc197ab0 0x7f10fc1a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f1101bae640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10fc0fee80 0x7f10fc197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 -- 172.21.15.44:0/3784523210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10fc0fee80 msgr2=0x7f10fc197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10fc0fee80 0x7f10fc197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 -- 172.21.15.44:0/3784523210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10fc197ab0 msgr2=0x7f10fc1a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10fc197ab0 0x7f10fc1a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 -- 172.21.15.44:0/3784523210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ec009560 con 0x7f10fc103fd0 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.340+0000 7f11013ad640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 0x7f10fc197570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005ab0 tx=0x7f10ec0329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.341+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec032df0 con 0x7f10fc103fd0 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.341+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10ec005490 con 0x7f10fc103fd0 2024-09-13T17:27:53.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.341+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc1a2d00 con 0x7f10fc103fd0 2024-09-13T17:27:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.341+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0035f0 con 0x7f10fc103fd0 2024-09-13T17:27:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.341+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc1a3160 con 0x7f10fc103fd0 2024-09-13T17:27:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.342+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10fc100350 con 0x7f10fc103fd0 2024-09-13T17:27:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.343+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f10ec0047c0 con 0x7f10fc103fd0 2024-09-13T17:27:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.343+0000 7f10f2ffd640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10e80a13b0 0x7f10e80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.344+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f10ec0edd60 con 0x7f10fc103fd0 2024-09-13T17:27:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.344+0000 7f1101bae640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10e80a13b0 0x7f10e80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.344+0000 7f1101bae640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10e80a13b0 0x7f10e80a3870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009bd0 tx=0x7f10f8005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.348+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f10ec0b6d00 con 0x7f10fc103fd0 2024-09-13T17:27:53.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.561+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10fc1071e0 con 0x7f10fc103fd0 2024-09-13T17:27:53.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.562+0000 7f10f2ffd640 1 -- 172.21.15.44:0/3784523210 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f10ec0ba320 con 0x7f10fc103fd0 2024-09-13T17:27:53.563 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:27:53.563 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:27:53.563 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.563+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10e80a13b0 msgr2=0x7f10e80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.563+0000 7f1103e12640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10e80a13b0 0x7f10e80a3870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009bd0 tx=0x7f10f8005f90 comp rx=0 tx=0).stop 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 msgr2=0x7f10fc197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 0x7f10fc197570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005ab0 tx=0x7f10ec0329f0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 shutdown_connections 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10e80a13b0 0x7f10e80a3870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10fc197ab0 0x7f10fc1a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10fc103fd0 0x7f10fc197570 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 --2- 172.21.15.44:0/3784523210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10fc0fee80 0x7f10fc197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 >> 172.21.15.44:0/3784523210 conn(0x7f10fc075c60 msgr2=0x7f10fc102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 shutdown_connections 2024-09-13T17:27:53.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.564+0000 7f1103e12640 1 -- 172.21.15.44:0/3784523210 wait complete. 2024-09-13T17:27:53.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:27:53.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 -- 172.21.15.44:0/3371999903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 msgr2=0x7fd374101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/3371999903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd374101320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd370009900 tx=0x7fd37002a480 comp rx=0 tx=0).stop 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 -- 172.21.15.44:0/3371999903 shutdown_connections 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/3371999903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37410a310 0x7fd37410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/3371999903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd374101860 0x7fd374109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/3371999903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd374101320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 -- 172.21.15.44:0/3371999903 >> 172.21.15.44:0/3371999903 conn(0x7fd374075c60 msgr2=0x7fd3740fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 -- 172.21.15.44:0/3371999903 shutdown_connections 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.682+0000 7fd37b2e5640 1 -- 172.21.15.44:0/3371999903 wait complete. 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 Processor -- start 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 -- start start 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd3740feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd374101860 0x7fd3740ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37410a310 0x7fd3740ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741a4510 con 0x7fd374100f40 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741a4680 con 0x7fd37410a310 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd37b2e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741a4840 con 0x7fd374101860 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd379081640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd3740feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd379081640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd3740feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35428/0 (socket says 172.21.15.44:35428) 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd379081640 1 -- 172.21.15.44:0/1795891715 learned_addr learned my addr 172.21.15.44:0/1795891715 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd378880640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd374101860 0x7fd3740ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.683+0000 7fd379882640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37410a310 0x7fd3740ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd379081640 1 -- 172.21.15.44:0/1795891715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd374101860 msgr2=0x7fd3740ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd379081640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd374101860 0x7fd3740ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd379081640 1 -- 172.21.15.44:0/1795891715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37410a310 msgr2=0x7fd3740ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd379081640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37410a310 0x7fd3740ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd379081640 1 -- 172.21.15.44:0/1795891715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370009560 con 0x7fd374100f40 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd378880640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd374101860 0x7fd3740ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd379081640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd3740feb90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd3700057c0 tx=0x7fd370032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd36a7fc640 1 -- 172.21.15.44:0/1795891715 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370005000 con 0x7fd374100f40 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd36a7fc640 1 -- 172.21.15.44:0/1795891715 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3700046c0 con 0x7fd374100f40 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd36a7fc640 1 -- 172.21.15.44:0/1795891715 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370003850 con 0x7fd374100f40 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3741a4ac0 con 0x7fd374100f40 2024-09-13T17:27:53.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.684+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3741a4ef0 con 0x7fd374100f40 2024-09-13T17:27:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.686+0000 7fd36a7fc640 1 -- 172.21.15.44:0/1795891715 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fd370005160 con 0x7fd374100f40 2024-09-13T17:27:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 Processor -- start 2024-09-13T17:27:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 -- start start 2024-09-13T17:27:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb82410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb84d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bae1cc0 con 0x55cd9bb77790 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9badf450 con 0x55cd9bb780b0 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd37e007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9badad40 con 0x55cd9bb82950 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd379882640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd379882640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb77b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35430/0 (socket says 172.21.15.44:35430) 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd379882640 1 -- 172.21.15.44:0/3640003058 learned_addr learned my addr 172.21.15.44:0/3640003058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd36a7fc640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd3600a13b0 0x7fd3600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd379081640 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb82410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd378880640 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb84d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.687+0000 7fd379882640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd3600a13b0 0x7fd3600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd36a7fc640 1 -- 172.21.15.44:0/1795891715 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd3700eddc0 con 0x7fd374100f40 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 -- 172.21.15.44:0/3640003058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 msgr2=0x55cd9bb84d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb84d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 -- 172.21.15.44:0/3640003058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 msgr2=0x55cd9bb82410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb82410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 -- 172.21.15.44:0/3640003058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9baa2620 con 0x55cd9bb77790 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb77b70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0049d0 tx=0x7fd36c013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856788e38afc50a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd35f7fe640 1 -- 172.21.15.44:0/3640003058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c012820 con 0x55cd9bb77790 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd35f7fe640 1 -- 172.21.15.44:0/3640003058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd36c012e10 con 0x55cd9bb77790 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd35f7fe640 1 -- 172.21.15.44:0/3640003058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c010960 con 0x55cd9bb77790 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.688+0000 7fd379882640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd3600a13b0 0x7fd3600a3870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd36c023000 tx=0x7fd36c021000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 -- 172.21.15.44:0/3640003058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 msgr2=0x55cd9bb77b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb77b70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0049d0 tx=0x7fd36c013f50 comp rx=0 tx=0).stop 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 -- 172.21.15.44:0/3640003058 shutdown_connections 2024-09-13T17:27:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb84d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb82410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 --2- 172.21.15.44:0/3640003058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb77b70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 -- 172.21.15.44:0/3640003058 >> 172.21.15.44:0/3640003058 conn(0x55cd9bb73d00 msgr2=0x55cd9bb740e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 -- 172.21.15.44:0/3640003058 shutdown_connections 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 -- 172.21.15.44:0/3640003058 wait complete. 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 -- start start 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.689+0000 7fd37e007740 1 Processor -- start 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb77790 0x55cd9bb81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb7f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 0x55cd9bb7f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9ba99180 con 0x55cd9bb82950 2024-09-13T17:27:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9b7b3430 con 0x55cd9bb780b0 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9ba5e5e0 con 0x55cd9bb77790 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd37e007740 10 client.0 _ll_register_callbacks cb 0x55cd9bacaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 0x55cd9bb7f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 0x55cd9bb7f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35446/0 (socket says 172.21.15.44:35446) 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 -- 172.21.15.44:0/2500058058 learned_addr learned my addr 172.21.15.44:0/2500058058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd378880640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb7f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379081640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb77790 0x55cd9bb81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 -- 172.21.15.44:0/2500058058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb77790 msgr2=0x55cd9bb81f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb77790 0x55cd9bb81f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 -- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 msgr2=0x55cd9bb7f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb7f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.690+0000 7fd379882640 1 -- 172.21.15.44:0/2500058058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36c0136a0 con 0x55cd9bb82950 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd379882640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 0x55cd9bb7f790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0134c0 tx=0x7fd36c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbd39cd79499b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd35cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd35cff9640 1 -- 172.21.15.44:0/2500058058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c02c070 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd35cff9640 1 -- 172.21.15.44:0/2500058058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd36c010430 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd9bb04b80 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd35cff9640 1 -- 172.21.15.44:0/2500058058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd36c012260 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 10 client.15963 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cd9bad93c0 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd35cff9640 1 -- 172.21.15.44:0/2500058058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cd9bb04b80 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd35cff9640 1 -- 172.21.15.44:0/2500058058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd36c0274b0 con 0x55cd9bb82950 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 10 client.15963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 20 client.15963 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bb92230 0x55cd9bb94650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 4 client.15963 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.691+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd9bacd1f0 con 0x55cd9bb92230 2024-09-13T17:27:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.692+0000 7fd379081640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bb92230 0x55cd9bb94650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.692+0000 7fd379081640 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bb92230 0x55cd9bb94650 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a450335c899a756 server_cookie=aad1fdbad58cc577 in_seq=0 out_seq=0 2024-09-13T17:27:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.692+0000 7fd35cff9640 10 client.15963 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd35cff9640 1 -- 172.21.15.44:0/2500058058 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cd9bacd1f0 con 0x55cd9bb92230 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd35cff9640 10 client.15963 handle_command_reply: tid=0 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 client.15963 shutdown 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 msgr2=0x55cd9bb7f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 0x55cd9bb7f790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0134c0 tx=0x7fd36c005c40 comp rx=0 tx=0).stop 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 shutdown_connections 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bb92230 0x55cd9bb94650 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb82950 0x55cd9bb7f790 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb7f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 --2- 172.21.15.44:0/2500058058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb77790 0x55cd9bb81f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd35cff9640 0 client.15963 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 >> 172.21.15.44:0/2500058058 conn(0x55cd9bb73d00 msgr2=0x55cd9bb79a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 shutdown_connections 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 1 -- 172.21.15.44:0/2500058058 wait complete. 2024-09-13T17:27:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.693+0000 7fd37e007740 20 client.15963 trim_cache size 0 max 16384 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.704+0000 7fd37e007740 1 Processor -- start 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.704+0000 7fd37e007740 1 -- start start 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd37e007740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd37e007740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd37e007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb89b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd37e007740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bb74f10 con 0x55cd9bb77790 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd37e007740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bb6e3b0 con 0x55cd9bb780b0 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd37e007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bb82520 con 0x55cd9bb82950 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35456/0 (socket says 172.21.15.44:35456) 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 -- 172.21.15.44:0/713804270 learned_addr learned my addr 172.21.15.44:0/713804270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd378880640 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379882640 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb89b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 -- 172.21.15.44:0/713804270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 msgr2=0x55cd9bb89b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb89b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 -- 172.21.15.44:0/713804270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 msgr2=0x55cd9bb89610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb89610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 -- 172.21.15.44:0/713804270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9ba7b770 con 0x55cd9bb77790 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.705+0000 7fd379081640 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb736a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd3700eb0f0 tx=0x7fd37008cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78458ecadc2eeb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd35ffff640 1 -- 172.21.15.44:0/713804270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700baa20 con 0x55cd9bb77790 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd35ffff640 1 -- 172.21.15.44:0/713804270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3700be7a0 con 0x55cd9bb77790 2024-09-13T17:27:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd35ffff640 1 -- 172.21.15.44:0/713804270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700ba170 con 0x55cd9bb77790 2024-09-13T17:27:53.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 -- 172.21.15.44:0/713804270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 msgr2=0x55cd9bb736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb736a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd3700eb0f0 tx=0x7fd37008cb00 comp rx=0 tx=0).stop 2024-09-13T17:27:53.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 -- 172.21.15.44:0/713804270 shutdown_connections 2024-09-13T17:27:53.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bb89b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb780b0 0x55cd9bb89610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 --2- 172.21.15.44:0/713804270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb77790 0x55cd9bb736a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 -- 172.21.15.44:0/713804270 >> 172.21.15.44:0/713804270 conn(0x55cd9bb73d00 msgr2=0x55cd9bb98fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 -- 172.21.15.44:0/713804270 shutdown_connections 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.706+0000 7fd37e007740 1 -- 172.21.15.44:0/713804270 wait complete. 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 -- start start 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 Processor -- start 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb77790 0x55cd9bb85c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 0x55cd9bb861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bbe06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bbe0c70 con 0x55cd9bb780b0 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bbe0de0 con 0x55cd9bb77790 2024-09-13T17:27:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bbe0fa0 con 0x55cd9bb82950 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd37e007740 10 client.0 _ll_register_callbacks cb 0x7fd36c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd378880640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 0x55cd9bb861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd378880640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 0x55cd9bb861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35464/0 (socket says 172.21.15.44:35464) 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd378880640 1 -- 172.21.15.44:0/3568978608 learned_addr learned my addr 172.21.15.44:0/3568978608 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.707+0000 7fd379882640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bbe06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd379081640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb77790 0x55cd9bb85c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd378880640 1 -- 172.21.15.44:0/3568978608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 msgr2=0x55cd9bbe06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd378880640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bbe06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd378880640 1 -- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb77790 msgr2=0x55cd9bb85c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd378880640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb77790 0x55cd9bb85c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd378880640 1 -- 172.21.15.44:0/3568978608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3741a4ef0 con 0x55cd9bb780b0 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd379081640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb77790 0x55cd9bb85c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:53.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd378880640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 0x55cd9bb861d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd364005d00 tx=0x7fd364012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90a1e3faa6a497f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd35cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd35cff9640 1 -- 172.21.15.44:0/3568978608 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364016040 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd36c012e10 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd35cff9640 1 -- 172.21.15.44:0/3568978608 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd364010c30 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd35cff9640 1 -- 172.21.15.44:0/3568978608 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd364010e30 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd37e007740 10 client.15969 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cd9baa0e80 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.708+0000 7fd35cff9640 1 -- 172.21.15.44:0/3568978608 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd36c012e10 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd35cff9640 1 -- 172.21.15.44:0/3568978608 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd364004cd0 con 0x55cd9bb780b0 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd37e007740 10 client.15969 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd37e007740 10 client.15969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd37e007740 20 client.15969 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd37e007740 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bbe9a30 0x55cd9bbebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd37e007740 4 client.15969 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd9b7ce4e0 con 0x55cd9bbe9a30 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd379081640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bbe9a30 0x55cd9bbebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd35cff9640 10 client.15969 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd379081640 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bbe9a30 0x55cd9bbebe50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2eaf5f26bc62e7d0 server_cookie=2718081a9918928f in_seq=0 out_seq=0 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd35cff9640 1 -- 172.21.15.44:0/3568978608 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd9b7ce4e0 con 0x55cd9bbe9a30 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.709+0000 7fd35cff9640 10 client.15969 handle_command_reply: tid=0 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 client.15969 shutdown 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 msgr2=0x55cd9bb861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 0x55cd9bb861d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd364005d00 tx=0x7fd364012e00 comp rx=0 tx=0).stop 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 shutdown_connections 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55cd9bbe9a30 0x55cd9bbebe50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd9bb82950 0x55cd9bbe06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd35cff9640 0 client.15969 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd9bb780b0 0x55cd9bb861d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 --2- 172.21.15.44:0/3568978608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd9bb77790 0x55cd9bb85c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 >> 172.21.15.44:0/3568978608 conn(0x55cd9bb73d00 msgr2=0x55cd9bb7dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 shutdown_connections 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 1 -- 172.21.15.44:0/3568978608 wait complete. 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.710+0000 7fd37e007740 20 client.15969 trim_cache size 0 max 16384 2024-09-13T17:27:53.717 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:27:53.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd3600a13b0 msgr2=0x7fd3600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd3600a13b0 0x7fd3600a3870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd36c023000 tx=0x7fd36c021000 comp rx=0 tx=0).stop 2024-09-13T17:27:53.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 msgr2=0x7fd3740feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd3740feb90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd3700057c0 tx=0x7fd370032be0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 shutdown_connections 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd3600a13b0 0x7fd3600a3870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd374101860 0x7fd3740ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37410a310 0x7fd3740ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 --2- 172.21.15.44:0/1795891715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374100f40 0x7fd3740feb90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 >> 172.21.15.44:0/1795891715 conn(0x7fd374075c60 msgr2=0x7fd3740fb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 shutdown_connections 2024-09-13T17:27:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.711+0000 7fd37b2e5640 1 -- 172.21.15.44:0/1795891715 wait complete. 2024-09-13T17:27:53.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 -- 172.21.15.44:0/746585070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 msgr2=0x7fdcf810cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 --2- 172.21.15.44:0/746585070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf810cd90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009cd0 tx=0x7fdcf002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 -- 172.21.15.44:0/746585070 shutdown_connections 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 --2- 172.21.15.44:0/746585070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf810cd90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 --2- 172.21.15.44:0/746585070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcf80ff530 0x7fdcf8105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 --2- 172.21.15.44:0/746585070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcf80feb80 0x7fdcf80fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 -- 172.21.15.44:0/746585070 >> 172.21.15.44:0/746585070 conn(0x7fdcf8074110 msgr2=0x7fdcf8074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 -- 172.21.15.44:0/746585070 shutdown_connections 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.830+0000 7fdcfd419640 1 -- 172.21.15.44:0/746585070 wait complete. 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 Processor -- start 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 -- start start 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcf80feb80 0x7fdcf81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf8196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcf819ac20 0x7fdcf819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf819b610 con 0x7fdcf8106440 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf819b780 con 0x7fdcf819ac20 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcfd419640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf819b8f0 con 0x7fdcf80feb80 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcf67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf8196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcf67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf8196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35490/0 (socket says 172.21.15.44:35490) 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.831+0000 7fdcf67fc640 1 -- 172.21.15.44:0/62860294 learned_addr learned my addr 172.21.15.44:0/62860294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcf819ac20 0x7fdcf819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcf80feb80 0x7fdcf81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf67fc640 1 -- 172.21.15.44:0/62860294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcf80feb80 msgr2=0x7fdcf81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcf80feb80 0x7fdcf81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf67fc640 1 -- 172.21.15.44:0/62860294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcf819ac20 msgr2=0x7fdcf819b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcf819ac20 0x7fdcf819b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf67fc640 1 -- 172.21.15.44:0/62860294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0009900 con 0x7fdcf8106440 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf8196830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e9f0 tx=0x7fdcec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcdffff640 1 -- 172.21.15.44:0/62860294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec009700 con 0x7fdcf8106440 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcdffff640 1 -- 172.21.15.44:0/62860294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcec00cd80 con 0x7fdcf8106440 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf81a2ed0 con 0x7fdcf8106440 2024-09-13T17:27:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcdffff640 1 -- 172.21.15.44:0/62860294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0187e0 con 0x7fdcf8106440 2024-09-13T17:27:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.832+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf81a32f0 con 0x7fdcf8106440 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.835+0000 7fdcdffff640 1 -- 172.21.15.44:0/62860294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fdcec010430 con 0x7fdcf8106440 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.835+0000 7fdcdffff640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdce00a13b0 0x7fdce00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.835+0000 7fdcdffff640 1 -- 172.21.15.44:0/62860294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fdcec014070 con 0x7fdcf8106440 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.835+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdce00a13b0 0x7fdce00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 Processor -- start 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 -- start start 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdce00a13b0 0x7fdce00a3870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009b60 tx=0x7fdce8005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a51790 0x558442a51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a5c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a5ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584429bbcc0 con 0x558442a5c950 2024-09-13T17:27:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584429b9450 con 0x558442a51790 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdd00214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584429b4d40 con 0x558442a520b0 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdcf77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a5ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdcf77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a5ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35506/0 (socket says 172.21.15.44:35506) 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdcf77fe640 1 -- 172.21.15.44:0/1790874972 learned_addr learned my addr 172.21.15.44:0/1790874972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a51790 0x558442a51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.836+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a5c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcf77fe640 1 -- 172.21.15.44:0/1790874972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 msgr2=0x558442a5c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a5c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcf77fe640 1 -- 172.21.15.44:0/1790874972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a51790 msgr2=0x558442a51b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a51790 0x558442a51b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcf77fe640 1 -- 172.21.15.44:0/1790874972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55844297c620 con 0x558442a5c950 2024-09-13T17:27:53.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a5ed40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009cd0 tx=0x7fdcf00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8b641320846936 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.841 INFO:tasks.workunit.client.0.smithi044.stderr:Updating files: 90% (11438/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-13T17:27:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcdd7fa640 1 -- 172.21.15.44:0/1790874972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00305a0 con 0x558442a5c950 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcdd7fa640 1 -- 172.21.15.44:0/1790874972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcf003f070 con 0x558442a5c950 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.837+0000 7fdcdd7fa640 1 -- 172.21.15.44:0/1790874972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0003bd0 con 0x558442a5c950 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 -- 172.21.15.44:0/1790874972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 msgr2=0x558442a5ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a5ed40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009cd0 tx=0x7fdcf00093e0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 -- 172.21.15.44:0/1790874972 shutdown_connections 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a5c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a5ed40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 --2- 172.21.15.44:0/1790874972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a51790 0x558442a51b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 -- 172.21.15.44:0/1790874972 >> 172.21.15.44:0/1790874972 conn(0x558442a4dd00 msgr2=0x558442a4e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 -- 172.21.15.44:0/1790874972 shutdown_connections 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 -- 172.21.15.44:0/1790874972 wait complete. 2024-09-13T17:27:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 -- start start 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.838+0000 7fdd00214740 1 Processor -- start 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a51790 0x558442a5bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a520b0 0x558442a5c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a59410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442973180 con 0x558442a5c950 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844268d430 con 0x558442a520b0 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584429385e0 con 0x558442a51790 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdd00214740 10 client.0 _ll_register_callbacks cb 0x5584429a4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a59410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a59410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35510/0 (socket says 172.21.15.44:35510) 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 -- 172.21.15.44:0/2018299321 learned_addr learned my addr 172.21.15.44:0/2018299321 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a51790 0x558442a5bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a520b0 0x558442a5c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 -- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a51790 msgr2=0x558442a5bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a51790 0x558442a5bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 -- 172.21.15.44:0/2018299321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a520b0 msgr2=0x558442a5c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a520b0 0x558442a5c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 -- 172.21.15.44:0/2018299321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf810d580 con 0x558442a5c950 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a59410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009bf0 tx=0x7fdcf0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f603c0ab08d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.839+0000 7fdcdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdcdaff5640 1 -- 172.21.15.44:0/2018299321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00318f0 con 0x558442a5c950 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdcdaff5640 1 -- 172.21.15.44:0/2018299321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fdcf0041070 con 0x558442a5c950 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584429deb80 con 0x558442a5c950 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdcdaff5640 1 -- 172.21.15.44:0/2018299321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcf0003970 con 0x558442a5c950 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 10 client.15981 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5584429b33c0 con 0x558442a5c950 2024-09-13T17:27:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdcdaff5640 1 -- 172.21.15.44:0/2018299321 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5584429deb80 con 0x558442a5c950 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdcdaff5640 1 -- 172.21.15.44:0/2018299321 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdcf00046e0 con 0x558442a5c950 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 10 client.15981 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 10 client.15981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 20 client.15981 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442a6e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 4 client.15981 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.840+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584429a71f0 con 0x558442a6c040 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.841+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442a6e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.841+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442a6e430 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24bdcc54be6cf6e9 server_cookie=2ec6590884e5c93c in_seq=0 out_seq=0 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.841+0000 7fdcdaff5640 10 client.15981 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdcdaff5640 1 -- 172.21.15.44:0/2018299321 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5584429a71f0 con 0x558442a6c040 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdcdaff5640 10 client.15981 handle_command_reply: tid=0 2024-09-13T17:27:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 client.15981 shutdown 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 msgr2=0x558442a59410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a59410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009bf0 tx=0x7fdcf0009540 comp rx=0 tx=0).stop 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 shutdown_connections 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442a6e430 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdcdaff5640 0 client.15981 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a5c950 0x558442a59410 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a520b0 0x558442a5c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 --2- 172.21.15.44:0/2018299321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a51790 0x558442a5bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 >> 172.21.15.44:0/2018299321 conn(0x558442a4dd00 msgr2=0x558442a4ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 shutdown_connections 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 1 -- 172.21.15.44:0/2018299321 wait complete. 2024-09-13T17:27:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.842+0000 7fdd00214740 20 client.15981 trim_cache size 0 max 16384 2024-09-13T17:27:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 Processor -- start 2024-09-13T17:27:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 -- start start 2024-09-13T17:27:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a72d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a63620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442a6be60 con 0x558442a51790 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442a58180 con 0x558442a5c950 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdd00214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442a576f0 con 0x558442a520b0 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdcf6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a72d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdcf6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a72d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35520/0 (socket says 172.21.15.44:35520) 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.853+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/2473358455 learned_addr learned my addr 172.21.15.44:0/2473358455 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a63620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a63b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/2473358455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 msgr2=0x558442a63620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a63620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/2473358455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 msgr2=0x558442a63b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a63b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/2473358455 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558442955770 con 0x558442a51790 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a72d60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009600 tx=0x7fdce8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aec820b0ee9f7eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcddffb640 1 -- 172.21.15.44:0/2473358455 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8025040 con 0x558442a51790 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcddffb640 1 -- 172.21.15.44:0/2473358455 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdce8016070 con 0x558442a51790 2024-09-13T17:27:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.854+0000 7fdcddffb640 1 -- 172.21.15.44:0/2473358455 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800f3d0 con 0x558442a51790 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 -- 172.21.15.44:0/2473358455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 msgr2=0x558442a72d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a72d60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009600 tx=0x7fdce8014000 comp rx=0 tx=0).stop 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 -- 172.21.15.44:0/2473358455 shutdown_connections 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a63620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a63b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 --2- 172.21.15.44:0/2473358455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a72d60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 -- 172.21.15.44:0/2473358455 >> 172.21.15.44:0/2473358455 conn(0x558442a4dd00 msgr2=0x558442a57a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 -- 172.21.15.44:0/2473358455 shutdown_connections 2024-09-13T17:27:53.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.855+0000 7fdd00214740 1 -- 172.21.15.44:0/2473358455 wait complete. 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 -- start start 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 Processor -- start 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a63970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a61de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a62320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442a62a20 con 0x558442a51790 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442a62b90 con 0x558442a5c950 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558442ac24f0 con 0x558442a520b0 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdd00214740 10 client.0 _ll_register_callbacks cb 0x7fdcec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdcf6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a63970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdcf6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a63970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35522/0 (socket says 172.21.15.44:35522) 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/58558939 learned_addr learned my addr 172.21.15.44:0/58558939 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:53.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.856+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a61de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a62320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 msgr2=0x558442a61de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a61de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/58558939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 msgr2=0x558442a62320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a62320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf6ffd640 1 -- 172.21.15.44:0/58558939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce8011e70 con 0x558442a51790 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf77fe640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcf6ffd640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a63970 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdce8011870 tx=0x7fdce80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b2b599169937fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcdaff5640 1 -- 172.21.15.44:0/58558939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8025040 con 0x558442a51790 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcf003f070 con 0x558442a51790 2024-09-13T17:27:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcdaff5640 1 -- 172.21.15.44:0/58558939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fdce8018070 con 0x558442a51790 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcdaff5640 1 -- 172.21.15.44:0/58558939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdce802a830 con 0x558442a51790 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdd00214740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55844297ae80 con 0x558442a51790 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.857+0000 7fdcdaff5640 1 -- 172.21.15.44:0/58558939 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcf003f070 con 0x558442a51790 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdcdaff5640 1 -- 172.21.15.44:0/58558939 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdce8033cb0 con 0x558442a51790 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 10 client.15987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 20 client.15987 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442ac5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 4 client.15987 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584426a84e0 con 0x558442a6c040 2024-09-13T17:27:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442ac5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdcdaff5640 10 client.15987 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdcf67fc640 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442ac5bc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5941ca16f76f91cd server_cookie=3163c91c56c3b60 in_seq=0 out_seq=0 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdcdaff5640 1 -- 172.21.15.44:0/58558939 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584426a84e0 con 0x558442a6c040 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdcdaff5640 10 client.15987 handle_command_reply: tid=0 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.858+0000 7fdd00214740 1 client.15987 shutdown 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 msgr2=0x558442a63970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a63970 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdce8011870 tx=0x7fdce80056d0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 shutdown_connections 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x558442a6c040 0x558442ac5bc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558442a520b0 0x558442a61de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558442a5c950 0x558442a62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdcdaff5640 0 client.15987 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:27:53.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 --2- 172.21.15.44:0/58558939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558442a51790 0x558442a63970 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 >> 172.21.15.44:0/58558939 conn(0x558442a4dd00 msgr2=0x558442a586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 shutdown_connections 2024-09-13T17:27:53.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 1 -- 172.21.15.44:0/58558939 wait complete. 2024-09-13T17:27:53.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdd00214740 20 client.15987 trim_cache size 0 max 16384 2024-09-13T17:27:53.867 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:27:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdce00a13b0 msgr2=0x7fdce00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdcfd419640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdce00a13b0 0x7fdce00a3870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009b60 tx=0x7fdce8005f90 comp rx=0 tx=0).stop 2024-09-13T17:27:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 msgr2=0x7fdcf8196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.859+0000 7fdcfd419640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf8196830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e9f0 tx=0x7fdcec00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 shutdown_connections 2024-09-13T17:27:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdce00a13b0 0x7fdce00a3870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcf819ac20 0x7fdcf819b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf8106440 0x7fdcf8196830 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 --2- 172.21.15.44:0/62860294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcf80feb80 0x7fdcf81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 >> 172.21.15.44:0/62860294 conn(0x7fdcf8074110 msgr2=0x7fdcf8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 shutdown_connections 2024-09-13T17:27:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:53.860+0000 7fdcfd419640 1 -- 172.21.15.44:0/62860294 wait complete. 2024-09-13T17:27:53.872 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:27:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 -- 172.21.15.44:0/1276459159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854100f40 msgr2=0x7f9854101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 --2- 172.21.15.44:0/1276459159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854100f40 0x7f9854101320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9844009900 tx=0x7f984402a480 comp rx=0 tx=0).stop 2024-09-13T17:27:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 -- 172.21.15.44:0/1276459159 shutdown_connections 2024-09-13T17:27:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 --2- 172.21.15.44:0/1276459159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854101860 0x7f9854109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 --2- 172.21.15.44:0/1276459159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 0x7f985410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 --2- 172.21.15.44:0/1276459159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854100f40 0x7f9854101320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 -- 172.21.15.44:0/1276459159 >> 172.21.15.44:0/1276459159 conn(0x7f9854075c60 msgr2=0x7f98540fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 -- 172.21.15.44:0/1276459159 shutdown_connections 2024-09-13T17:27:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 -- 172.21.15.44:0/1276459159 wait complete. 2024-09-13T17:27:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.017+0000 7f98591db640 1 Processor -- start 2024-09-13T17:27:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 -- start start 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854100f40 0x7f9854196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 0x7f9854196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 0x7f985419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985419b030 con 0x7f9854101860 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985419b1a0 con 0x7f985410a310 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f98591db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985419b360 con 0x7f9854100f40 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 0x7f9854196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 0x7f9854196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35542/0 (socket says 172.21.15.44:35542) 2024-09-13T17:27:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 -- 172.21.15.44:0/3851163179 learned_addr learned my addr 172.21.15.44:0/3851163179 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f9852d9d640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854100f40 0x7f9854196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985359e640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 0x7f985419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 -- 172.21.15.44:0/3851163179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854100f40 msgr2=0x7f9854196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854100f40 0x7f9854196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 -- 172.21.15.44:0/3851163179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 msgr2=0x7f985419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 0x7f985419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985259c640 1 -- 172.21.15.44:0/3851163179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009560 con 0x7f9854101860 2024-09-13T17:27:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.018+0000 7f985359e640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 0x7f985419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.019+0000 7f985259c640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 0x7f9854196640 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f984800eb90 tx=0x7f984800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.019+0000 7f983bfff640 1 -- 172.21.15.44:0/3851163179 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480099b0 con 0x7f9854101860 2024-09-13T17:27:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.019+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985419b640 con 0x7f9854101860 2024-09-13T17:27:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.019+0000 7f983bfff640 1 -- 172.21.15.44:0/3851163179 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f984800ee90 con 0x7f9854101860 2024-09-13T17:27:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.019+0000 7f983bfff640 1 -- 172.21.15.44:0/3851163179 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848010690 con 0x7f9854101860 2024-09-13T17:27:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.019+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98541a2ec0 con 0x7f9854101860 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.021+0000 7f983bfff640 1 -- 172.21.15.44:0/3851163179 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f9848005020 con 0x7f9854101860 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.021+0000 7f985bfd8740 1 Processor -- start 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 -- start start 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992eb7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992e21cc0 con 0x561992ec2950 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992e1f450 con 0x561992eb80b0 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985bfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992e1ad40 con 0x561992eb7790 2024-09-13T17:27:54.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992eb7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992eb7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:43774/0 (socket says 172.21.15.44:43774) 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 -- 172.21.15.44:0/113745852 learned_addr learned my addr 172.21.15.44:0/113745852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f9852d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f983bfff640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f983c0a13b0 0x7f983c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 -- 172.21.15.44:0/113745852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 msgr2=0x561992ec2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 -- 172.21.15.44:0/113745852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 msgr2=0x561992ec4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f985359e640 1 -- 172.21.15.44:0/113745852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561992de2620 con 0x561992eb7790 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.022+0000 7f983bfff640 1 -- 172.21.15.44:0/3851163179 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9848014070 con 0x7f9854101860 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.023+0000 7f985359e640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f983c0a13b0 0x7f983c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.023+0000 7f985359e640 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992eb7b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f984c00cd40 tx=0x7f984c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e81f06baff2a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.023+0000 7f98397fa640 1 -- 172.21.15.44:0/113745852 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c0124b0 con 0x561992eb7790 2024-09-13T17:27:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.023+0000 7f98397fa640 1 -- 172.21.15.44:0/113745852 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f984c01c470 con 0x561992eb7790 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.023+0000 7f98397fa640 1 -- 172.21.15.44:0/113745852 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c01a660 con 0x561992eb7790 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.023+0000 7f985359e640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f983c0a13b0 0x7f983c0a3870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f984c005940 tx=0x7f984c020000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 -- 172.21.15.44:0/113745852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 msgr2=0x561992eb7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992eb7b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f984c00cd40 tx=0x7f984c00e720 comp rx=0 tx=0).stop 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 -- 172.21.15.44:0/113745852 shutdown_connections 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 --2- 172.21.15.44:0/113745852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992eb7b70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 -- 172.21.15.44:0/113745852 >> 172.21.15.44:0/113745852 conn(0x561992eb3d00 msgr2=0x561992eb40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 -- 172.21.15.44:0/113745852 shutdown_connections 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 -- 172.21.15.44:0/113745852 wait complete. 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 -- start start 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.024+0000 7f985bfd8740 1 Processor -- start 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ebf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ebf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992dd9180 con 0x561992ec2950 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992af3430 con 0x561992eb80b0 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992d9e5e0 con 0x561992eb7790 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 10 client.0 _ll_register_callbacks cb 0x561992e0aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ebf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ebf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35560/0 (socket says 172.21.15.44:35560) 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f9852d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 -- 172.21.15.44:0/1369231120 learned_addr learned my addr 172.21.15.44:0/1369231120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985259c640 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ebf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 -- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 msgr2=0x561992ec1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 -- 172.21.15.44:0/1369231120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 msgr2=0x561992ebf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ebf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 -- 172.21.15.44:0/1369231120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c00c870 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985359e640 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ebf7a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f984c014010 tx=0x7f984c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0abff2855858e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f9836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f9836ff5640 1 -- 172.21.15.44:0/1369231120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c02c070 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f9836ff5640 1 -- 172.21.15.44:0/1369231120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f984c012450 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.025+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561992e44b80 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f9836ff5640 1 -- 172.21.15.44:0/1369231120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984c011830 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 10 client.15999 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561992e193c0 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f9836ff5640 1 -- 172.21.15.44:0/1369231120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561992e44b80 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f9836ff5640 1 -- 172.21.15.44:0/1369231120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f984c01ac80 con 0x561992ec2950 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 10 client.15999 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:54.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 20 client.15999 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992ed2190 0x561992ed4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 4 client.15999 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561992e0d1f0 con 0x561992ed2190 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.026+0000 7f9852d9d640 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992ed2190 0x561992ed4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.027+0000 7f9836ff5640 10 client.15999 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.027+0000 7f9852d9d640 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992ed2190 0x561992ed4580 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b8d687dd38006f server_cookie=f492882b2e0d6c15 in_seq=0 out_seq=0 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.027+0000 7f9836ff5640 1 -- 172.21.15.44:0/1369231120 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561992e0d1f0 con 0x561992ed2190 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.027+0000 7f9836ff5640 10 client.15999 handle_command_reply: tid=0 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 client.15999 shutdown 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 msgr2=0x561992ebf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ebf7a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f984c014010 tx=0x7f984c01c810 comp rx=0 tx=0).stop 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 shutdown_connections 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992ed2190 0x561992ed4580 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ebf7a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ebf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f9836ff5640 0 client.15999 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1369231120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 >> 172.21.15.44:0/1369231120 conn(0x561992eb3d00 msgr2=0x561992eb2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 shutdown_connections 2024-09-13T17:27:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 1 -- 172.21.15.44:0/1369231120 wait complete. 2024-09-13T17:27:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.028+0000 7f985bfd8740 20 client.15999 trim_cache size 0 max 16384 2024-09-13T17:27:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 Processor -- start 2024-09-13T17:27:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 -- start start 2024-09-13T17:27:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ebdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 0x561992ebe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992ec2950 0x561992eba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992ebe590 con 0x561992eb80b0 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992ec06d0 con 0x561992ec2950 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985bfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992ebfae0 con 0x561992eb7790 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 0x561992ebe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 0x561992ebe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35572/0 (socket says 172.21.15.44:35572) 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985259c640 1 -- 172.21.15.44:0/1641100609 learned_addr learned my addr 172.21.15.44:0/1641100609 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f985359e640 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992ec2950 0x561992eba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.039+0000 7f9852d9d640 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ebdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f985259c640 1 -- 172.21.15.44:0/1641100609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 msgr2=0x561992ebdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f985259c640 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ebdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f985259c640 1 -- 172.21.15.44:0/1641100609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992ec2950 msgr2=0x561992eba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f985259c640 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992ec2950 0x561992eba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f985259c640 1 -- 172.21.15.44:0/1641100609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561992dbb770 con 0x561992eb80b0 2024-09-13T17:27:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f9852d9d640 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ebdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f985259c640 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 0x561992ebe050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98480049d0 tx=0x7f98480ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3f6f3c96431eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f9839ffb640 1 -- 172.21.15.44:0/1641100609 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480ba0b0 con 0x561992eb80b0 2024-09-13T17:27:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f9839ffb640 1 -- 172.21.15.44:0/1641100609 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f984809ad30 con 0x561992eb80b0 2024-09-13T17:27:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.040+0000 7f9839ffb640 1 -- 172.21.15.44:0/1641100609 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480a2ab0 con 0x561992eb80b0 2024-09-13T17:27:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 -- 172.21.15.44:0/1641100609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 msgr2=0x561992ebe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 0x561992ebe050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98480049d0 tx=0x7f98480ba040 comp rx=0 tx=0).stop 2024-09-13T17:27:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 -- 172.21.15.44:0/1641100609 shutdown_connections 2024-09-13T17:27:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992ec2950 0x561992eba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992eb80b0 0x561992ebe050 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 --2- 172.21.15.44:0/1641100609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ebdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 -- 172.21.15.44:0/1641100609 >> 172.21.15.44:0/1641100609 conn(0x561992eb3d00 msgr2=0x561992ed8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 -- 172.21.15.44:0/1641100609 shutdown_connections 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 -- 172.21.15.44:0/1641100609 wait complete. 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 -- start start 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.041+0000 7f985bfd8740 1 Processor -- start 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992ec8bb0 con 0x561992ec2950 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992ec8d20 con 0x561992eb80b0 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561992f28640 con 0x561992eb7790 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985bfd8740 10 client.0 _ll_register_callbacks cb 0x7f9854069b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35576/0 (socket says 172.21.15.44:35576) 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 -- 172.21.15.44:0/2327508090 learned_addr learned my addr 172.21.15.44:0/2327508090 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985259c640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f9852d9d640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 -- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 msgr2=0x561992ec9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 -- 172.21.15.44:0/2327508090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 msgr2=0x561992ec7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 -- 172.21.15.44:0/2327508090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98541a2ec0 con 0x561992ec2950 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f9852d9d640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f985359e640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec84b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f984c014010 tx=0x7f984c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72953129774a3046 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.042+0000 7f9836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f9836ff5640 1 -- 172.21.15.44:0/2327508090 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c02c040 con 0x561992ec2950 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f9836ff5640 1 -- 172.21.15.44:0/2327508090 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f984c012450 con 0x561992ec2950 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f984c01c470 con 0x561992ec2950 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f9836ff5640 1 -- 172.21.15.44:0/2327508090 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984c004640 con 0x561992ec2950 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561992de0e80 con 0x561992ec2950 2024-09-13T17:27:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f9836ff5640 1 -- 172.21.15.44:0/2327508090 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f984c01c470 con 0x561992ec2950 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f9836ff5640 1 -- 172.21.15.44:0/2327508090 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f984c038040 con 0x561992ec2950 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 20 client.16005 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992f298c0 0x561992f2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 4 client.16005 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.043+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561992b0e4e0 con 0x561992f298c0 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f9852d9d640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992f298c0 0x561992f2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f9852d9d640 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992f298c0 0x561992f2bcb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b284b93385a415 server_cookie=497206f49486721c in_seq=0 out_seq=0 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f9836ff5640 10 client.16005 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f9836ff5640 1 -- 172.21.15.44:0/2327508090 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561992b0e4e0 con 0x561992f298c0 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f9836ff5640 10 client.16005 handle_command_reply: tid=0 2024-09-13T17:27:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 client.16005 shutdown 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 msgr2=0x561992ec84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec84b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f984c014010 tx=0x7f984c005c40 comp rx=0 tx=0).stop 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 shutdown_connections 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561992f298c0 0x561992f2bcb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561992ec2950 0x561992ec84b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561992eb80b0 0x561992ec7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f9836ff5640 0 client.16005 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 --2- 172.21.15.44:0/2327508090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561992eb7790 0x561992ec9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 >> 172.21.15.44:0/2327508090 conn(0x561992eb3d00 msgr2=0x561992ece0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.044+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 shutdown_connections 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f985bfd8740 1 -- 172.21.15.44:0/2327508090 wait complete. 2024-09-13T17:27:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f985bfd8740 20 client.16005 trim_cache size 0 max 16384 2024-09-13T17:27:54.053 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:27:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f983c0a13b0 msgr2=0x7f983c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f983c0a13b0 0x7f983c0a3870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f984c005940 tx=0x7f984c020000 comp rx=0 tx=0).stop 2024-09-13T17:27:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 msgr2=0x7f9854196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 0x7f9854196640 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f984800eb90 tx=0x7f984800c6a0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 shutdown_connections 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f983c0a13b0 0x7f983c0a3870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f985410a310 0x7f985419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9854101860 0x7f9854196640 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 --2- 172.21.15.44:0/3851163179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854100f40 0x7f9854196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 >> 172.21.15.44:0/3851163179 conn(0x7f9854075c60 msgr2=0x7f98540fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 shutdown_connections 2024-09-13T17:27:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:54.045+0000 7f98591db640 1 -- 172.21.15.44:0/3851163179 wait complete. 2024-09-13T17:27:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:54 smithi070 ceph-mon[34502]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:54 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3784523210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:27:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:54 smithi100 ceph-mon[35055]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:54.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:54 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3784523210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:27:54.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:54 smithi044 ceph-mon[31175]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:54.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:54 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3784523210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:27:54.881 INFO:tasks.workunit.client.0.smithi044.stderr:Note: switching to '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'. 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: git switch -c 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr:Or undo this operation with: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.882 INFO:tasks.workunit.client.0.smithi044.stderr: git switch - 2024-09-13T17:27:54.883 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.883 INFO:tasks.workunit.client.0.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T17:27:54.883 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-09-13T17:27:54.883 INFO:tasks.workunit.client.0.smithi044.stderr:HEAD is now at 6c6b083f148 Merge PR #56690 into wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:27:54.888 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T17:27:54.920 INFO:tasks.workunit.client.0.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T17:27:54.922 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T17:27:54.923 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T17:27:55.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:27:55.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:27:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3549157308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac101800 msgr2=0x7f99ac109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3549157308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac101800 0x7f99ac109ce0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009900 tx=0x7f99a002a410 comp rx=0 tx=0).stop 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3549157308 shutdown_connections 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3549157308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 0x7f99ac10c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3549157308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac101800 0x7f99ac109ce0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3549157308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac100e50 0x7f99ac101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3549157308 >> 172.21.15.44:0/3549157308 conn(0x7f99ac0fa930 msgr2=0x7f99ac0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3549157308 shutdown_connections 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.165+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3549157308 wait complete. 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 Processor -- start 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 -- start start 2024-09-13T17:27:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 0x7f99ac196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac101800 0x7f99ac196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 0x7f99ac19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19b160 con 0x7f99ac100e50 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19b2d0 con 0x7f99ac10a350 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99b1fdb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19b490 con 0x7f99ac101800 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 0x7f99ac196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 0x7f99ac196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35600/0 (socket says 172.21.15.44:35600) 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.166+0000 7f99ab7fe640 1 -- 172.21.15.44:0/3551341540 learned_addr learned my addr 172.21.15.44:0/3551341540 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99aaffd640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac101800 0x7f99ac196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99abfff640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 0x7f99ac19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99ab7fe640 1 -- 172.21.15.44:0/3551341540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac101800 msgr2=0x7f99ac196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac101800 0x7f99ac196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99ab7fe640 1 -- 172.21.15.44:0/3551341540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 msgr2=0x7f99ac19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 0x7f99ac19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99ab7fe640 1 -- 172.21.15.44:0/3551341540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009560 con 0x7f99ac100e50 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99aaffd640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac101800 0x7f99ac196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99abfff640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 0x7f99ac19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 0x7f99ac196230 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f999c00b4a0 tx=0x7f999c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99a8ff9640 1 -- 172.21.15.44:0/3551341540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c010610 con 0x7f99ac100e50 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99a8ff9640 1 -- 172.21.15.44:0/3551341540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f999c010c00 con 0x7f99ac100e50 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac19b770 con 0x7f99ac100e50 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99a8ff9640 1 -- 172.21.15.44:0/3551341540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c017ca0 con 0x7f99ac100e50 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.167+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac1a2ff0 con 0x7f99ac100e50 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.169+0000 7f99a8ff9640 1 -- 172.21.15.44:0/3551341540 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f999c010770 con 0x7f99ac100e50 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 Processor -- start 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 -- start start 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99a8ff9640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f99940a13b0 0x7f99940a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 0x555bd8431ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84320e0 0x555bd843c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd843ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd839cc60 con 0x555bd843c980 2024-09-13T17:27:55.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd839a3f0 con 0x555bd84320e0 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8395ce0 con 0x555bd84317c0 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99abfff640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f99940a13b0 0x7f99940a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99a8ff9640 1 -- 172.21.15.44:0/3551341540 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f999c0ccc60 con 0x7f99ac100e50 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 0x555bd8431ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 0x555bd8431ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:43814/0 (socket says 172.21.15.44:43814) 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99ab7fe640 1 -- 172.21.15.44:0/53659294 learned_addr learned my addr 172.21.15.44:0/53659294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99abfff640 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd843ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.170+0000 7f99aaffd640 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84320e0 0x555bd843c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99ab7fe640 1 -- 172.21.15.44:0/53659294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84320e0 msgr2=0x555bd843c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84320e0 0x555bd843c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99ab7fe640 1 -- 172.21.15.44:0/53659294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 msgr2=0x555bd843ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd843ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99ab7fe640 1 -- 172.21.15.44:0/53659294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd8351970 con 0x555bd84317c0 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99abfff640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f99940a13b0 0x7f99940a3870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99a400eba0 tx=0x7f99a4009d70 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 0x555bd8431ba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f999c0ca050 tx=0x7f999c095d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a54300350f153f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f9991ffb640 1 -- 172.21.15.44:0/53659294 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c017360 con 0x555bd84317c0 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f9991ffb640 1 -- 172.21.15.44:0/53659294 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f999c09a020 con 0x555bd84317c0 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.171+0000 7f9991ffb640 1 -- 172.21.15.44:0/53659294 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c09e4a0 con 0x555bd84317c0 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- 172.21.15.44:0/53659294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 msgr2=0x555bd8431ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 0x555bd8431ba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f999c0ca050 tx=0x7f999c095d90 comp rx=0 tx=0).stop 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- 172.21.15.44:0/53659294 shutdown_connections 2024-09-13T17:27:55.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd843ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84320e0 0x555bd843c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/53659294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84317c0 0x555bd8431ba0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- 172.21.15.44:0/53659294 >> 172.21.15.44:0/53659294 conn(0x555bd842dd50 msgr2=0x555bd842e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- 172.21.15.44:0/53659294 shutdown_connections 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- 172.21.15.44:0/53659294 wait complete. 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- start start 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 Processor -- start 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 0x555bd843c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd8439560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd843c980 0x555bd8439b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8353f80 con 0x555bd84317c0 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8353c20 con 0x555bd843c980 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.172+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8320ab0 con 0x555bd84320e0 2024-09-13T17:27:55.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99b4dd7740 10 client.0 _ll_register_callbacks cb 0x555bd838f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 0x555bd843c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 0x555bd843c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35610/0 (socket says 172.21.15.44:35610) 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 -- 172.21.15.44:0/2858687532 learned_addr learned my addr 172.21.15.44:0/2858687532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd8439560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99abfff640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd843c980 0x555bd8439b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 -- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 msgr2=0x555bd8439560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd8439560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 -- 172.21.15.44:0/2858687532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd843c980 msgr2=0x555bd8439b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd843c980 0x555bd8439b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 -- 172.21.15.44:0/2858687532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac1a2ff0 con 0x555bd84317c0 2024-09-13T17:27:55.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd8439560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99abfff640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd843c980 0x555bd8439b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 0x555bd843c330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f999c00be60 tx=0x7f999c09a190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bbb502e1e655f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f998affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f998affd640 1 -- 172.21.15.44:0/2858687532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c09d750 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f998affd640 1 -- 172.21.15.44:0/2858687532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f999c010a20 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bd8089590 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f998affd640 1 -- 172.21.15.44:0/2858687532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c09ee50 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99b4dd7740 10 client.16017 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.173+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555bd8394360 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f998affd640 1 -- 172.21.15.44:0/2858687532 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555bd8089590 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f998affd640 1 -- 172.21.15.44:0/2858687532 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f999c021070 con 0x555bd84317c0 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99b4dd7740 10 client.16017 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:55.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99b4dd7740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99b4dd7740 20 client.16017 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd844c1b0 0x555bd844e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99b4dd7740 4 client.16017 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bd809a860 con 0x555bd844c1b0 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd844c1b0 0x555bd844e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f998affd640 10 client.16017 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.174+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd844c1b0 0x555bd844e5a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a02b5fe081c455 server_cookie=531595240f90eae in_seq=0 out_seq=0 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f998affd640 1 -- 172.21.15.44:0/2858687532 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555bd809a860 con 0x555bd844c1b0 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f998affd640 10 client.16017 handle_command_reply: tid=0 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 client.16017 shutdown 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 msgr2=0x555bd843c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 0x555bd843c330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f999c00be60 tx=0x7f999c09a190 comp rx=0 tx=0).stop 2024-09-13T17:27:55.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 shutdown_connections 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd844c1b0 0x555bd844e5a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd8439560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd843c980 0x555bd8439b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2858687532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84317c0 0x555bd843c330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f998affd640 0 client.16017 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 >> 172.21.15.44:0/2858687532 conn(0x555bd842dd50 msgr2=0x555bd84331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 shutdown_connections 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2858687532 wait complete. 2024-09-13T17:27:55.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.175+0000 7f99b4dd7740 20 client.16017 trim_cache size 0 max 16384 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.189+0000 7f99b4dd7740 1 Processor -- start 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 -- start start 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd8439510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 0x555bd8443650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd843c980 0x555bd8443b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd84493c0 con 0x555bd84320e0 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8448560 con 0x555bd84317c0 2024-09-13T17:27:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8431ed0 con 0x555bd843c980 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 0x555bd8443650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 0x555bd8443650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35612/0 (socket says 172.21.15.44:35612) 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 -- 172.21.15.44:0/2968997303 learned_addr learned my addr 172.21.15.44:0/2968997303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd8439510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99abfff640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd843c980 0x555bd8443b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 -- 172.21.15.44:0/2968997303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd843c980 msgr2=0x555bd8443b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd843c980 0x555bd8443b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 -- 172.21.15.44:0/2968997303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 msgr2=0x555bd8439510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd8439510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 -- 172.21.15.44:0/2968997303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd83361c0 con 0x555bd84320e0 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd8439510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99abfff640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd843c980 0x555bd8443b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:27:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.190+0000 7f99aaffd640 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 0x555bd8443650 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f99a002a930 tx=0x7f99a0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee2801d9515b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99927fc640 1 -- 172.21.15.44:0/2968997303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0042040 con 0x555bd84320e0 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99927fc640 1 -- 172.21.15.44:0/2968997303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99a0005490 con 0x555bd84320e0 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99927fc640 1 -- 172.21.15.44:0/2968997303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a00442c0 con 0x555bd84320e0 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2968997303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 msgr2=0x555bd8443650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 0x555bd8443650 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f99a002a930 tx=0x7f99a0005e30 comp rx=0 tx=0).stop 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2968997303 shutdown_connections 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd843c980 0x555bd8443b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd84320e0 0x555bd8443650 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/2968997303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd8439510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2968997303 >> 172.21.15.44:0/2968997303 conn(0x555bd842dd50 msgr2=0x555bd8438e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2968997303 shutdown_connections 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 -- 172.21.15.44:0/2968997303 wait complete. 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.191+0000 7f99b4dd7740 1 -- start start 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 Processor -- start 2024-09-13T17:27:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd849c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd849cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd8442810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8442f10 con 0x555bd843c980 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8443080 con 0x555bd84317c0 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd8443240 con 0x555bd84320e0 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99b4dd7740 10 client.0 _ll_register_callbacks cb 0x7f99a400e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd849c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd8442810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd849c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:46154/0 (socket says 172.21.15.44:46154) 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99ab7fe640 1 -- 172.21.15.44:0/945456396 learned_addr learned my addr 172.21.15.44:0/945456396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99aaffd640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd849cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99abfff640 1 -- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 msgr2=0x555bd849cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99abfff640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd849cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99abfff640 1 -- 172.21.15.44:0/945456396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 msgr2=0x555bd849c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99abfff640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd849c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99abfff640 1 -- 172.21.15.44:0/945456396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac10d420 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.192+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd849c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99abfff640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd8442810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x555bd84287d0 tx=0x7f99a4008220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea02b6e90a3125 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f998affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f998affd640 1 -- 172.21.15.44:0/945456396 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40097a0 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f998affd640 1 -- 172.21.15.44:0/945456396 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f99a400cd20 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f999c09a020 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f998affd640 1 -- 172.21.15.44:0/945456396 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a400cee0 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 10 client.16023 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555bd835b990 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f998affd640 1 -- 172.21.15.44:0/945456396 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f999c09a020 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f998affd640 1 -- 172.21.15.44:0/945456396 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f99a401a3f0 con 0x555bd843c980 2024-09-13T17:27:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 20 client.16023 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd84a4790 0x555bd84a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 4 client.16023 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.193+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555bd80edeb0 con 0x555bd84a4790 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.194+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd84a4790 0x555bd84a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.194+0000 7f99ab7fe640 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd84a4790 0x555bd84a6bb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52db7fe345e5bdda server_cookie=9cbe1b78170506a8 in_seq=0 out_seq=0 2024-09-13T17:27:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.194+0000 7f998affd640 10 client.16023 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.195+0000 7f998affd640 1 -- 172.21.15.44:0/945456396 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555bd80edeb0 con 0x555bd84a4790 2024-09-13T17:27:55.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.195+0000 7f998affd640 10 client.16023 handle_command_reply: tid=0 2024-09-13T17:27:55.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.195+0000 7f99b4dd7740 1 client.16023 shutdown 2024-09-13T17:27:55.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.195+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 msgr2=0x555bd8442810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.195+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd8442810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x555bd84287d0 tx=0x7f99a4008220 comp rx=0 tx=0).stop 2024-09-13T17:27:55.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 shutdown_connections 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555bd84a4790 0x555bd84a6bb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bd84320e0 0x555bd849cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bd843c980 0x555bd8442810 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f998affd640 0 client.16023 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 --2- 172.21.15.44:0/945456396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bd84317c0 0x555bd849c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 >> 172.21.15.44:0/945456396 conn(0x555bd842dd50 msgr2=0x555bd842c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 shutdown_connections 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 1 -- 172.21.15.44:0/945456396 wait complete. 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b4dd7740 20 client.16023 trim_cache size 0 max 16384 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:27:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:27:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "0aebdb94-e447-4775-af72-f758c8ae39da", 2024-09-13T17:27:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:27:55.203 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:27:55.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f99940a13b0 msgr2=0x7f99940a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f99940a13b0 0x7f99940a3870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99a400eba0 tx=0x7f99a4009d70 comp rx=0 tx=0).stop 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 msgr2=0x7f99ac196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 0x7f99ac196230 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f999c00b4a0 tx=0x7f999c00b960 comp rx=0 tx=0).stop 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 shutdown_connections 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f99940a13b0 0x7f99940a3870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac101800 0x7f99ac196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99ac10a350 0x7f99ac19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 --2- 172.21.15.44:0/3551341540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99ac100e50 0x7f99ac196230 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 >> 172.21.15.44:0/3551341540 conn(0x7f99ac0fa930 msgr2=0x7f99ac0fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 shutdown_connections 2024-09-13T17:27:55.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.196+0000 7f99b1fdb640 1 -- 172.21.15.44:0/3551341540 wait complete. 2024-09-13T17:27:55.208 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 -- 172.21.15.44:0/3216014867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd0103fa0 msgr2=0x7f5cd010baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 --2- 172.21.15.44:0/3216014867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd0103fa0 0x7f5cd010baf0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009cd0 tx=0x7f5cc802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 -- 172.21.15.44:0/3216014867 shutdown_connections 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 --2- 172.21.15.44:0/3216014867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd00ff800 0x7f5cd0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 --2- 172.21.15.44:0/3216014867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd0103fa0 0x7f5cd010baf0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 --2- 172.21.15.44:0/3216014867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 0x7f5cd00ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 -- 172.21.15.44:0/3216014867 >> 172.21.15.44:0/3216014867 conn(0x7f5cd00fa8c0 msgr2=0x7f5cd00fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 -- 172.21.15.44:0/3216014867 shutdown_connections 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.311+0000 7f5cd655e640 1 -- 172.21.15.44:0/3216014867 wait complete. 2024-09-13T17:27:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 Processor -- start 2024-09-13T17:27:55.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 -- start start 2024-09-13T17:27:55.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 0x7f5cd01961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd0103fa0 0x7f5cd0196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 0x7f5cd019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd019b510 con 0x7f5cd019ab20 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd019b680 con 0x7f5cd00fee50 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd655e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd019b7f0 con 0x7f5cd0103fa0 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd4afb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 0x7f5cd019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd4afb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 0x7f5cd019afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35650/0 (socket says 172.21.15.44:35650) 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5cd4afb640 1 -- 172.21.15.44:0/2544934389 learned_addr learned my addr 172.21.15.44:0/2544934389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.312+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd0103fa0 0x7f5cd0196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5ccffff640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 0x7f5cd01961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd4afb640 1 -- 172.21.15.44:0/2544934389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd0103fa0 msgr2=0x7f5cd0196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd4afb640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd0103fa0 0x7f5cd0196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd4afb640 1 -- 172.21.15.44:0/2544934389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 msgr2=0x7f5cd01961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd4afb640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 0x7f5cd01961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd4afb640 1 -- 172.21.15.44:0/2544934389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8009900 con 0x7f5cd019ab20 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5ccffff640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 0x7f5cd01961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd4afb640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 0x7f5cd019afd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8004af0 tx=0x7f5cc8030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5ccd7fa640 1 -- 172.21.15.44:0/2544934389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8031b80 con 0x7f5cd019ab20 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5ccd7fa640 1 -- 172.21.15.44:0/2544934389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cc8003e00 con 0x7f5cd019ab20 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd01a2dd0 con 0x7f5cd019ab20 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5ccd7fa640 1 -- 172.21.15.44:0/2544934389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8045410 con 0x7f5cd019ab20 2024-09-13T17:27:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.313+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd01a3250 con 0x7f5cd019ab20 2024-09-13T17:27:55.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.315+0000 7f5ccd7fa640 1 -- 172.21.15.44:0/2544934389 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f5cc800a040 con 0x7f5cd019ab20 2024-09-13T17:27:55.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 Processor -- start 2024-09-13T17:27:55.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 -- start start 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae17c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae186a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae0e6f20 con 0x5581ae17be10 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae0e46b0 con 0x5581ae186fd0 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd9286740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae0dffa0 con 0x5581ae17c730 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae17c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae17c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35652/0 (socket says 172.21.15.44:35652) 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/2088856080 learned_addr learned my addr 172.21.15.44:0/2088856080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccd7fa640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5cb80a13b0 0x7f5cb80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccffff640 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae186a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/2088856080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 msgr2=0x5581ae186a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae186a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/2088856080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 msgr2=0x5581ae1893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/2088856080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ae09a840 con 0x5581ae17be10 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5ccd7fa640 1 -- 172.21.15.44:0/2544934389 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5cc80ee710 con 0x7f5cd019ab20 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.316+0000 7f5cd4afb640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5cb80a13b0 0x7f5cb80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.317+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae17c1f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bf60 tx=0x7f5cc400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a26849344fd3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.317+0000 7f5cb67fc640 1 -- 172.21.15.44:0/2088856080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004020 con 0x5581ae17be10 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.317+0000 7f5cb67fc640 1 -- 172.21.15.44:0/2088856080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cc4005020 con 0x5581ae17be10 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.317+0000 7f5cd4afb640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5cb80a13b0 0x7f5cb80a3870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80eb960 tx=0x7f5cc80a6040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.317+0000 7f5cb67fc640 1 -- 172.21.15.44:0/2088856080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004a00 con 0x5581ae17be10 2024-09-13T17:27:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.317+0000 7f5cd9286740 1 -- 172.21.15.44:0/2088856080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 msgr2=0x5581ae17c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae17c1f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bf60 tx=0x7f5cc400b650 comp rx=0 tx=0).stop 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- 172.21.15.44:0/2088856080 shutdown_connections 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae186a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- 172.21.15.44:0/2088856080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae17c1f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- 172.21.15.44:0/2088856080 >> 172.21.15.44:0/2088856080 conn(0x5581ae178380 msgr2=0x5581ae178760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- 172.21.15.44:0/2088856080 shutdown_connections 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- 172.21.15.44:0/2088856080 wait complete. 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- start start 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 Processor -- start 2024-09-13T17:27:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae186960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae183be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae0812a0 con 0x5581ae17be10 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581addb8430 con 0x5581ae186fd0 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.318+0000 7f5cd9286740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae063680 con 0x5581ae17c730 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5cd9286740 10 client.0 _ll_register_callbacks cb 0x5581ae0d9610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae186960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae186960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35666/0 (socket says 172.21.15.44:35666) 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/1925454710 learned_addr learned my addr 172.21.15.44:0/1925454710 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccffff640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae183be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 msgr2=0x5581ae183be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae183be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/1925454710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 msgr2=0x5581ae1841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/1925454710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc400bcb0 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae186960 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005770 tx=0x7f5cc400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534588faa73323ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5caf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5caf7fe640 1 -- 172.21.15.44:0/1925454710 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004bd0 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5caf7fe640 1 -- 172.21.15.44:0/1925454710 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5cc4004d30 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581addd34e0 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5caf7fe640 1 -- 172.21.15.44:0/1925454710 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cc4010460 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5cd9286740 10 client.16035 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.319+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5581ae0de620 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5caf7fe640 1 -- 172.21.15.44:0/1925454710 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5581addd34e0 con 0x5581ae17be10 2024-09-13T17:27:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5caf7fe640 1 -- 172.21.15.44:0/1925454710 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5cc401c070 con 0x5581ae17be10 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5cd9286740 10 client.16035 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5cd9286740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5cd9286740 20 client.16035 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5cd9286740 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae196c80 0x5581ae1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5cd9286740 4 client.16035 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581adde47e0 con 0x5581ae196c80 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.320+0000 7f5ccffff640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae196c80 0x5581ae1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5caf7fe640 10 client.16035 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5ccffff640 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae196c80 0x5581ae1990a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bdd8a922bf3c68e server_cookie=f3b01c270f53e45f in_seq=0 out_seq=0 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5caf7fe640 1 -- 172.21.15.44:0/1925454710 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5581adde47e0 con 0x5581ae196c80 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5caf7fe640 10 client.16035 handle_command_reply: tid=0 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 client.16035 shutdown 2024-09-13T17:27:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 msgr2=0x5581ae186960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae186960 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005770 tx=0x7f5cc400b7e0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 shutdown_connections 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae196c80 0x5581ae1990a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17c730 0x5581ae183be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae1841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 --2- 172.21.15.44:0/1925454710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17be10 0x5581ae186960 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5caf7fe640 0 client.16035 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.321+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 >> 172.21.15.44:0/1925454710 conn(0x5581ae178380 msgr2=0x5581ae17e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.322+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 shutdown_connections 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.322+0000 7f5cd9286740 1 -- 172.21.15.44:0/1925454710 wait complete. 2024-09-13T17:27:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.322+0000 7f5cd9286740 20 client.16035 trim_cache size 0 max 16384 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 Processor -- start 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 -- start start 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae1841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 0x5581ae19da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae17eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae17cca0 con 0x5581ae17c730 2024-09-13T17:27:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae17cfb0 con 0x5581ae186fd0 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5cd9286740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae185150 con 0x5581ae17be10 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 0x5581ae19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 0x5581ae19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35678/0 (socket says 172.21.15.44:35678) 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 -- 172.21.15.44:0/3713409038 learned_addr learned my addr 172.21.15.44:0/3713409038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae17eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae1841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 -- 172.21.15.44:0/3713409038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 msgr2=0x5581ae1841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae1841b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 -- 172.21.15.44:0/3713409038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 msgr2=0x5581ae17eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae17eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccffff640 1 -- 172.21.15.44:0/3713409038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ae080790 con 0x5581ae17c730 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.336+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae17eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.337+0000 7f5ccffff640 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 0x5581ae19da00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5581ae1999b0 tx=0x7f5cc0014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba6893eb6604dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.337+0000 7f5cb6ffd640 1 -- 172.21.15.44:0/3713409038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0021030 con 0x5581ae17c730 2024-09-13T17:27:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.337+0000 7f5cb6ffd640 1 -- 172.21.15.44:0/3713409038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cc00044c0 con 0x5581ae17c730 2024-09-13T17:27:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.337+0000 7f5cb6ffd640 1 -- 172.21.15.44:0/3713409038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0018040 con 0x5581ae17c730 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.337+0000 7f5cd9286740 1 -- 172.21.15.44:0/3713409038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 msgr2=0x5581ae19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.337+0000 7f5cd9286740 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 0x5581ae19da00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5581ae1999b0 tx=0x7f5cc0014900 comp rx=0 tx=0).stop 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- 172.21.15.44:0/3713409038 shutdown_connections 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae186fd0 0x5581ae17eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae17c730 0x5581ae19da00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 --2- 172.21.15.44:0/3713409038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae1841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- 172.21.15.44:0/3713409038 >> 172.21.15.44:0/3713409038 conn(0x5581ae178380 msgr2=0x5581ae178760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- 172.21.15.44:0/3713409038 shutdown_connections 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- 172.21.15.44:0/3713409038 wait complete. 2024-09-13T17:27:55.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- start start 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 Processor -- start 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae18df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae17c730 0x5581ae18a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 0x5581ae18a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae18ae90 con 0x5581ae186fd0 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae18b000 con 0x5581ae17c730 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ae18b1c0 con 0x5581ae17be10 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.338+0000 7f5cd9286740 10 client.0 _ll_register_callbacks cb 0x7f5cc40123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 0x5581ae18a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 0x5581ae18a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35692/0 (socket says 172.21.15.44:35692) 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/190541372 learned_addr learned my addr 172.21.15.44:0/190541372 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:55.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccffff640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae17c730 0x5581ae18a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae18df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 msgr2=0x5581ae18df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae18df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/190541372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae17c730 msgr2=0x5581ae18a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae17c730 0x5581ae18a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 -- 172.21.15.44:0/190541372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc0009a60 con 0x5581ae186fd0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5ccf7fe640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 0x5581ae18a920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005d00 tx=0x7f5cc400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63503f12c5b34ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5caf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5caf7fe640 1 -- 172.21.15.44:0/190541372 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004660 con 0x5581ae186fd0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5caf7fe640 1 -- 172.21.15.44:0/190541372 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5cc4000e40 con 0x5581ae186fd0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cc4005020 con 0x5581ae186fd0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5caf7fe640 1 -- 172.21.15.44:0/190541372 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cc400bed0 con 0x5581ae186fd0 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5cd9286740 10 client.16041 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.339+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5581ae0a5ea0 con 0x5581ae186fd0 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5caf7fe640 1 -- 172.21.15.44:0/190541372 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cc4005020 con 0x5581ae186fd0 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5caf7fe640 1 -- 172.21.15.44:0/190541372 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5cc401c070 con 0x5581ae186fd0 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5cd9286740 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5cd9286740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5cd9286740 20 client.16041 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5cd9286740 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae1ee310 0x5581ae1f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5cd9286740 4 client.16041 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5581ade37e30 con 0x5581ae1ee310 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5ccffff640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae1ee310 0x5581ae1f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5ccffff640 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae1ee310 0x5581ae1f0700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5293eb4110259f17 server_cookie=2f270bb41c4731b2 in_seq=0 out_seq=0 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.340+0000 7f5caf7fe640 10 client.16041 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5caf7fe640 1 -- 172.21.15.44:0/190541372 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5581ade37e30 con 0x5581ae1ee310 2024-09-13T17:27:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5caf7fe640 10 client.16041 handle_command_reply: tid=0 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 client.16041 shutdown 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 msgr2=0x5581ae18a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 0x5581ae18a920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005d00 tx=0x7f5cc400b7e0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 shutdown_connections 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5581ae1ee310 0x5581ae1f0700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581ae186fd0 0x5581ae18a920 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5caf7fe640 0 client.16041 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5581ae17c730 0x5581ae18a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 --2- 172.21.15.44:0/190541372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae17be10 0x5581ae18df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 >> 172.21.15.44:0/190541372 conn(0x5581ae178380 msgr2=0x5581ae178760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 shutdown_connections 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 1 -- 172.21.15.44:0/190541372 wait complete. 2024-09-13T17:27:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd9286740 20 client.16041 trim_cache size 0 max 16384 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: "0aebdb94-e447-4775-af72-f758c8ae39da": { 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0aebdb94-e447-4775-af72-f758c8ae39da", 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:27:55.349 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:27:55.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5cb80a13b0 msgr2=0x7f5cb80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd655e640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5cb80a13b0 0x7f5cb80a3870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80eb960 tx=0x7f5cc80a6040 comp rx=0 tx=0).stop 2024-09-13T17:27:55.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 msgr2=0x7f5cd019afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:55.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.341+0000 7f5cd655e640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 0x7f5cd019afd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8004af0 tx=0x7f5cc8030e50 comp rx=0 tx=0).stop 2024-09-13T17:27:55.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 shutdown_connections 2024-09-13T17:27:55.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5cb80a13b0 0x7f5cb80a3870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd0103fa0 0x7f5cd0196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cd019ab20 0x7f5cd019afd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 --2- 172.21.15.44:0/2544934389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd00fee50 0x7f5cd01961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 >> 172.21.15.44:0/2544934389 conn(0x7f5cd00fa8c0 msgr2=0x7f5cd00688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 shutdown_connections 2024-09-13T17:27:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:27:55.342+0000 7f5cd655e640 1 -- 172.21.15.44:0/2544934389 wait complete. 2024-09-13T17:27:55.353 INFO:tasks.cephfs.filesystem:scrub status for tag:0aebdb94-e447-4775-af72-f758c8ae39da - {'path': '/', 'tag': '0aebdb94-e447-4775-af72-f758c8ae39da', 'options': 'recursive,force'} 2024-09-13T17:27:55.689 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T17:27:55.769 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T17:27:55.809 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T17:27:55.811 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T17:27:55.811 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T17:27:55.851 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T17:27:55.855 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:27:55.855 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T17:27:55.871 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-13T17:27:55.873 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-13T17:27:55.873 DEBUG:teuthology.orchestra.run.smithi044:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-13T17:27:55.938 INFO:tasks.workunit.client.0.smithi044.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-13T17:27:55.938 INFO:tasks.workunit.client.0.smithi044.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-13T17:27:55.938 INFO:tasks.workunit.client.0.smithi044.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-13T17:27:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:56 smithi070 ceph-mon[34502]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:56 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:27:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:56 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:56 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:27:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:56 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:56 smithi100 ceph-mon[35055]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:56 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:27:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:56 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:56 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:27:56.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:56 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:56 smithi044 ceph-mon[31175]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-13T17:27:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:56 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:27:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:56 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:56 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:27:56.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:56 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:27:58.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:58 smithi070 ceph-mon[34502]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:58 smithi100 ceph-mon[35055]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1508784524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc410a280 msgr2=0x7f6dc410c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1508784524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc410a280 0x7f6dc410c740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009ca0 tx=0x7f6dbc02a950 comp rx=0 tx=0).stop 2024-09-13T17:27:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1508784524 shutdown_connections 2024-09-13T17:27:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1508784524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc410a280 0x7f6dc410c740 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1508784524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4101780 0x7f6dc4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1508784524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc4100e60 0x7f6dc4101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1508784524 >> 172.21.15.44:0/1508784524 conn(0x7f6dc40fa890 msgr2=0x7f6dc40fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1508784524 shutdown_connections 2024-09-13T17:27:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.575+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1508784524 wait complete. 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 Processor -- start 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 -- start start 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4100e60 0x7f6dc4196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 0x7f6dc41965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc410a280 0x7f6dc419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc419afc0 con 0x7f6dc4101780 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc419b130 con 0x7f6dc4100e60 2024-09-13T17:27:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc8bfb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc419b2f0 con 0x7f6dc410a280 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 0x7f6dc41965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 0x7f6dc41965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35704/0 (socket says 172.21.15.44:35704) 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/1292512926 learned_addr learned my addr 172.21.15.44:0/1292512926 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc410a280 0x7f6dc419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc259c640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4100e60 0x7f6dc4196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/1292512926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc410a280 msgr2=0x7f6dc419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc410a280 0x7f6dc419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/1292512926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4100e60 msgr2=0x7f6dc4196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4100e60 0x7f6dc4196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/1292512926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009900 con 0x7f6dc4101780 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc410a280 0x7f6dc419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.576+0000 7f6dc259c640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4100e60 0x7f6dc4196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.577+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 0x7f6dc41965d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6db800e9f0 tx=0x7f6db800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.577+0000 7f6db37fe640 1 -- 172.21.15.44:0/1292512926 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8009700 con 0x7f6dc4101780 2024-09-13T17:27:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.577+0000 7f6db37fe640 1 -- 172.21.15.44:0/1292512926 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6db800cd80 con 0x7f6dc4101780 2024-09-13T17:27:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.577+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc419b5d0 con 0x7f6dc4101780 2024-09-13T17:27:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.577+0000 7f6db37fe640 1 -- 172.21.15.44:0/1292512926 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db80187e0 con 0x7f6dc4101780 2024-09-13T17:27:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.577+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc41a2e50 con 0x7f6dc4101780 2024-09-13T17:27:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.579+0000 7f6db37fe640 1 -- 172.21.15.44:0/1292512926 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f6db8010430 con 0x7f6dc4101780 2024-09-13T17:27:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 Processor -- start 2024-09-13T17:27:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 -- start start 2024-09-13T17:27:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e2060 0x5580977ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558097748c30 con 0x5580977ec900 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977463c0 con 0x5580977e2060 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558097741cb0 con 0x5580977e1740 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6db37fe640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6dac0a13b0 0x7f6dac0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35708/0 (socket says 172.21.15.44:35708) 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/646503117 learned_addr learned my addr 172.21.15.44:0/646503117 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.580+0000 7f6dc259c640 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e2060 0x5580977ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6db37fe640 1 -- 172.21.15.44:0/1292512926 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6db8014070 con 0x7f6dc4101780 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6dac0a13b0 0x7f6dac0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/646503117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 msgr2=0x5580977e1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977e1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/646503117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e2060 msgr2=0x5580977ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e2060 0x5580977ec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/646503117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580976e8910 con 0x5580977ec900 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977eecf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6db80ca870 tx=0x7f6db8010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c9ac8ddf999795 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6dac0a13b0 0x7f6dac0a3870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc032000 tx=0x7f6dbc0095c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6db0ff9640 1 -- 172.21.15.44:0/646503117 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8094ea0 con 0x5580977ec900 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6db0ff9640 1 -- 172.21.15.44:0/646503117 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6db809e0c0 con 0x5580977ec900 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.581+0000 7f6db0ff9640 1 -- 172.21.15.44:0/646503117 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db80cd560 con 0x5580977ec900 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/646503117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 msgr2=0x5580977eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977eecf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6db80ca870 tx=0x7f6db8010de0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/646503117 shutdown_connections 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977eecf0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e2060 0x5580977ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/646503117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977e1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/646503117 >> 172.21.15.44:0/646503117 conn(0x5580977ddcd0 msgr2=0x5580977de0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/646503117 shutdown_connections 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/646503117 wait complete. 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.582+0000 7f6dcb9f3740 1 -- start start 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 Processor -- start 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977e9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977ec900 0x5580977e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976ffe80 con 0x5580977e2060 2024-09-13T17:27:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976ffb20 con 0x5580977ec900 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976c5260 con 0x5580977e1740 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dcb9f3740 10 client.0 _ll_register_callbacks cb 0x5580976fadf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977e9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35716/0 (socket says 172.21.15.44:35716) 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/914490872 learned_addr learned my addr 172.21.15.44:0/914490872 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977ec900 0x5580977e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc259c640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 msgr2=0x5580977ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977ec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/914490872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977ec900 msgr2=0x5580977e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977ec900 0x5580977e9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/914490872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc41a2e50 con 0x5580977e2060 2024-09-13T17:27:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.583+0000 7f6dc259c640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977e9540 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6db809a220 tx=0x7f6db8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f25dd93c156e583 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6da9ffb640 1 -- 172.21.15.44:0/914490872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8094ea0 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6da9ffb640 1 -- 172.21.15.44:0/914490872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6db80b2e40 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55809776a680 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6da9ffb640 1 -- 172.21.15.44:0/914490872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db80108f0 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 10 client.16053 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558097740330 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6da9ffb640 1 -- 172.21.15.44:0/914490872 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55809776a680 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6da9ffb640 1 -- 172.21.15.44:0/914490872 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6db80b2030 con 0x5580977e2060 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 10 client.16053 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 20 client.16053 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5580977fbf80 0x5580977fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 4 client.16053 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:27:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.584+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580977355d0 con 0x5580977fbf80 2024-09-13T17:27:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.585+0000 7f6dc259c640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5580977fbf80 0x5580977fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.585+0000 7f6dc259c640 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5580977fbf80 0x5580977fe370 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4cbdc10b48d689f server_cookie=95fa7aaf8731aa9d in_seq=0 out_seq=0 2024-09-13T17:27:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.585+0000 7f6da9ffb640 10 client.16053 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6da9ffb640 1 -- 172.21.15.44:0/914490872 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5580977355d0 con 0x5580977fbf80 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6da9ffb640 10 client.16053 handle_command_reply: tid=0 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 client.16053 shutdown 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 msgr2=0x5580977e9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977e9540 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6db809a220 tx=0x7f6db8010cd0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 shutdown_connections 2024-09-13T17:27:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5580977fbf80 0x5580977fe370 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977ec900 0x5580977e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977e9540 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6da9ffb640 0 client.16053 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/914490872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e1740 0x5580977ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 >> 172.21.15.44:0/914490872 conn(0x5580977ddcd0 msgr2=0x5580977d8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 shutdown_connections 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/914490872 wait complete. 2024-09-13T17:27:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.586+0000 7f6dcb9f3740 20 client.16053 trim_cache size 0 max 16384 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 Processor -- start 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 -- start start 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977ec900 0x5580977e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977e2c80 con 0x5580977e2060 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977ea980 con 0x5580977e1740 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977f8930 con 0x5580977ec900 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35726/0 (socket says 172.21.15.44:35726) 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/3013998415 learned_addr learned my addr 172.21.15.44:0/3013998415 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dc259c640 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.599+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977ec900 0x5580977e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/3013998415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977ec900 msgr2=0x5580977e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977ec900 0x5580977e7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/3013998415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 msgr2=0x5580977f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dc1d9b640 1 -- 172.21.15.44:0/3013998415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580976e2300 con 0x5580977e2060 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977f3b10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6db80985e0 tx=0x7f6db80caa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6929a6e0b670d282 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6db17fa640 1 -- 172.21.15.44:0/3013998415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8094ea0 con 0x5580977e2060 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6db17fa640 1 -- 172.21.15.44:0/3013998415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6db80108f0 con 0x5580977e2060 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6db17fa640 1 -- 172.21.15.44:0/3013998415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8083e10 con 0x5580977e2060 2024-09-13T17:27:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/3013998415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 msgr2=0x5580977f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977f3b10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6db80985e0 tx=0x7f6db80caa30 comp rx=0 tx=0).stop 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/3013998415 shutdown_connections 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977ec900 0x5580977e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/3013998415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.600+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/3013998415 >> 172.21.15.44:0/3013998415 conn(0x5580977ddcd0 msgr2=0x5580977e4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/3013998415 shutdown_connections 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/3013998415 wait complete. 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 -- start start 2024-09-13T17:27:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 Processor -- start 2024-09-13T17:27:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977e62a0 con 0x5580977ec900 2024-09-13T17:27:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977e6410 con 0x5580977e1740 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580977e65d0 con 0x5580977e2060 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dcb9f3740 10 client.0 _ll_register_callbacks cb 0x7f6db809a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dc2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dc2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35732/0 (socket says 172.21.15.44:35732) 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dc2d9d640 1 -- 172.21.15.44:0/41079332 learned_addr learned my addr 172.21.15.44:0/41079332 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.601+0000 7f6dc259c640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc1d9b640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc2d9d640 1 -- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e2060 msgr2=0x5580977f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc2d9d640 1 -- 172.21.15.44:0/41079332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 msgr2=0x5580977f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc2d9d640 1 -- 172.21.15.44:0/41079332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558097740330 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc259c640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dc2d9d640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977e5ba0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009ca0 tx=0x7f6dbc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be28e0c9e2f2c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6da9ffb640 1 -- 172.21.15.44:0/41079332 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc003d80 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6da9ffb640 1 -- 172.21.15.44:0/41079332 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6dbc030440 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6db809e0c0 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6da9ffb640 1 -- 172.21.15.44:0/41079332 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dbc030640 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dcb9f3740 10 client.16059 fetch_fsmap learned FSMap version 25 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.602+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5580977078e0 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6da9ffb640 1 -- 172.21.15.44:0/41079332 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6db809e0c0 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6da9ffb640 1 -- 172.21.15.44:0/41079332 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6dbc053070 con 0x5580977ec900 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dcb9f3740 10 client.16059 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dcb9f3740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:27:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dcb9f3740 20 client.16059 populate_metadata read hostname 'smithi044' 2024-09-13T17:27:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558097853660 0x558097855a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:27:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dcb9f3740 4 client.16059 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:27:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558097435300 con 0x558097853660 2024-09-13T17:27:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dc259c640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558097853660 0x558097855a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:27:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6da9ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.603+0000 7f6dc259c640 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558097853660 0x558097855a80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d9e7acd28ad713 server_cookie=330a58630c38ed45 in_seq=0 out_seq=0 2024-09-13T17:27:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:58 smithi044 ceph-mon[31175]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.612+0000 7f6da9ffb640 1 -- 172.21.15.44:0/41079332 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558097435300 con 0x558097853660 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.612+0000 7f6da9ffb640 10 client.16059 handle_command_reply: tid=0 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.612+0000 7f6dcb9f3740 1 client.16059 shutdown 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 msgr2=0x5580977e5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977e5ba0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009ca0 tx=0x7f6dbc02a950 comp rx=0 tx=0).stop 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 shutdown_connections 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558097853660 0x558097855a80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580977e2060 0x5580977f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580977ec900 0x5580977e5ba0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6da9ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 --2- 172.21.15.44:0/41079332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580977e1740 0x5580977f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 >> 172.21.15.44:0/41079332 conn(0x5580977ddcd0 msgr2=0x5580977d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 shutdown_connections 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 1 -- 172.21.15.44:0/41079332 wait complete. 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dcb9f3740 20 client.16059 trim_cache size 0 max 16384 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:27:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:27:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:27:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6dac0a13b0 msgr2=0x7f6dac0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6dac0a13b0 0x7f6dac0a3870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc032000 tx=0x7f6dbc0095c0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 msgr2=0x7f6dc41965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:27:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 0x7f6dc41965d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6db800e9f0 tx=0x7f6db800eeb0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 shutdown_connections 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6dac0a13b0 0x7f6dac0a3870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc410a280 0x7f6dc419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dc4101780 0x7f6dc41965d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.613+0000 7f6dc8bfb640 1 --2- 172.21.15.44:0/1292512926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc4100e60 0x7f6dc4196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.614+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 >> 172.21.15.44:0/1292512926 conn(0x7f6dc40fa890 msgr2=0x7f6dc40fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.614+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 shutdown_connections 2024-09-13T17:27:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:27:58.614+0000 7f6dc8bfb640 1 -- 172.21.15.44:0/1292512926 wait complete. 2024-09-13T17:27:59.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:27:59 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:27:59.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:27:59 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:27:59.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:27:59 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:28:00.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:00 smithi070 ceph-mon[34502]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-09-13T17:28:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:00 smithi100 ceph-mon[35055]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-09-13T17:28:00.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:00 smithi044 ceph-mon[31175]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-09-13T17:28:02.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:02 smithi070 ceph-mon[34502]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-13T17:28:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:02 smithi100 ceph-mon[35055]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-13T17:28:02.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:02 smithi044 ceph-mon[31175]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-13T17:28:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:04 smithi070 ceph-mon[34502]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-13T17:28:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:04 smithi100 ceph-mon[35055]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-13T17:28:04.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:04 smithi044 ceph-mon[31175]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-13T17:28:06.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:06 smithi070 ceph-mon[34502]: pgmap v44: 129 pgs: 129 active+clean; 1.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-09-13T17:28:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:06 smithi100 ceph-mon[35055]: pgmap v44: 129 pgs: 129 active+clean; 1.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-09-13T17:28:06.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:06 smithi044 ceph-mon[31175]: pgmap v44: 129 pgs: 129 active+clean; 1.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-09-13T17:28:08.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:08 smithi070 ceph-mon[34502]: pgmap v45: 129 pgs: 129 active+clean; 1.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 27 op/s 2024-09-13T17:28:08.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:08 smithi100 ceph-mon[35055]: pgmap v45: 129 pgs: 129 active+clean; 1.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 27 op/s 2024-09-13T17:28:08.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:08 smithi044 ceph-mon[31175]: pgmap v45: 129 pgs: 129 active+clean; 1.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 27 op/s 2024-09-13T17:28:10.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:10 smithi070 ceph-mon[34502]: pgmap v46: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 32 op/s 2024-09-13T17:28:10.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:10 smithi100 ceph-mon[35055]: pgmap v46: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 32 op/s 2024-09-13T17:28:10.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:10 smithi044 ceph-mon[31175]: pgmap v46: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 32 op/s 2024-09-13T17:28:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.711+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/1913000810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 msgr2=0x7ff2cc10c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.711+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/1913000810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc10c750 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009ca0 tx=0x7ff2c402a950 comp rx=0 tx=0).stop 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/1913000810 shutdown_connections 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/1913000810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc10c750 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/1913000810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2cc101820 0x7ff2cc109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/1913000810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2cc100f00 0x7ff2cc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/1913000810 >> 172.21.15.44:0/1913000810 conn(0x7ff2cc0fa8f0 msgr2=0x7ff2cc0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/1913000810 shutdown_connections 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/1913000810 wait complete. 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.712+0000 7ff2d1cb8640 1 Processor -- start 2024-09-13T17:28:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 -- start start 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2cc100f00 0x7ff2cc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2cc101820 0x7ff2cc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc19d1f0 con 0x7ff2cc10a290 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc19d360 con 0x7ff2cc100f00 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2d1cb8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc19d520 con 0x7ff2cc101820 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39818/0 (socket says 172.21.15.44:39818) 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 -- 172.21.15.44:0/3958577567 learned_addr learned my addr 172.21.15.44:0/3958577567 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2caffd640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2cc101820 0x7ff2cc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2cc100f00 0x7ff2cc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 -- 172.21.15.44:0/3958577567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2cc101820 msgr2=0x7ff2cc198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2cc101820 0x7ff2cc198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 -- 172.21.15.44:0/3958577567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2cc100f00 msgr2=0x7ff2cc1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2cc100f00 0x7ff2cc1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 -- 172.21.15.44:0/3958577567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c4009900 con 0x7ff2cc10a290 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.713+0000 7ff2cbfff640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc19cbf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff2c402af40 tx=0x7ff2c402fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.714+0000 7ff2c8ff9640 1 -- 172.21.15.44:0/3958577567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003d80 con 0x7ff2cc10a290 2024-09-13T17:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.714+0000 7ff2c8ff9640 1 -- 172.21.15.44:0/3958577567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2c40308d0 con 0x7ff2cc10a290 2024-09-13T17:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.714+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc19d7a0 con 0x7ff2cc10a290 2024-09-13T17:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.714+0000 7ff2c8ff9640 1 -- 172.21.15.44:0/3958577567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4004790 con 0x7ff2cc10a290 2024-09-13T17:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.714+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc19da70 con 0x7ff2cc10a290 2024-09-13T17:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.716+0000 7ff2c8ff9640 1 -- 172.21.15.44:0/3958577567 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7ff2c402e050 con 0x7ff2cc10a290 2024-09-13T17:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 Processor -- start 2024-09-13T17:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 -- start start 2024-09-13T17:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24330 0x561e3ea24710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 0x561e3ea2efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3e98fc30 con 0x561e3ea24c50 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3e98d3c0 con 0x561e3ea2f4f0 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2d49da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3e988cb0 con 0x561e3ea24330 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 0x561e3ea2efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 0x561e3ea2efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39830/0 (socket says 172.21.15.44:39830) 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 -- 172.21.15.44:0/2032969456 learned_addr learned my addr 172.21.15.44:0/2032969456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2c8ff9640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff2b40a13b0 0x7ff2b40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2caffd640 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24330 0x561e3ea24710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2caffd640 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cbfff640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff2b40a13b0 0x7ff2b40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 -- 172.21.15.44:0/2032969456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24330 msgr2=0x561e3ea24710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24330 0x561e3ea24710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 -- 172.21.15.44:0/2032969456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 msgr2=0x561e3ea318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea318e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2cb7fe640 1 -- 172.21.15.44:0/2032969456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e3e92f910 con 0x561e3ea24c50 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.717+0000 7ff2c8ff9640 1 -- 172.21.15.44:0/3958577567 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff2c40edf10 con 0x7ff2cc10a290 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.718+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 0x561e3ea2efb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00b760 tx=0x7ff2bc00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbe137b3ceae8e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.718+0000 7ff2cbfff640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff2b40a13b0 0x7ff2b40a3870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40eb160 tx=0x7ff2c40be040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.718+0000 7ff2b1ffb640 1 -- 172.21.15.44:0/2032969456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc003d50 con 0x561e3ea24c50 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.718+0000 7ff2b1ffb640 1 -- 172.21.15.44:0/2032969456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2bc00be70 con 0x561e3ea24c50 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.718+0000 7ff2b1ffb640 1 -- 172.21.15.44:0/2032969456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc004d40 con 0x561e3ea24c50 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 -- 172.21.15.44:0/2032969456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 msgr2=0x561e3ea2efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 0x561e3ea2efb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00b760 tx=0x7ff2bc00bc20 comp rx=0 tx=0).stop 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 -- 172.21.15.44:0/2032969456 shutdown_connections 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24c50 0x561e3ea2efb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2032969456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24330 0x561e3ea24710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 -- 172.21.15.44:0/2032969456 >> 172.21.15.44:0/2032969456 conn(0x561e3ea208c0 msgr2=0x561e3ea20ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 -- 172.21.15.44:0/2032969456 shutdown_connections 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 -- 172.21.15.44:0/2032969456 wait complete. 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 -- start start 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.719+0000 7ff2d49da740 1 Processor -- start 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea2c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3e946e80 con 0x561e3ea24330 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3e946b20 con 0x561e3ea2f4f0 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3e90c260 con 0x561e3ea24c50 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2d49da740 10 client.0 _ll_register_callbacks cb 0x561e3e941df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39846/0 (socket says 172.21.15.44:39846) 2024-09-13T17:28:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 -- 172.21.15.44:0/468449510 learned_addr learned my addr 172.21.15.44:0/468449510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea2c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 -- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 msgr2=0x561e3ea2c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea2c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 -- 172.21.15.44:0/468449510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 msgr2=0x561e3ea2c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 -- 172.21.15.44:0/468449510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc004370 con 0x561e3ea24330 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2caffd640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2eeb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0010a50 tx=0x7ff2c0009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cb407586c019bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.720+0000 7ff2aaffd640 1 -- 172.21.15.44:0/468449510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0016040 con 0x561e3ea24330 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2aaffd640 1 -- 172.21.15.44:0/468449510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff2c000fb60 con 0x561e3ea24330 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e3e9b1680 con 0x561e3ea24330 2024-09-13T17:28:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2aaffd640 1 -- 172.21.15.44:0/468449510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c000fd20 con 0x561e3ea24330 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 10 client.16071 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e3e987330 con 0x561e3ea24330 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2aaffd640 1 -- 172.21.15.44:0/468449510 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561e3e9b1680 con 0x561e3ea24330 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2aaffd640 1 -- 172.21.15.44:0/468449510 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2c001e3c0 con 0x561e3ea24330 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 10 client.16071 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 20 client.16071 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea3ef40 0x561e3ea41360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 4 client.16071 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.721+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e3e97c5d0 con 0x561e3ea3ef40 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.722+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea3ef40 0x561e3ea41360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.722+0000 7ff2aaffd640 10 client.16071 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.722+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea3ef40 0x561e3ea41360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44f9e493d05d1d5 server_cookie=4d351cc37f13b0ce in_seq=0 out_seq=0 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2aaffd640 1 -- 172.21.15.44:0/468449510 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561e3e97c5d0 con 0x561e3ea3ef40 2024-09-13T17:28:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2aaffd640 10 client.16071 handle_command_reply: tid=0 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 client.16071 shutdown 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 msgr2=0x561e3ea2eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2eeb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0010a50 tx=0x7ff2c0009a40 comp rx=0 tx=0).stop 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 shutdown_connections 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea3ef40 0x561e3ea41360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea2c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2aaffd640 0 client.16071 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 --2- 172.21.15.44:0/468449510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2eeb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 >> 172.21.15.44:0/468449510 conn(0x561e3ea208c0 msgr2=0x561e3ea1fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 shutdown_connections 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 1 -- 172.21.15.44:0/468449510 wait complete. 2024-09-13T17:28:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.723+0000 7ff2d49da740 20 client.16071 trim_cache size 0 max 16384 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 Processor -- start 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 -- start start 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24c50 0x561e3ea2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3ea21550 con 0x561e3ea24330 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3ea31fa0 con 0x561e3ea24c50 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2d49da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3ea2d5c0 con 0x561e3ea2f4f0 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39860/0 (socket says 172.21.15.44:39860) 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.736+0000 7ff2caffd640 1 -- 172.21.15.44:0/2389690280 learned_addr learned my addr 172.21.15.44:0/2389690280 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24c50 0x561e3ea2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2caffd640 1 -- 172.21.15.44:0/2389690280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea2f4f0 msgr2=0x561e3ea2af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2caffd640 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2caffd640 1 -- 172.21.15.44:0/2389690280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24c50 msgr2=0x561e3ea2a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2caffd640 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24c50 0x561e3ea2a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2caffd640 1 -- 172.21.15.44:0/2389690280 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e3e929300 con 0x561e3ea24330 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24c50 0x561e3ea2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2caffd640 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2ea10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0010a20 tx=0x7ff2c0010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5121c1a3ba0c826 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2b27fc640 1 -- 172.21.15.44:0/2389690280 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c001e9c0 con 0x561e3ea24330 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2b27fc640 1 -- 172.21.15.44:0/2389690280 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2c0014070 con 0x561e3ea24330 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2b27fc640 1 -- 172.21.15.44:0/2389690280 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0005040 con 0x561e3ea24330 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2d49da740 1 -- 172.21.15.44:0/2389690280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 msgr2=0x561e3ea2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.737+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2ea10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0010a20 tx=0x7ff2c0010430 comp rx=0 tx=0).stop 2024-09-13T17:28:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- 172.21.15.44:0/2389690280 shutdown_connections 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea2f4f0 0x561e3ea2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24c50 0x561e3ea2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 --2- 172.21.15.44:0/2389690280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea24330 0x561e3ea2ea10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- 172.21.15.44:0/2389690280 >> 172.21.15.44:0/2389690280 conn(0x561e3ea208c0 msgr2=0x561e3ea27410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- 172.21.15.44:0/2389690280 shutdown_connections 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- 172.21.15.44:0/2389690280 wait complete. 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- start start 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 Processor -- start 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24330 0x561e3ea32be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea33120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 0x561e3ea33660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3ea95420 con 0x561e3ea2f4f0 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3ea95560 con 0x561e3ea24330 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3ea956a0 con 0x561e3ea24c50 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.738+0000 7ff2d49da740 10 client.0 _ll_register_callbacks cb 0x7ff2bc004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 0x561e3ea33660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 0x561e3ea33660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39870/0 (socket says 172.21.15.44:39870) 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 -- 172.21.15.44:0/1500926159 learned_addr learned my addr 172.21.15.44:0/1500926159 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea33120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24330 0x561e3ea32be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 -- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 msgr2=0x561e3ea33120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea33120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 -- 172.21.15.44:0/1500926159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24330 msgr2=0x561e3ea32be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24330 0x561e3ea32be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 -- 172.21.15.44:0/1500926159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e3e987330 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2caffd640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 0x561e3ea33660 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0005e00 tx=0x7ff2c0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f8e20e5d9f1ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2aaffd640 1 -- 172.21.15.44:0/1500926159 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0016070 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2aaffd640 1 -- 172.21.15.44:0/1500926159 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff2c0000e40 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2bc00be70 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2aaffd640 1 -- 172.21.15.44:0/1500926159 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c001ec00 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2d49da740 10 client.16077 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.739+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e3e94e8e0 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2aaffd640 1 -- 172.21.15.44:0/1500926159 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2bc00be70 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2aaffd640 1 -- 172.21.15.44:0/1500926159 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2c0003970 con 0x561e3ea2f4f0 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2d49da740 10 client.16077 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2d49da740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2d49da740 20 client.16077 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2d49da740 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea96a30 0x561e3ea98e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2d49da740 4 client.16077 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:28:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e3e67c300 con 0x561e3ea96a30 2024-09-13T17:28:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea96a30 0x561e3ea98e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2cb7fe640 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea96a30 0x561e3ea98e20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c97555b14a7456 server_cookie=fcdf3527d0bcab15 in_seq=0 out_seq=0 2024-09-13T17:28:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.740+0000 7ff2aaffd640 10 client.16077 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.752+0000 7ff2aaffd640 1 -- 172.21.15.44:0/1500926159 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e3e67c300 con 0x561e3ea96a30 2024-09-13T17:28:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.752+0000 7ff2aaffd640 10 client.16077 handle_command_reply: tid=0 2024-09-13T17:28:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 client.16077 shutdown 2024-09-13T17:28:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 msgr2=0x561e3ea33660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 0x561e3ea33660 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0005e00 tx=0x7ff2c0009e90 comp rx=0 tx=0).stop 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 shutdown_connections 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561e3ea96a30 0x561e3ea98e20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e3ea24c50 0x561e3ea33120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2aaffd640 0 client.16077 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e3ea2f4f0 0x561e3ea33660 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 --2- 172.21.15.44:0/1500926159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561e3ea24330 0x561e3ea32be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 >> 172.21.15.44:0/1500926159 conn(0x561e3ea208c0 msgr2=0x561e3ea3aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 shutdown_connections 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 1 -- 172.21.15.44:0/1500926159 wait complete. 2024-09-13T17:28:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d49da740 20 client.16077 trim_cache size 0 max 16384 2024-09-13T17:28:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:28:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:28:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:28:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:28:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff2b40a13b0 msgr2=0x7ff2b40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff2b40a13b0 0x7ff2b40a3870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40eb160 tx=0x7ff2c40be040 comp rx=0 tx=0).stop 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 msgr2=0x7ff2cc19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.753+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc19cbf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff2c402af40 tx=0x7ff2c402fc60 comp rx=0 tx=0).stop 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 shutdown_connections 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff2b40a13b0 0x7ff2b40a3870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2cc101820 0x7ff2cc198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2cc10a290 0x7ff2cc19cbf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 --2- 172.21.15.44:0/3958577567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2cc100f00 0x7ff2cc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 >> 172.21.15.44:0/3958577567 conn(0x7ff2cc0fa8f0 msgr2=0x7ff2cc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 shutdown_connections 2024-09-13T17:28:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:11.754+0000 7ff2d1cb8640 1 -- 172.21.15.44:0/3958577567 wait complete. 2024-09-13T17:28:12.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:12 smithi070 ceph-mon[34502]: pgmap v47: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-09-13T17:28:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:12 smithi100 ceph-mon[35055]: pgmap v47: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-09-13T17:28:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:12 smithi044 ceph-mon[31175]: pgmap v47: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-09-13T17:28:14.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:14 smithi070 ceph-mon[34502]: pgmap v48: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-09-13T17:28:14.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:14 smithi100 ceph-mon[35055]: pgmap v48: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-09-13T17:28:14.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:14 smithi044 ceph-mon[31175]: pgmap v48: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-09-13T17:28:16.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:16 smithi100 ceph-mon[35055]: pgmap v49: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 36 op/s 2024-09-13T17:28:16.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:16 smithi100 ceph-mon[35055]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-13T17:28:16.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:16 smithi044 ceph-mon[31175]: pgmap v49: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 36 op/s 2024-09-13T17:28:16.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:16 smithi044 ceph-mon[31175]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-13T17:28:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:16 smithi070 ceph-mon[34502]: pgmap v49: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 36 op/s 2024-09-13T17:28:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:16 smithi070 ceph-mon[34502]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-13T17:28:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:18 smithi100 ceph-mon[35055]: pgmap v50: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:18.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:18 smithi044 ceph-mon[31175]: pgmap v50: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:18 smithi070 ceph-mon[34502]: pgmap v50: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:20 smithi100 ceph-mon[35055]: pgmap v51: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:20 smithi044 ceph-mon[31175]: pgmap v51: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:20 smithi070 ceph-mon[34502]: pgmap v51: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:22 smithi100 ceph-mon[35055]: pgmap v52: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:22 smithi044 ceph-mon[31175]: pgmap v52: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:22 smithi070 ceph-mon[34502]: pgmap v52: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:24 smithi100 ceph-mon[35055]: pgmap v53: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:24 smithi044 ceph-mon[31175]: pgmap v53: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:24 smithi070 ceph-mon[34502]: pgmap v53: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.853+0000 7fe64c8c9640 1 -- 172.21.15.44:0/238052073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480ff7d0 msgr2=0x7fe648103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.853+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/238052073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480ff7d0 0x7fe648103810 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009930 tx=0x7fe63c02a420 comp rx=0 tx=0).stop 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 -- 172.21.15.44:0/238052073 shutdown_connections 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/238052073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe648103f70 0x7fe64810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/238052073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480ff7d0 0x7fe648103810 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/238052073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6480fee20 0x7fe6480ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 -- 172.21.15.44:0/238052073 >> 172.21.15.44:0/238052073 conn(0x7fe6480fa8f0 msgr2=0x7fe6480fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 -- 172.21.15.44:0/238052073 shutdown_connections 2024-09-13T17:28:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 -- 172.21.15.44:0/238052073 wait complete. 2024-09-13T17:28:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 Processor -- start 2024-09-13T17:28:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.854+0000 7fe64c8c9640 1 -- start start 2024-09-13T17:28:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64c8c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 0x7fe64810c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64c8c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe648103f70 0x7fe64810c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64c8c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe64810ce50 0x7fe6481a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64c8c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481a4930 con 0x7fe6480fee20 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64c8c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481a4aa0 con 0x7fe648103f70 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64c8c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481a4c60 con 0x7fe64810ce50 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 0x7fe64810c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 0x7fe64810c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42020/0 (socket says 172.21.15.44:42020) 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 -- 172.21.15.44:0/1588893215 learned_addr learned my addr 172.21.15.44:0/1588893215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe646d9d640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe64810ce50 0x7fe6481a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe645d9b640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe648103f70 0x7fe64810c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 -- 172.21.15.44:0/1588893215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe64810ce50 msgr2=0x7fe6481a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe64810ce50 0x7fe6481a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 -- 172.21.15.44:0/1588893215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe648103f70 msgr2=0x7fe64810c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe648103f70 0x7fe64810c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 -- 172.21.15.44:0/1588893215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009490 con 0x7fe6480fee20 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe64659c640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 0x7fe64810c3d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe63800ea60 tx=0x7fe63800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.855+0000 7fe6377fe640 1 -- 172.21.15.44:0/1588893215 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800cde0 con 0x7fe6480fee20 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.856+0000 7fe6377fe640 1 -- 172.21.15.44:0/1588893215 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6380044b0 con 0x7fe6480fee20 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.856+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c009560 con 0x7fe6480fee20 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.856+0000 7fe6377fe640 1 -- 172.21.15.44:0/1588893215 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638010430 con 0x7fe6480fee20 2024-09-13T17:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.856+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6481a5280 con 0x7fe6480fee20 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.858+0000 7fe6377fe640 1 -- 172.21.15.44:0/1588893215 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe638004000 con 0x7fe6480fee20 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.858+0000 7fe64f5eb740 1 Processor -- start 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 -- start start 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd19fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bc7ac30 con 0x56064bd0f330 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bc783c0 con 0x56064bd1a4f0 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bc73cb0 con 0x56064bd0fc50 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe6377fe640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6300a12b0 0x7fe6300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd0f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42032/0 (socket says 172.21.15.44:42032) 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 -- 172.21.15.44:0/202576848 learned_addr learned my addr 172.21.15.44:0/202576848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe64659c640 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd19fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe645d9b640 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd1c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6300a12b0 0x7fe6300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe6377fe640 1 -- 172.21.15.44:0/1588893215 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe638014070 con 0x7fe6480fee20 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 -- 172.21.15.44:0/202576848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 msgr2=0x56064bd19fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd19fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 -- 172.21.15.44:0/202576848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 msgr2=0x56064bd1c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd1c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.859+0000 7fe646d9d640 1 -- 172.21.15.44:0/202576848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064bc1a910 con 0x56064bd0f330 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.860+0000 7fe646d9d640 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd0f710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe640005f40 tx=0x7fe640012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1891f982cce083b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.860+0000 7fe646d9d640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6300a12b0 0x7fe6300a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe64001b000 tx=0x7fe64001d000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.860+0000 7fe634ff9640 1 -- 172.21.15.44:0/202576848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640011c40 con 0x56064bd0f330 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.860+0000 7fe634ff9640 1 -- 172.21.15.44:0/202576848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe640011da0 con 0x56064bd0f330 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.860+0000 7fe634ff9640 1 -- 172.21.15.44:0/202576848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000f3c0 con 0x56064bd0f330 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 -- 172.21.15.44:0/202576848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 msgr2=0x56064bd0f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd0f710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe640005f40 tx=0x7fe640012f00 comp rx=0 tx=0).stop 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 -- 172.21.15.44:0/202576848 shutdown_connections 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd19fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd1c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/202576848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd0f710 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 -- 172.21.15.44:0/202576848 >> 172.21.15.44:0/202576848 conn(0x56064bd0b8c0 msgr2=0x56064bd0bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 -- 172.21.15.44:0/202576848 shutdown_connections 2024-09-13T17:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 -- 172.21.15.44:0/202576848 wait complete. 2024-09-13T17:28:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 -- start start 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.861+0000 7fe64f5eb740 1 Processor -- start 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd19ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd17100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bc31e80 con 0x56064bd0f330 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bc31b20 con 0x56064bd1a4f0 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bbf7260 con 0x56064bd0fc50 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64f5eb740 10 client.0 _ll_register_callbacks cb 0x56064bc2cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd19ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd19ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42046/0 (socket says 172.21.15.44:42046) 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 -- 172.21.15.44:0/4131371482 learned_addr learned my addr 172.21.15.44:0/4131371482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe646d9d640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe645d9b640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd17100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 -- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 msgr2=0x56064bd17100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd17100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 -- 172.21.15.44:0/4131371482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 msgr2=0x56064bd176d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd176d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 -- 172.21.15.44:0/4131371482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe640012650 con 0x56064bd0f330 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe646d9d640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe64659c640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd19ed0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe63809a1f0 tx=0x7fe638014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea5e7a9c1825cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.862+0000 7fe62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe62dffb640 1 -- 172.21.15.44:0/4131371482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638067ea0 con 0x56064bd0f330 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe62dffb640 1 -- 172.21.15.44:0/4131371482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe63809a960 con 0x56064bd0f330 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56064bc9c680 con 0x56064bd0f330 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe62dffb640 1 -- 172.21.15.44:0/4131371482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe63809a260 con 0x56064bd0f330 2024-09-13T17:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 10 client.16089 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56064bc72330 con 0x56064bd0f330 2024-09-13T17:28:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe62dffb640 1 -- 172.21.15.44:0/4131371482 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56064bc9c680 con 0x56064bd0f330 2024-09-13T17:28:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe62dffb640 1 -- 172.21.15.44:0/4131371482 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe63801b160 con 0x56064bd0f330 2024-09-13T17:28:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 10 client.16089 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 20 client.16089 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd29e60 0x56064bd2c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 4 client.16089 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56064bc675d0 con 0x56064bd29e60 2024-09-13T17:28:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.863+0000 7fe645d9b640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd29e60 0x56064bd2c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.864+0000 7fe645d9b640 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd29e60 0x56064bd2c250 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1df73af42d5baf server_cookie=63876a089068ae4b in_seq=0 out_seq=0 2024-09-13T17:28:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.864+0000 7fe62dffb640 10 client.16089 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe62dffb640 1 -- 172.21.15.44:0/4131371482 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56064bc675d0 con 0x56064bd29e60 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe62dffb640 10 client.16089 handle_command_reply: tid=0 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 client.16089 shutdown 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 msgr2=0x56064bd19ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd19ed0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe63809a1f0 tx=0x7fe638014f20 comp rx=0 tx=0).stop 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 shutdown_connections 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd29e60 0x56064bd2c250 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd17100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd1a4f0 0x56064bd176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe62dffb640 0 client.16089 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/4131371482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd19ed0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 >> 172.21.15.44:0/4131371482 conn(0x56064bd0b8c0 msgr2=0x56064bd0ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 shutdown_connections 2024-09-13T17:28:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 1 -- 172.21.15.44:0/4131371482 wait complete. 2024-09-13T17:28:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.865+0000 7fe64f5eb740 20 client.16089 trim_cache size 0 max 16384 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 Processor -- start 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 -- start start 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0f330 0x56064bd199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd14340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 0x56064bd14880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bd102d0 con 0x56064bd1a4f0 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bd10870 con 0x56064bd0f330 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bd0eb90 con 0x56064bd0fc50 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 0x56064bd14880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 0x56064bd14880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42056/0 (socket says 172.21.15.44:42056) 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 -- 172.21.15.44:0/123459798 learned_addr learned my addr 172.21.15.44:0/123459798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe645d9b640 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd14340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe64659c640 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0f330 0x56064bd199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 -- 172.21.15.44:0/123459798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 msgr2=0x56064bd14340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd14340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 -- 172.21.15.44:0/123459798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0f330 msgr2=0x56064bd199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0f330 0x56064bd199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 -- 172.21.15.44:0/123459798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064bc14300 con 0x56064bd1a4f0 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.878+0000 7fe646d9d640 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 0x56064bd14880 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe640011f50 tx=0x7fe640009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fb29254ea91cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe6357fa640 1 -- 172.21.15.44:0/123459798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640011c40 con 0x56064bd1a4f0 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe6357fa640 1 -- 172.21.15.44:0/123459798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe640004e10 con 0x56064bd1a4f0 2024-09-13T17:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe6357fa640 1 -- 172.21.15.44:0/123459798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640027bd0 con 0x56064bd1a4f0 2024-09-13T17:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 -- 172.21.15.44:0/123459798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 msgr2=0x56064bd14880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 0x56064bd14880 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe640011f50 tx=0x7fe640009440 comp rx=0 tx=0).stop 2024-09-13T17:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 -- 172.21.15.44:0/123459798 shutdown_connections 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd0fc50 0x56064bd14340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd1a4f0 0x56064bd14880 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/123459798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0f330 0x56064bd199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 -- 172.21.15.44:0/123459798 >> 172.21.15.44:0/123459798 conn(0x56064bd0b8c0 msgr2=0x56064bd159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 -- 172.21.15.44:0/123459798 shutdown_connections 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.879+0000 7fe64f5eb740 1 -- 172.21.15.44:0/123459798 wait complete. 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 -- start start 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 Processor -- start 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd1fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0fc50 0x56064bd20270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd1a4f0 0x56064bd207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bd802e0 con 0x56064bd0f330 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bd80420 con 0x56064bd0fc50 2024-09-13T17:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064bd80560 con 0x56064bd1a4f0 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64f5eb740 10 client.0 _ll_register_callbacks cb 0x7fe640009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd1fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd1fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42062/0 (socket says 172.21.15.44:42062) 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 -- 172.21.15.44:0/1482447471 learned_addr learned my addr 172.21.15.44:0/1482447471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe645d9b640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0fc50 0x56064bd20270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe646d9d640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd1a4f0 0x56064bd207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 -- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd1a4f0 msgr2=0x56064bd207b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd1a4f0 0x56064bd207b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 -- 172.21.15.44:0/1482447471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0fc50 msgr2=0x56064bd20270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0fc50 0x56064bd20270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe64659c640 1 -- 172.21.15.44:0/1482447471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56064bc1a910 con 0x56064bd0f330 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.880+0000 7fe646d9d640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd1a4f0 0x56064bd207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64659c640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd1fd30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe6380985b0 tx=0x7fe638014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b837f92de4bc96d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe62dffb640 1 -- 172.21.15.44:0/1482447471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380af020 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe62dffb640 1 -- 172.21.15.44:0/1482447471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe6380a26e0 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe640011da0 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe62dffb640 1 -- 172.21.15.44:0/1482447471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6380cad30 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 10 client.16095 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56064bc398e0 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe62dffb640 1 -- 172.21.15.44:0/1482447471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe63807f020 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe62dffb640 1 -- 172.21.15.44:0/1482447471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6380a22e0 con 0x56064bd0f330 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 10 client.16095 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 20 client.16095 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd81820 0x56064bd83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 4 client.16095 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:28:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.881+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56064b967300 con 0x56064bd81820 2024-09-13T17:28:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.882+0000 7fe645d9b640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd81820 0x56064bd83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.882+0000 7fe645d9b640 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd81820 0x56064bd83c10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7200f7bd730f5276 server_cookie=1219a8db5cd3762b in_seq=0 out_seq=0 2024-09-13T17:28:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.882+0000 7fe62dffb640 10 client.16095 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe62dffb640 1 -- 172.21.15.44:0/1482447471 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56064b967300 con 0x56064bd81820 2024-09-13T17:28:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe62dffb640 10 client.16095 handle_command_reply: tid=0 2024-09-13T17:28:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 client.16095 shutdown 2024-09-13T17:28:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 msgr2=0x56064bd1fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd1fd30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe6380985b0 tx=0x7fe638014f90 comp rx=0 tx=0).stop 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 shutdown_connections 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56064bd81820 0x56064bd83c10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064bd1a4f0 0x56064bd207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56064bd0fc50 0x56064bd20270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 --2- 172.21.15.44:0/1482447471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064bd0f330 0x56064bd1fd30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe62dffb640 0 client.16095 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 >> 172.21.15.44:0/1482447471 conn(0x56064bd0b8c0 msgr2=0x56064bd16850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 shutdown_connections 2024-09-13T17:28:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 1 -- 172.21.15.44:0/1482447471 wait complete. 2024-09-13T17:28:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64f5eb740 20 client.16095 trim_cache size 0 max 16384 2024-09-13T17:28:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:28:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:28:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:28:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6300a12b0 msgr2=0x7fe6300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6300a12b0 0x7fe6300a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe64001b000 tx=0x7fe64001d000 comp rx=0 tx=0).stop 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 msgr2=0x7fe64810c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.894+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 0x7fe64810c3d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe63800ea60 tx=0x7fe63800ef20 comp rx=0 tx=0).stop 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 shutdown_connections 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6300a12b0 0x7fe6300a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe64810ce50 0x7fe6481a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe648103f70 0x7fe64810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 --2- 172.21.15.44:0/1588893215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6480fee20 0x7fe64810c3d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 >> 172.21.15.44:0/1588893215 conn(0x7fe6480fa8f0 msgr2=0x7fe6480fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 shutdown_connections 2024-09-13T17:28:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:24.895+0000 7fe64c8c9640 1 -- 172.21.15.44:0/1588893215 wait complete. 2024-09-13T17:28:25.354 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:28:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/277958041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c080fee60 msgr2=0x7f1c080ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/277958041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c080fee60 0x7f1c080ff240 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009cd0 tx=0x7f1bf802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/277958041 shutdown_connections 2024-09-13T17:28:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/277958041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c080ff810 0x7f1c08103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/277958041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c08103fb0 0x7f1c0810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/277958041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c080fee60 0x7f1c080ff240 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/277958041 >> 172.21.15.44:0/277958041 conn(0x7f1c08075c60 msgr2=0x7f1c080fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/277958041 shutdown_connections 2024-09-13T17:28:25.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/277958041 wait complete. 2024-09-13T17:28:25.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.472+0000 7f1c0e1f4640 1 Processor -- start 2024-09-13T17:28:25.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 -- start start 2024-09-13T17:28:25.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c080fee60 0x7f1c08196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c08103fb0 0x7f1c08196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 0x7f1c0819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0819b560 con 0x7f1c0819ab70 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0819b6d0 con 0x7f1c080fee60 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c0e1f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0819b840 con 0x7f1c08103fb0 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 0x7f1c0819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c080fee60 0x7f1c08196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 0x7f1c0819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42072/0 (socket says 172.21.15.44:42072) 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 -- 172.21.15.44:0/700202841 learned_addr learned my addr 172.21.15.44:0/700202841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c08103fb0 0x7f1c08196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 -- 172.21.15.44:0/700202841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c08103fb0 msgr2=0x7f1c08196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c08103fb0 0x7f1c08196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 -- 172.21.15.44:0/700202841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c080fee60 msgr2=0x7f1c08196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c080fee60 0x7f1c08196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c07fff640 1 -- 172.21.15.44:0/700202841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf8009900 con 0x7f1c0819ab70 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.473+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c08103fb0 0x7f1c08196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.474+0000 7f1c07fff640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 0x7f1c0819b020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b3e0 tx=0x7f1c00009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.474+0000 7f1c04ff9640 1 -- 172.21.15.44:0/700202841 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00011830 con 0x7f1c0819ab70 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.474+0000 7f1c04ff9640 1 -- 172.21.15.44:0/700202841 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c00011e20 con 0x7f1c0819ab70 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.474+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c081a2e20 con 0x7f1c0819ab70 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.474+0000 7f1c04ff9640 1 -- 172.21.15.44:0/700202841 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010e50 con 0x7f1c0819ab70 2024-09-13T17:28:25.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.474+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c081a32d0 con 0x7f1c0819ab70 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.476+0000 7f1c04ff9640 1 -- 172.21.15.44:0/700202841 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1c00011990 con 0x7f1c0819ab70 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.476+0000 7f1c10fee740 1 Processor -- start 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 -- start start 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b6e10 0x56123e1b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b7730 0x56123e1c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 0x56123e1c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e121f20 con 0x56123e1c1fd0 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e11f6b0 con 0x56123e1b6e10 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c10fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e11afa0 con 0x56123e1b7730 2024-09-13T17:28:25.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 0x56123e1c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 0x56123e1c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42086/0 (socket says 172.21.15.44:42086) 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 -- 172.21.15.44:0/308902790 learned_addr learned my addr 172.21.15.44:0/308902790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c04ff9640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1bf00a1280 0x7f1bf00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c077fe640 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b7730 0x56123e1c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b6e10 0x56123e1b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c07fff640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1bf00a1280 0x7f1bf00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 -- 172.21.15.44:0/308902790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b7730 msgr2=0x56123e1c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b7730 0x56123e1c1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 -- 172.21.15.44:0/308902790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b6e10 msgr2=0x56123e1b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b6e10 0x56123e1b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c06ffd640 1 -- 172.21.15.44:0/308902790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123e0d5840 con 0x56123e1c1fd0 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.477+0000 7f1c04ff9640 1 -- 172.21.15.44:0/700202841 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1c000cc9f0 con 0x7f1c0819ab70 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 0x56123e1c43c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc013600 tx=0x7f1bfc00b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496d4ecb28ebf998 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1bedffb640 1 -- 172.21.15.44:0/308902790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc004230 con 0x56123e1c1fd0 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1bedffb640 1 -- 172.21.15.44:0/308902790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bfc004390 con 0x56123e1c1fd0 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c07fff640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1bf00a1280 0x7f1bf00a3740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c00099820 tx=0x7f1c00064d20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1bedffb640 1 -- 172.21.15.44:0/308902790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc004c50 con 0x56123e1c1fd0 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 -- 172.21.15.44:0/308902790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 msgr2=0x56123e1c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 0x56123e1c43c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc013600 tx=0x7f1bfc00b4e0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 -- 172.21.15.44:0/308902790 shutdown_connections 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b7730 0x56123e1c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1c1fd0 0x56123e1c43c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 --2- 172.21.15.44:0/308902790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b6e10 0x56123e1b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 -- 172.21.15.44:0/308902790 >> 172.21.15.44:0/308902790 conn(0x56123e1b3380 msgr2=0x56123e1b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 -- 172.21.15.44:0/308902790 shutdown_connections 2024-09-13T17:28:25.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.478+0000 7f1c10fee740 1 -- 172.21.15.44:0/308902790 wait complete. 2024-09-13T17:28:25.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 -- start start 2024-09-13T17:28:25.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 Processor -- start 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e0bc2a0 con 0x56123e1b7730 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123ddf3430 con 0x56123e1c1fd0 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e09e680 con 0x56123e1b6e10 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c10fee740 10 client.0 _ll_register_callbacks cb 0x56123e114610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42090/0 (socket says 172.21.15.44:42090) 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.479+0000 7f1c077fe640 1 -- 172.21.15.44:0/3406811403 learned_addr learned my addr 172.21.15.44:0/3406811403 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c077fe640 1 -- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 msgr2=0x56123e1c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c077fe640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c077fe640 1 -- 172.21.15.44:0/3406811403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 msgr2=0x56123e1bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c077fe640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c077fe640 1 -- 172.21.15.44:0/3406811403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc00be20 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c077fe640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1bec90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009430 tx=0x7f1bf8030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527a9c791584812d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1be6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1be6ffd640 1 -- 172.21.15.44:0/3406811403 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8041070 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1be6ffd640 1 -- 172.21.15.44:0/3406811403 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1bf8031980 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56123de0e4e0 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1be6ffd640 1 -- 172.21.15.44:0/3406811403 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf8031b40 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c10fee740 10 client.16107 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.480+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56123e119620 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1be6ffd640 1 -- 172.21.15.44:0/3406811403 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56123de0e4e0 con 0x56123e1b7730 2024-09-13T17:28:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1be6ffd640 1 -- 172.21.15.44:0/3406811403 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1bf804d070 con 0x56123e1b7730 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c10fee740 10 client.16107 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c10fee740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c10fee740 20 client.16107 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e1d1e00 0x56123e1d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c10fee740 4 client.16107 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56123de1f7e0 con 0x56123e1d1e00 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.481+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e1d1e00 0x56123e1d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e1d1e00 0x56123e1d4220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9fe04881fe589b server_cookie=18252a40ff5dd435 in_seq=0 out_seq=0 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1be6ffd640 10 client.16107 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1be6ffd640 1 -- 172.21.15.44:0/3406811403 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56123de1f7e0 con 0x56123e1d1e00 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1be6ffd640 10 client.16107 handle_command_reply: tid=0 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 client.16107 shutdown 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 msgr2=0x56123e1bec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1bec90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009430 tx=0x7f1bf8030dd0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 shutdown_connections 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e1d1e00 0x56123e1d4220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1bec90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1be6ffd640 0 client.16107 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3406811403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 >> 172.21.15.44:0/3406811403 conn(0x56123e1b3380 msgr2=0x56123e1b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 shutdown_connections 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.482+0000 7f1c10fee740 1 -- 172.21.15.44:0/3406811403 wait complete. 2024-09-13T17:28:25.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.483+0000 7f1c10fee740 20 client.16107 trim_cache size 0 max 16384 2024-09-13T17:28:25.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 Processor -- start 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 -- start start 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e1b7ca0 con 0x56123e1b7730 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e1b4010 con 0x56123e1c1fd0 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c10fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e1b42e0 con 0x56123e1b6e10 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1ba4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42100/0 (socket says 172.21.15.44:42100) 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 -- 172.21.15.44:0/3194271033 learned_addr learned my addr 172.21.15.44:0/3194271033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 -- 172.21.15.44:0/3194271033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 msgr2=0x56123e1b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 -- 172.21.15.44:0/3194271033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 msgr2=0x56123e1bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.497+0000 7f1c077fe640 1 -- 172.21.15.44:0/3194271033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123e0bb790 con 0x56123e1b7730 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c077fe640 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1ba4d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bf802ae00 tx=0x7f1bf80309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692c8bf632810b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1bee7fc640 1 -- 172.21.15.44:0/3194271033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8041040 con 0x56123e1b7730 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1bee7fc640 1 -- 172.21.15.44:0/3194271033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bf8032e90 con 0x56123e1b7730 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1bee7fc640 1 -- 172.21.15.44:0/3194271033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf803cda0 con 0x56123e1b7730 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 -- 172.21.15.44:0/3194271033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 msgr2=0x56123e1ba4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1ba4d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bf802ae00 tx=0x7f1bf80309c0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 -- 172.21.15.44:0/3194271033 shutdown_connections 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1c1fd0 0x56123e1bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b7730 0x56123e1ba4d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 --2- 172.21.15.44:0/3194271033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1b6e10 0x56123e1b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.498+0000 7f1c10fee740 1 -- 172.21.15.44:0/3194271033 >> 172.21.15.44:0/3194271033 conn(0x56123e1b3380 msgr2=0x56123e1d8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 -- 172.21.15.44:0/3194271033 shutdown_connections 2024-09-13T17:28:25.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 -- 172.21.15.44:0/3194271033 wait complete. 2024-09-13T17:28:25.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 -- start start 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 Processor -- start 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 0x56123e1c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b7730 0x56123e1c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1c1fd0 0x56123e1c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e1c5c40 con 0x56123e1b6e10 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e1c5db0 con 0x56123e1b7730 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123e1c5f70 con 0x56123e1c1fd0 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.499+0000 7f1c10fee740 10 client.0 _ll_register_callbacks cb 0x7f1bf80315a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 0x56123e1c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 0x56123e1c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42108/0 (socket says 172.21.15.44:42108) 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 -- 172.21.15.44:0/2874159263 learned_addr learned my addr 172.21.15.44:0/2874159263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c077fe640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b7730 0x56123e1c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1c1fd0 0x56123e1c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 -- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1c1fd0 msgr2=0x56123e1c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1c1fd0 0x56123e1c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 -- 172.21.15.44:0/2874159263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b7730 msgr2=0x56123e1c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b7730 0x56123e1c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 -- 172.21.15.44:0/2874159263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56123e119620 con 0x56123e1b6e10 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c06ffd640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 0x56123e1c8cc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc02c010 tx=0x7f1bf8031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7750325c886d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1be6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:25.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1be6ffd640 1 -- 172.21.15.44:0/2874159263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc02f070 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1be6ffd640 1 -- 172.21.15.44:0/2874159263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1bfc000e40 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bfc004390 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1be6ffd640 1 -- 172.21.15.44:0/2874159263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bfc02a690 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c10fee740 10 client.16113 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.500+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56123e0e0ea0 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1be6ffd640 1 -- 172.21.15.44:0/2874159263 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bfc004390 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1be6ffd640 1 -- 172.21.15.44:0/2874159263 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1bfc01b030 con 0x56123e1b6e10 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c10fee740 10 client.16113 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c10fee740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c10fee740 20 client.16113 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c10fee740 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e22d990 0x56123e22fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c10fee740 4 client.16113 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56123de72e30 con 0x56123e22d990 2024-09-13T17:28:25.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c077fe640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e22d990 0x56123e22fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1be6ffd640 10 client.16113 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.501+0000 7f1c077fe640 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e22d990 0x56123e22fd80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d9579bf2a8d23e server_cookie=46f5d8f1d11599b1 in_seq=0 out_seq=0 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1be6ffd640 1 -- 172.21.15.44:0/2874159263 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56123de72e30 con 0x56123e22d990 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1be6ffd640 10 client.16113 handle_command_reply: tid=0 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 client.16113 shutdown 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 msgr2=0x56123e1c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 0x56123e1c8cc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc02c010 tx=0x7f1bf8031510 comp rx=0 tx=0).stop 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 shutdown_connections 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56123e22d990 0x56123e22fd80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56123e1c1fd0 0x56123e1c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56123e1b7730 0x56123e1c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 --2- 172.21.15.44:0/2874159263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56123e1b6e10 0x56123e1c8cc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1be6ffd640 0 client.16113 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 >> 172.21.15.44:0/2874159263 conn(0x56123e1b3380 msgr2=0x56123e1b9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 shutdown_connections 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 1 -- 172.21.15.44:0/2874159263 wait complete. 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c10fee740 20 client.16113 trim_cache size 0 max 16384 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:28:25.510 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:28:25.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1bf00a1280 msgr2=0x7f1bf00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1bf00a1280 0x7f1bf00a3740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c00099820 tx=0x7f1c00064d20 comp rx=0 tx=0).stop 2024-09-13T17:28:25.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 msgr2=0x7f1c0819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.502+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 0x7f1c0819b020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b3e0 tx=0x7f1c00009f90 comp rx=0 tx=0).stop 2024-09-13T17:28:25.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 shutdown_connections 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1bf00a1280 0x7f1bf00a3740 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c08103fb0 0x7f1c08196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c0819ab70 0x7f1c0819b020 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 --2- 172.21.15.44:0/700202841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c080fee60 0x7f1c08196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 >> 172.21.15.44:0/700202841 conn(0x7f1c08075c60 msgr2=0x7f1c080677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 shutdown_connections 2024-09-13T17:28:25.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.503+0000 7f1c0e1f4640 1 -- 172.21.15.44:0/700202841 wait complete. 2024-09-13T17:28:25.515 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:28:25.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 -- 172.21.15.44:0/3279430230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ff840 msgr2=0x7faf90103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 --2- 172.21.15.44:0/3279430230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ff840 0x7faf90103880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7faf84009930 tx=0x7faf8402a420 comp rx=0 tx=0).stop 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 -- 172.21.15.44:0/3279430230 shutdown_connections 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 --2- 172.21.15.44:0/3279430230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90103fe0 0x7faf9010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 --2- 172.21.15.44:0/3279430230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ff840 0x7faf90103880 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 --2- 172.21.15.44:0/3279430230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf900fee90 0x7faf900ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 -- 172.21.15.44:0/3279430230 >> 172.21.15.44:0/3279430230 conn(0x7faf900fa900 msgr2=0x7faf900fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 -- 172.21.15.44:0/3279430230 shutdown_connections 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.616+0000 7faf981ae640 1 -- 172.21.15.44:0/3279430230 wait complete. 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 Processor -- start 2024-09-13T17:28:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 -- start start 2024-09-13T17:28:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf900fee90 0x7faf90197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 0x7faf90197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90197b80 0x7faf901a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a2820 con 0x7faf90103fe0 2024-09-13T17:28:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a2990 con 0x7faf90197b80 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf981ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a2b50 con 0x7faf900fee90 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf95749640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 0x7faf90197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf95749640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 0x7faf90197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42126/0 (socket says 172.21.15.44:42126) 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf95749640 1 -- 172.21.15.44:0/1048010806 learned_addr learned my addr 172.21.15.44:0/1048010806 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.617+0000 7faf95f4a640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf900fee90 0x7faf90197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf9674b640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90197b80 0x7faf901a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf95749640 1 -- 172.21.15.44:0/1048010806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf900fee90 msgr2=0x7faf90197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf95749640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf900fee90 0x7faf90197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf95749640 1 -- 172.21.15.44:0/1048010806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90197b80 msgr2=0x7faf901a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf95749640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90197b80 0x7faf901a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf95749640 1 -- 172.21.15.44:0/1048010806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf84009560 con 0x7faf90103fe0 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf9674b640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90197b80 0x7faf901a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf95749640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 0x7faf90197640 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faf84005ab0 tx=0x7faf840329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84032df0 con 0x7faf90103fe0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf84005490 con 0x7faf90103fe0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840035f0 con 0x7faf90103fe0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf901a2dd0 con 0x7faf90103fe0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.618+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901a31d0 con 0x7faf90103fe0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.619+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf90100360 con 0x7faf90103fe0 2024-09-13T17:28:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.620+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf840047c0 con 0x7faf90103fe0 2024-09-13T17:28:25.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.621+0000 7faf8effd640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf780a1200 0x7faf780a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.621+0000 7faf95f4a640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf780a1200 0x7faf780a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.621+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7faf840edef0 con 0x7faf90103fe0 2024-09-13T17:28:25.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.621+0000 7faf95f4a640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf780a1200 0x7faf780a36c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf88005fd0 tx=0x7faf88009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.624+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faf840b7020 con 0x7faf90103fe0 2024-09-13T17:28:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.832+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf900ff270 con 0x7faf90103fe0 2024-09-13T17:28:25.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.833+0000 7faf8effd640 1 -- 172.21.15.44:0/1048010806 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7faf8404a090 con 0x7faf90103fe0 2024-09-13T17:28:25.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:28:25.835 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:28:25.836 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:28:25.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf780a1200 msgr2=0x7faf780a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf780a1200 0x7faf780a36c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf88005fd0 tx=0x7faf88009500 comp rx=0 tx=0).stop 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 msgr2=0x7faf90197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 0x7faf90197640 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faf84005ab0 tx=0x7faf840329f0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 shutdown_connections 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf780a1200 0x7faf780a36c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf90197b80 0x7faf901a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf90103fe0 0x7faf90197640 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 --2- 172.21.15.44:0/1048010806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf900fee90 0x7faf90197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.835+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 >> 172.21.15.44:0/1048010806 conn(0x7faf900fa900 msgr2=0x7faf900fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.836+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 shutdown_connections 2024-09-13T17:28:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.836+0000 7faf981ae640 1 -- 172.21.15.44:0/1048010806 wait complete. 2024-09-13T17:28:25.848 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:28:25.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 -- 172.21.15.44:0/1579046327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc100ee0 msgr2=0x7f45fc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 --2- 172.21.15.44:0/1579046327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc100ee0 0x7f45fc1012c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f802a480 comp rx=0 tx=0).stop 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 -- 172.21.15.44:0/1579046327 shutdown_connections 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 --2- 172.21.15.44:0/1579046327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc10a330 0x7f45fc10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 --2- 172.21.15.44:0/1579046327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45fc101800 0x7f45fc109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 --2- 172.21.15.44:0/1579046327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc100ee0 0x7f45fc1012c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 -- 172.21.15.44:0/1579046327 >> 172.21.15.44:0/1579046327 conn(0x7f45fc0fa8d0 msgr2=0x7f45fc0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 -- 172.21.15.44:0/1579046327 shutdown_connections 2024-09-13T17:28:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.951+0000 7f4604384640 1 -- 172.21.15.44:0/1579046327 wait complete. 2024-09-13T17:28:25.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 Processor -- start 2024-09-13T17:28:25.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 -- start start 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc100ee0 0x7f45fc196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 0x7f45fc1967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45fc10a330 0x7f45fc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc19b1c0 con 0x7f45fc101800 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc19b330 con 0x7f45fc10a330 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4604384640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc19b4f0 con 0x7f45fc100ee0 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f460191f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 0x7f45fc1967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f460191f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 0x7f45fc1967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42142/0 (socket says 172.21.15.44:42142) 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f460191f640 1 -- 172.21.15.44:0/2263341295 learned_addr learned my addr 172.21.15.44:0/2263341295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4602120640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc100ee0 0x7f45fc196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.952+0000 7f4602921640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45fc10a330 0x7f45fc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f460191f640 1 -- 172.21.15.44:0/2263341295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc100ee0 msgr2=0x7f45fc196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f460191f640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc100ee0 0x7f45fc196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f460191f640 1 -- 172.21.15.44:0/2263341295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45fc10a330 msgr2=0x7f45fc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f460191f640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45fc10a330 0x7f45fc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f460191f640 1 -- 172.21.15.44:0/2263341295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8009560 con 0x7f45fc101800 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f460191f640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 0x7f45fc1967d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00d740 tx=0x7f45ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f45f37fe640 1 -- 172.21.15.44:0/2263341295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec009c30 con 0x7f45fc101800 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f45f37fe640 1 -- 172.21.15.44:0/2263341295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45ec00de50 con 0x7f45fc101800 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc19b7d0 con 0x7f45fc101800 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f45f37fe640 1 -- 172.21.15.44:0/2263341295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00f620 con 0x7f45fc101800 2024-09-13T17:28:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.953+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc1a3050 con 0x7f45fc101800 2024-09-13T17:28:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.955+0000 7f45f37fe640 1 -- 172.21.15.44:0/2263341295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f45ec010460 con 0x7f45fc101800 2024-09-13T17:28:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.955+0000 7f46070a4740 1 Processor -- start 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 -- start start 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c5eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c69410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c69950 0x55c001c6bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001bc8cc0 con 0x55c001c5e790 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001bc6450 con 0x55c001c5f0b0 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f46070a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001bc1d40 con 0x55c001c69950 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c5eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c5eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42156/0 (socket says 172.21.15.44:42156) 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 -- 172.21.15.44:0/501246046 learned_addr learned my addr 172.21.15.44:0/501246046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602120640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c69410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f45f37fe640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45e80a1280 0x7f45e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f460191f640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c69950 0x55c001c6bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 -- 172.21.15.44:0/501246046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c69950 msgr2=0x55c001c6bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c69950 0x55c001c6bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 -- 172.21.15.44:0/501246046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 msgr2=0x55c001c69410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c69410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602921640 1 -- 172.21.15.44:0/501246046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c001b89620 con 0x55c001c5e790 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f460191f640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c69950 0x55c001c6bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f4602120640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c69410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.956+0000 7f45f37fe640 1 -- 172.21.15.44:0/2263341295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f45ec0ccae0 con 0x7f45fc101800 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.957+0000 7f4602921640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45e80a1280 0x7f45e80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.957+0000 7f4602921640 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c5eb70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45f400a5d0 tx=0x7f45f400aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e96e80e90612f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.957+0000 7f45f0e05640 1 -- 172.21.15.44:0/501246046 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4021ba0 con 0x55c001c5e790 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.957+0000 7f45f0e05640 1 -- 172.21.15.44:0/501246046 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45f4025070 con 0x55c001c5e790 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.957+0000 7f45f0e05640 1 -- 172.21.15.44:0/501246046 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4010030 con 0x55c001c5e790 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 -- 172.21.15.44:0/501246046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 msgr2=0x55c001c5eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f4602921640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45e80a1280 0x7f45e80a3740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f45f4020000 tx=0x7f45f4021000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c5eb70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45f400a5d0 tx=0x7f45f400aa90 comp rx=0 tx=0).stop 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 -- 172.21.15.44:0/501246046 shutdown_connections 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c69950 0x55c001c6bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c69410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- 172.21.15.44:0/501246046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c5eb70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 -- 172.21.15.44:0/501246046 >> 172.21.15.44:0/501246046 conn(0x55c001c5ad00 msgr2=0x55c001c5b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 -- 172.21.15.44:0/501246046 shutdown_connections 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 -- 172.21.15.44:0/501246046 wait complete. 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 -- start start 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 Processor -- start 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5e790 0x55c001c68f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.958+0000 7f46070a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f46070a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001b80180 con 0x55c001c69950 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f46070a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00189a430 con 0x55c001c5f0b0 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f46070a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001b455e0 con 0x55c001c5e790 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f46070a4740 10 client.0 _ll_register_callbacks cb 0x55c001bb1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42166/0 (socket says 172.21.15.44:42166) 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 -- 172.21.15.44:0/3287288998 learned_addr learned my addr 172.21.15.44:0/3287288998 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f460191f640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602120640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5e790 0x55c001c68f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 -- 172.21.15.44:0/3287288998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5e790 msgr2=0x55c001c68f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5e790 0x55c001c68f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 -- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 msgr2=0x55c001c66260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c66260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 -- 172.21.15.44:0/3287288998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4009cb0 con 0x55c001c69950 2024-09-13T17:28:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f4602921640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c667a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45f4004b90 tx=0x7f45f4009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23189323b8665a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f45e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f45e5ffb640 1 -- 172.21.15.44:0/3287288998 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4021920 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f45e5ffb640 1 -- 172.21.15.44:0/3287288998 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f45f4011070 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.959+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c001bebb80 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f45e5ffb640 1 -- 172.21.15.44:0/3287288998 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f4021a80 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 10 client.16131 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c001bc03c0 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f45e5ffb640 1 -- 172.21.15.44:0/3287288998 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c001bebb80 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f45e5ffb640 1 -- 172.21.15.44:0/3287288998 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45f4030020 con 0x55c001c69950 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 10 client.16131 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 10 client.16131 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 20 client.16131 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001c791b0 0x55c001c7b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 4 client.16131 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c001bb41f0 con 0x55c001c791b0 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.960+0000 7f4602120640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001c791b0 0x55c001c7b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f4602120640 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001c791b0 0x55c001c7b5a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f4822b4f49e0fa7 server_cookie=45f7e49213a223d in_seq=0 out_seq=0 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f45e5ffb640 10 client.16131 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f45e5ffb640 1 -- 172.21.15.44:0/3287288998 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c001bb41f0 con 0x55c001c791b0 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f45e5ffb640 10 client.16131 handle_command_reply: tid=0 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f46070a4740 1 client.16131 shutdown 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 msgr2=0x55c001c667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f46070a4740 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c667a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45f4004b90 tx=0x7f45f4009970 comp rx=0 tx=0).stop 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.961+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 shutdown_connections 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001c791b0 0x55c001c7b5a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c667a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5f0b0 0x55c001c66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f45e5ffb640 0 client.16131 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 --2- 172.21.15.44:0/3287288998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5e790 0x55c001c68f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 >> 172.21.15.44:0/3287288998 conn(0x55c001c5ad00 msgr2=0x55c001c60a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 shutdown_connections 2024-09-13T17:28:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 1 -- 172.21.15.44:0/3287288998 wait complete. 2024-09-13T17:28:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.962+0000 7f46070a4740 20 client.16131 trim_cache size 0 max 16384 2024-09-13T17:28:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 Processor -- start 2024-09-13T17:28:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 -- start start 2024-09-13T17:28:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c70610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c70b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c69950 0x55c001c61360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001c76000 con 0x55c001c5e790 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001c69520 con 0x55c001c69950 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f46070a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001c553b0 con 0x55c001c5f0b0 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c70610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c70610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42174/0 (socket says 172.21.15.44:42174) 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 -- 172.21.15.44:0/2561391113 learned_addr learned my addr 172.21.15.44:0/2561391113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602921640 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c69950 0x55c001c61360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 -- 172.21.15.44:0/2561391113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 msgr2=0x55c001c70b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c70b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 -- 172.21.15.44:0/2561391113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c69950 msgr2=0x55c001c61360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c69950 0x55c001c61360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 -- 172.21.15.44:0/2561391113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c001b62770 con 0x55c001c5e790 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602921640 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c69950 0x55c001c61360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f4602120640 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c70610 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f80036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711d6c72c134735 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.973+0000 7f45f1606640 1 -- 172.21.15.44:0/2561391113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8042030 con 0x55c001c5e790 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f45f1606640 1 -- 172.21.15.44:0/2561391113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45f8043040 con 0x55c001c5e790 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f45f1606640 1 -- 172.21.15.44:0/2561391113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8046a90 con 0x55c001c5e790 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 -- 172.21.15.44:0/2561391113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 msgr2=0x55c001c70610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c70610 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f80036a0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 -- 172.21.15.44:0/2561391113 shutdown_connections 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c70b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c69950 0x55c001c61360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 --2- 172.21.15.44:0/2561391113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c5e790 0x55c001c70610 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 -- 172.21.15.44:0/2561391113 >> 172.21.15.44:0/2561391113 conn(0x55c001c5ad00 msgr2=0x55c001c64b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 -- 172.21.15.44:0/2561391113 shutdown_connections 2024-09-13T17:28:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 -- 172.21.15.44:0/2561391113 wait complete. 2024-09-13T17:28:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 -- start start 2024-09-13T17:28:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.974+0000 7f46070a4740 1 Processor -- start 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5e790 0x55c001c70b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c62d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001c63bf0 con 0x55c001c69950 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001ccf680 con 0x55c001c5e790 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c001ccf7c0 con 0x55c001c5f0b0 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f46070a4740 10 client.0 _ll_register_callbacks cb 0x7f45f4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42180/0 (socket says 172.21.15.44:42180) 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 -- 172.21.15.44:0/1491747623 learned_addr learned my addr 172.21.15.44:0/1491747623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602120640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5e790 0x55c001c70b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f460191f640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c62d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 -- 172.21.15.44:0/1491747623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 msgr2=0x55c001c62d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c62d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 -- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5e790 msgr2=0x55c001c70b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5e790 0x55c001c70b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 -- 172.21.15.44:0/1491747623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f8004470 con 0x55c001c69950 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f460191f640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c62d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f4602921640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c634f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f45f4029cd0 tx=0x7f45f4029d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510a4ad53145fa4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.975+0000 7f45e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f45e5ffb640 1 -- 172.21.15.44:0/1491747623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f401e570 con 0x55c001c69950 2024-09-13T17:28:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f45e5ffb640 1 -- 172.21.15.44:0/1491747623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f45f401e6d0 con 0x55c001c69950 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45f4025070 con 0x55c001c69950 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f45e5ffb640 1 -- 172.21.15.44:0/1491747623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f401e890 con 0x55c001c69950 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 10 client.16137 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c001b87e80 con 0x55c001c69950 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f45e5ffb640 1 -- 172.21.15.44:0/1491747623 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45f4025070 con 0x55c001c69950 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f45e5ffb640 1 -- 172.21.15.44:0/1491747623 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45f4030030 con 0x55c001c69950 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 10 client.16137 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 10 client.16137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 20 client.16137 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001cd0a40 0x55c001cd2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 4 client.16137 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0018b54e0 con 0x55c001cd0a40 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.976+0000 7f4602120640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001cd0a40 0x55c001cd2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f45e5ffb640 10 client.16137 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f4602120640 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001cd0a40 0x55c001cd2e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9caf2c53ee54388b server_cookie=f7a3d36b6a8eefba in_seq=0 out_seq=0 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f45e5ffb640 1 -- 172.21.15.44:0/1491747623 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0018b54e0 con 0x55c001cd0a40 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f45e5ffb640 10 client.16137 handle_command_reply: tid=0 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 client.16137 shutdown 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 msgr2=0x55c001c634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c634f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f45f4029cd0 tx=0x7f45f4029d00 comp rx=0 tx=0).stop 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 shutdown_connections 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c001cd0a40 0x55c001cd2e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c001c5f0b0 0x55c001c62d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f45e5ffb640 0 client.16137 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c001c69950 0x55c001c634f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 --2- 172.21.15.44:0/1491747623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c001c5e790 0x55c001c70b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 >> 172.21.15.44:0/1491747623 conn(0x55c001c5ad00 msgr2=0x55c001c61360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 shutdown_connections 2024-09-13T17:28:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 1 -- 172.21.15.44:0/1491747623 wait complete. 2024-09-13T17:28:25.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.977+0000 7f46070a4740 20 client.16137 trim_cache size 0 max 16384 2024-09-13T17:28:25.986 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45e80a1280 msgr2=0x7f45e80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45e80a1280 0x7f45e80a3740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f45f4020000 tx=0x7f45f4021000 comp rx=0 tx=0).stop 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 msgr2=0x7f45fc1967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 0x7f45fc1967d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00d740 tx=0x7f45ec00dc00 comp rx=0 tx=0).stop 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 shutdown_connections 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45e80a1280 0x7f45e80a3740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45fc10a330 0x7f45fc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45fc101800 0x7f45fc1967d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 --2- 172.21.15.44:0/2263341295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc100ee0 0x7f45fc196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 >> 172.21.15.44:0/2263341295 conn(0x7f45fc0fa8d0 msgr2=0x7f45fc0fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 shutdown_connections 2024-09-13T17:28:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:25.978+0000 7f4604384640 1 -- 172.21.15.44:0/2263341295 wait complete. 2024-09-13T17:28:25.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:28:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.092+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1756704683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 msgr2=0x7fe4d8101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.092+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1756704683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8101380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009900 tx=0x7fe4c802a410 comp rx=0 tx=0).stop 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.092+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1756704683 shutdown_connections 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.092+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1756704683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d810a370 0x7fe4d810c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1756704683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4d81018c0 0x7fe4d8109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1756704683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8101380 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1756704683 >> 172.21.15.44:0/1756704683 conn(0x7fe4d80fa930 msgr2=0x7fe4d80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1756704683 shutdown_connections 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1756704683 wait complete. 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 Processor -- start 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- start start 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d81018c0 0x7fe4d8196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4d810a370 0x7fe4d819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d819b040 con 0x7fe4d8100fa0 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d819b1b0 con 0x7fe4d810a370 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4dd6e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d819b370 con 0x7fe4d81018c0 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42202/0 (socket says 172.21.15.44:42202) 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.093+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1049106041 learned_addr learned my addr 172.21.15.44:0/1049106041 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4d810a370 0x7fe4d819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4cffff640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d81018c0 0x7fe4d8196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1049106041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d81018c0 msgr2=0x7fe4d8196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d81018c0 0x7fe4d8196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1049106041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4d810a370 msgr2=0x7fe4d819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4d810a370 0x7fe4d819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1049106041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8009560 con 0x7fe4d8100fa0 2024-09-13T17:28:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8196110 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80057c0 tx=0x7fe4c8032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d4ff9640 1 -- 172.21.15.44:0/1049106041 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8005000 con 0x7fe4d8100fa0 2024-09-13T17:28:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d4ff9640 1 -- 172.21.15.44:0/1049106041 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4c80046c0 con 0x7fe4d8100fa0 2024-09-13T17:28:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d819b5f0 con 0x7fe4d8100fa0 2024-09-13T17:28:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4d4ff9640 1 -- 172.21.15.44:0/1049106041 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8003850 con 0x7fe4d8100fa0 2024-09-13T17:28:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.094+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d819b890 con 0x7fe4d8100fa0 2024-09-13T17:28:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4d4ff9640 1 -- 172.21.15.44:0/1049106041 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe4c8005160 con 0x7fe4d8100fa0 2024-09-13T17:28:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 Processor -- start 2024-09-13T17:28:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 -- start start 2024-09-13T17:28:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3af8cc0 con 0x55a2b3b8e790 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3af6450 con 0x55a2b3b99950 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.097+0000 7fe4e0404740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3af1d40 con 0x55a2b3b8f0b0 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b8eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42214/0 (socket says 172.21.15.44:42214) 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 -- 172.21.15.44:0/3443660487 learned_addr learned my addr 172.21.15.44:0/3443660487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d4ff9640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe4c40a1200 0x7fe4c40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4cffff640 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 -- 172.21.15.44:0/3443660487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 msgr2=0x55a2b3b99410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b99410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 -- 172.21.15.44:0/3443660487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 msgr2=0x55a2b3b9bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b9bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 -- 172.21.15.44:0/3443660487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b3ab9620 con 0x55a2b3b8e790 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe4c40a1200 0x7fe4c40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d4ff9640 1 -- 172.21.15.44:0/1049106041 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe4c80edb80 con 0x7fe4d8100fa0 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b8eb70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0011590 tx=0x7fe4d0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a2e01a39c5d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4cd7fa640 1 -- 172.21.15.44:0/3443660487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001bb90 con 0x55a2b3b8e790 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4cd7fa640 1 -- 172.21.15.44:0/3443660487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d001f070 con 0x55a2b3b8e790 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.098+0000 7fe4cd7fa640 1 -- 172.21.15.44:0/3443660487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0025d50 con 0x55a2b3b8e790 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 -- 172.21.15.44:0/3443660487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 msgr2=0x55a2b3b8eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe4c40a1200 0x7fe4c40a36c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0022000 tx=0x7fe4d0023000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b8eb70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0011590 tx=0x7fe4d0011a50 comp rx=0 tx=0).stop 2024-09-13T17:28:26.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 -- 172.21.15.44:0/3443660487 shutdown_connections 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b9bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 --2- 172.21.15.44:0/3443660487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b8eb70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 -- 172.21.15.44:0/3443660487 >> 172.21.15.44:0/3443660487 conn(0x55a2b3b8ad00 msgr2=0x55a2b3b8b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 -- 172.21.15.44:0/3443660487 shutdown_connections 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.099+0000 7fe4e0404740 1 -- 172.21.15.44:0/3443660487 wait complete. 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 -- start start 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 Processor -- start 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b98f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b96780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3ab0180 con 0x55a2b3b8e790 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b37ca430 con 0x55a2b3b99950 2024-09-13T17:28:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3a755e0 con 0x55a2b3b8f0b0 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4e0404740 10 client.0 _ll_register_callbacks cb 0x55a2b3ae1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b98f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b98f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42222/0 (socket says 172.21.15.44:42222) 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1118200699 learned_addr learned my addr 172.21.15.44:0/1118200699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b96780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4cffff640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 msgr2=0x55a2b3b96240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b96240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1118200699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 msgr2=0x55a2b3b96780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b96780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1118200699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d0009c50 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b98f20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80b7430 tx=0x7fe4c80ba3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fff0d17436d31c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.100+0000 7fe4be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4be7fc640 1 -- 172.21.15.44:0/1118200699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80ba9c0 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4be7fc640 1 -- 172.21.15.44:0/1118200699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe4c80f3030 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2b3b1bb80 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4be7fc640 1 -- 172.21.15.44:0/1118200699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c80eb1d0 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 10 client.16149 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a2b3af03c0 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4be7fc640 1 -- 172.21.15.44:0/1118200699 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4c80d2030 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4be7fc640 1 -- 172.21.15.44:0/1118200699 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4c80b9eb0 con 0x55a2b3b8e790 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 10 client.16149 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 10 client.16149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 20 client.16149 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3ba8fe0 0x55a2b3bab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 4 client.16149 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.101+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2b3ae41f0 con 0x55a2b3ba8fe0 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.102+0000 7fe4cffff640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3ba8fe0 0x55a2b3bab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.102+0000 7fe4be7fc640 10 client.16149 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.102+0000 7fe4cffff640 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3ba8fe0 0x55a2b3bab3d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0ebad7b1de8c8cf server_cookie=709f763fa79c689f in_seq=0 out_seq=0 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4be7fc640 1 -- 172.21.15.44:0/1118200699 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a2b3ae41f0 con 0x55a2b3ba8fe0 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4be7fc640 10 client.16149 handle_command_reply: tid=0 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 client.16149 shutdown 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 msgr2=0x55a2b3b98f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b98f20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80b7430 tx=0x7fe4c80ba3e0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 shutdown_connections 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3ba8fe0 0x55a2b3bab3d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b99950 0x55a2b3b96780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4be7fc640 0 client.16149 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1118200699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3b98f20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 >> 172.21.15.44:0/1118200699 conn(0x55a2b3b8ad00 msgr2=0x55a2b3b89eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 shutdown_connections 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 1 -- 172.21.15.44:0/1118200699 wait complete. 2024-09-13T17:28:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.103+0000 7fe4e0404740 20 client.16149 trim_cache size 0 max 16384 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.114+0000 7fe4e0404740 1 Processor -- start 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.114+0000 7fe4e0404740 1 -- start start 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.114+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3ba0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4e0404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3b8f750 con 0x55a2b3b8e790 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4e0404740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3b96af0 con 0x55a2b3b8f0b0 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4e0404740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3b8be70 con 0x55a2b3b99950 2024-09-13T17:28:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42234/0 (socket says 172.21.15.44:42234) 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1483913459 learned_addr learned my addr 172.21.15.44:0/1483913459 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1483913459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 msgr2=0x55a2b3b989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1483913459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 msgr2=0x55a2b3ba0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4cffff640 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3ba0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3ba0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/1483913459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b3a92770 con 0x55a2b3b8e790 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4cffff640 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3ba0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80b6ec0 tx=0x7fe4c80ba270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fa2c40fef6b80b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4cdffb640 1 -- 172.21.15.44:0/1483913459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80bac20 con 0x55a2b3b8e790 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4cdffb640 1 -- 172.21.15.44:0/1483913459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4c80d2030 con 0x55a2b3b8e790 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.115+0000 7fe4cdffb640 1 -- 172.21.15.44:0/1483913459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b70d0 con 0x55a2b3b8e790 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 -- 172.21.15.44:0/1483913459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 msgr2=0x55a2b3ba0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80b6ec0 tx=0x7fe4c80ba270 comp rx=0 tx=0).stop 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 -- 172.21.15.44:0/1483913459 shutdown_connections 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3ba0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 --2- 172.21.15.44:0/1483913459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0610 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 -- 172.21.15.44:0/1483913459 >> 172.21.15.44:0/1483913459 conn(0x55a2b3b8ad00 msgr2=0x55a2b3b94af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 -- 172.21.15.44:0/1483913459 shutdown_connections 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 -- 172.21.15.44:0/1483913459 wait complete. 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 -- start start 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.116+0000 7fe4e0404740 1 Processor -- start 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b92c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3b93810 con 0x55a2b3b8e790 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3b93980 con 0x55a2b3b8f0b0 2024-09-13T17:28:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b3bff450 con 0x55a2b3b99950 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 10 client.0 _ll_register_callbacks cb 0x7fe4d0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42244/0 (socket says 172.21.15.44:42244) 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/2116522712 learned_addr learned my addr 172.21.15.44:0/2116522712 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4cffff640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b92c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 msgr2=0x55a2b3b93160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b93160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/2116522712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 msgr2=0x55a2b3b92c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b92c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 -- 172.21.15.44:0/2116522712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2b3af03c0 con 0x55a2b3b8e790 2024-09-13T17:28:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4cffff640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b92c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d77fe640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4d6ffd640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0970 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe4c802aa00 tx=0x7fe4c80c2e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b796c8a73d85a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4be7fc640 1 -- 172.21.15.44:0/2116522712 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80bac20 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4be7fc640 1 -- 172.21.15.44:0/2116522712 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe4c80f3070 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d001f070 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.117+0000 7fe4be7fc640 1 -- 172.21.15.44:0/2116522712 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c80ba9c0 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 10 client.16155 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a2b3ab7e80 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4be7fc640 1 -- 172.21.15.44:0/2116522712 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d001f070 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4be7fc640 1 -- 172.21.15.44:0/2116522712 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4c80b67b0 con 0x55a2b3b8e790 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 10 client.16155 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 10 client.16155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:28:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 20 client.16155 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3c005d0 0x55a2b3c029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 4 client.16155 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2b37e54e0 con 0x55a2b3c005d0 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.118+0000 7fe4cffff640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3c005d0 0x55a2b3c029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4be7fc640 10 client.16155 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4cffff640 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3c005d0 0x55a2b3c029c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfb810ffdbeb5ff8 server_cookie=f8e6caad0c70ee74 in_seq=0 out_seq=0 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4be7fc640 1 -- 172.21.15.44:0/2116522712 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2b37e54e0 con 0x55a2b3c005d0 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4be7fc640 10 client.16155 handle_command_reply: tid=0 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 client.16155 shutdown 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 msgr2=0x55a2b3ba0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0970 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe4c802aa00 tx=0x7fe4c80c2e50 comp rx=0 tx=0).stop 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 shutdown_connections 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55a2b3c005d0 0x55a2b3c029c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b3b99950 0x55a2b3b93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4be7fc640 0 client.16155 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a2b3b8f0b0 0x55a2b3b92c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 --2- 172.21.15.44:0/2116522712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a2b3b8e790 0x55a2b3ba0970 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 >> 172.21.15.44:0/2116522712 conn(0x55a2b3b8ad00 msgr2=0x55a2b3b91360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 shutdown_connections 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 1 -- 172.21.15.44:0/2116522712 wait complete. 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.119+0000 7fe4e0404740 20 client.16155 trim_cache size 0 max 16384 2024-09-13T17:28:26.130 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:28:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe4c40a1200 msgr2=0x7fe4c40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe4c40a1200 0x7fe4c40a36c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0022000 tx=0x7fe4d0023000 comp rx=0 tx=0).stop 2024-09-13T17:28:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 msgr2=0x7fe4d8196110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8196110 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80057c0 tx=0x7fe4c8032be0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 shutdown_connections 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe4c40a1200 0x7fe4c40a36c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d81018c0 0x7fe4d8196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4d810a370 0x7fe4d819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 --2- 172.21.15.44:0/1049106041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d8100fa0 0x7fe4d8196110 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 >> 172.21.15.44:0/1049106041 conn(0x7fe4d80fa930 msgr2=0x7fe4d80fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 shutdown_connections 2024-09-13T17:28:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.120+0000 7fe4dd6e5640 1 -- 172.21.15.44:0/1049106041 wait complete. 2024-09-13T17:28:26.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 -- 172.21.15.44:0/2769571396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2694103fd0 msgr2=0x7f269410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 --2- 172.21.15.44:0/2769571396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2694103fd0 0x7f269410bb20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f268c009cd0 tx=0x7f268c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 -- 172.21.15.44:0/2769571396 shutdown_connections 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 --2- 172.21.15.44:0/2769571396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2694103fd0 0x7f269410bb20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 --2- 172.21.15.44:0/2769571396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940ff830 0x7f2694103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 --2- 172.21.15.44:0/2769571396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26940fee80 0x7f26940ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 -- 172.21.15.44:0/2769571396 >> 172.21.15.44:0/2769571396 conn(0x7f26940fa930 msgr2=0x7f26940fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 -- 172.21.15.44:0/2769571396 shutdown_connections 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.236+0000 7f2699d4c640 1 -- 172.21.15.44:0/2769571396 wait complete. 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 Processor -- start 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 -- start start 2024-09-13T17:28:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26940fee80 0x7f26941961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2694103fd0 0x7f2694196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 0x7f269419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419b500 con 0x7f269419ab10 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419b670 con 0x7f2694103fd0 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2699d4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419b7e0 con 0x7f26940fee80 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.237+0000 7f2693fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 0x7f269419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 0x7f269419afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42274/0 (socket says 172.21.15.44:42274) 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 -- 172.21.15.44:0/664322656 learned_addr learned my addr 172.21.15.44:0/664322656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2692ffd640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2694103fd0 0x7f2694196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f26937fe640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26940fee80 0x7f26941961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 -- 172.21.15.44:0/664322656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26940fee80 msgr2=0x7f26941961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26940fee80 0x7f26941961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 -- 172.21.15.44:0/664322656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2694103fd0 msgr2=0x7f2694196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2694103fd0 0x7f2694196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 -- 172.21.15.44:0/664322656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c009900 con 0x7f269419ab10 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2692ffd640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2694103fd0 0x7f2694196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2693fff640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 0x7f269419afc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f268c004af0 tx=0x7f268c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2690ff9640 1 -- 172.21.15.44:0/664322656 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c003970 con 0x7f269419ab10 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2690ff9640 1 -- 172.21.15.44:0/664322656 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f268c004040 con 0x7f269419ab10 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26941a2dc0 con 0x7f269419ab10 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2690ff9640 1 -- 172.21.15.44:0/664322656 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c045650 con 0x7f269419ab10 2024-09-13T17:28:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.238+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26941a3270 con 0x7f269419ab10 2024-09-13T17:28:26.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f2690ff9640 1 -- 172.21.15.44:0/664322656 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f268c00a040 con 0x7f269419ab10 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 Processor -- start 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 -- start start 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472ab9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 0x55d472ac4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472a23cc0 con 0x55d472aba0b0 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472a21450 con 0x55d472ac4950 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f269ca6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472a1cd40 con 0x55d472ab9790 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f26937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 0x55d472ac4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f26937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 0x55d472ac4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42282/0 (socket says 172.21.15.44:42282) 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f26937fe640 1 -- 172.21.15.44:0/3013506413 learned_addr learned my addr 172.21.15.44:0/3013506413 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f2690ff9640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f267c0a1280 0x7f267c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.241+0000 7f2692ffd640 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2692ffd640 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472ab9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f26937fe640 1 -- 172.21.15.44:0/3013506413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 msgr2=0x55d472ab9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2693fff640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f267c0a1280 0x7f267c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f26937fe640 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472ab9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f26937fe640 1 -- 172.21.15.44:0/3013506413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 msgr2=0x55d472ac6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f26937fe640 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f26937fe640 1 -- 172.21.15.44:0/3013506413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4729e4620 con 0x55d472aba0b0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2690ff9640 1 -- 172.21.15.44:0/664322656 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f268c0ee860 con 0x7f269419ab10 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f26937fe640 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 0x55d472ac4410 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2684009bc0 tx=0x7f268400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e45817d9167777 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2679ffb640 1 -- 172.21.15.44:0/3013506413 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400ee70 con 0x55d472aba0b0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2693fff640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f267c0a1280 0x7f267c0a3740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f268c0ec290 tx=0x7f268c0db040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2679ffb640 1 -- 172.21.15.44:0/3013506413 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26840054b0 con 0x55d472aba0b0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.242+0000 7f2679ffb640 1 -- 172.21.15.44:0/3013506413 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684011470 con 0x55d472aba0b0 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 -- 172.21.15.44:0/3013506413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 msgr2=0x55d472ac4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 0x55d472ac4410 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2684009bc0 tx=0x7f268400e6c0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 -- 172.21.15.44:0/3013506413 shutdown_connections 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472aba0b0 0x55d472ac4410 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3013506413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472ab9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 -- 172.21.15.44:0/3013506413 >> 172.21.15.44:0/3013506413 conn(0x55d472ab5d00 msgr2=0x55d472ab60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 -- 172.21.15.44:0/3013506413 shutdown_connections 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 -- 172.21.15.44:0/3013506413 wait complete. 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 -- start start 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.243+0000 7f269ca6f740 1 Processor -- start 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4729db180 con 0x55d472ab9790 2024-09-13T17:28:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4726f5430 con 0x55d472ac4950 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4729a05e0 con 0x55d472aba0b0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f269ca6f740 10 client.0 _ll_register_callbacks cb 0x55d472a0caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42296/0 (socket says 172.21.15.44:42296) 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 -- 172.21.15.44:0/1236114460 learned_addr learned my addr 172.21.15.44:0/1236114460 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f26937fe640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 -- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 msgr2=0x55d472ac1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 -- 172.21.15.44:0/1236114460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 msgr2=0x55d472ac1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 -- 172.21.15.44:0/1236114460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684009820 con 0x55d472ab9790 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2692ffd640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac3ee0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2688012d70 tx=0x7f268800e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ecd24ef2149485 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2672ffd640 1 -- 172.21.15.44:0/1236114460 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688004020 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.244+0000 7f2672ffd640 1 -- 172.21.15.44:0/1236114460 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2688004180 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d472a46b80 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f2672ffd640 1 -- 172.21.15.44:0/1236114460 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2688004340 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 10 client.16167 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d472a1b3c0 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f2672ffd640 1 -- 172.21.15.44:0/1236114460 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d472a46b80 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f2672ffd640 1 -- 172.21.15.44:0/1236114460 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2688011470 con 0x55d472ab9790 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 10 client.16167 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 20 client.16167 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472ad4290 0x55d472ad66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 4 client.16167 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d472a0f1f0 con 0x55d472ad4290 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.245+0000 7f26937fe640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472ad4290 0x55d472ad66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.246+0000 7f26937fe640 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472ad4290 0x55d472ad66b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d55cd82caf78ef5b server_cookie=afd73667f50fb39f in_seq=0 out_seq=0 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.246+0000 7f2672ffd640 10 client.16167 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.246+0000 7f2672ffd640 1 -- 172.21.15.44:0/1236114460 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d472a0f1f0 con 0x55d472ad4290 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.246+0000 7f2672ffd640 10 client.16167 handle_command_reply: tid=0 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.246+0000 7f269ca6f740 1 client.16167 shutdown 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 msgr2=0x55d472ac3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac3ee0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2688012d70 tx=0x7f268800e3b0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 shutdown_connections 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472ad4290 0x55d472ad66b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472ac1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f2672ffd640 0 client.16167 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:26.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 --2- 172.21.15.44:0/1236114460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac3ee0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 >> 172.21.15.44:0/1236114460 conn(0x55d472ab5d00 msgr2=0x55d472ab4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 shutdown_connections 2024-09-13T17:28:26.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 1 -- 172.21.15.44:0/1236114460 wait complete. 2024-09-13T17:28:26.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.247+0000 7f269ca6f740 20 client.16167 trim_cache size 0 max 16384 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 Processor -- start 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 -- start start 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472abfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472a46b80 con 0x55d472ab9790 2024-09-13T17:28:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472ac7400 con 0x55d472ac4950 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f269ca6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472ab0300 con 0x55d472aba0b0 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42298/0 (socket says 172.21.15.44:42298) 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 -- 172.21.15.44:0/2137541496 learned_addr learned my addr 172.21.15.44:0/2137541496 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f26937fe640 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 -- 172.21.15.44:0/2137541496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 msgr2=0x55d472ac1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 -- 172.21.15.44:0/2137541496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 msgr2=0x55d472abfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472abfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 -- 172.21.15.44:0/2137541496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4729bd770 con 0x55d472ab9790 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.258+0000 7f2692ffd640 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac11e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f268800eb90 tx=0x7f26880128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986395d18275eaea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f267a7fc640 1 -- 172.21.15.44:0/2137541496 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268802e070 con 0x55d472ab9790 2024-09-13T17:28:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f267a7fc640 1 -- 172.21.15.44:0/2137541496 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2688019030 con 0x55d472ab9790 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f267a7fc640 1 -- 172.21.15.44:0/2137541496 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688011a90 con 0x55d472ab9790 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 -- 172.21.15.44:0/2137541496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 msgr2=0x55d472ac11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac11e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f268800eb90 tx=0x7f26880128d0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 -- 172.21.15.44:0/2137541496 shutdown_connections 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472aba0b0 0x55d472ac1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472ac4950 0x55d472abfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 --2- 172.21.15.44:0/2137541496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ab9790 0x55d472ac11e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 -- 172.21.15.44:0/2137541496 >> 172.21.15.44:0/2137541496 conn(0x55d472ab5d00 msgr2=0x55d472ab60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 -- 172.21.15.44:0/2137541496 shutdown_connections 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.259+0000 7f269ca6f740 1 -- 172.21.15.44:0/2137541496 wait complete. 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 -- start start 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 Processor -- start 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472acb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472aba0b0 0x55d472ac7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 0x55d472ac82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472ac89a0 con 0x55d472ac4950 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472ac8b10 con 0x55d472aba0b0 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d472b2a790 con 0x55d472ab9790 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f269ca6f740 10 client.0 _ll_register_callbacks cb 0x7f2684009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f2692ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 0x55d472ac82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f2692ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 0x55d472ac82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42302/0 (socket says 172.21.15.44:42302) 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f2692ffd640 1 -- 172.21.15.44:0/3271728627 learned_addr learned my addr 172.21.15.44:0/3271728627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f2692ffd640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472acb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.260+0000 7f26937fe640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472aba0b0 0x55d472ac7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2692ffd640 1 -- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 msgr2=0x55d472acb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2692ffd640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472acb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2692ffd640 1 -- 172.21.15.44:0/3271728627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472aba0b0 msgr2=0x55d472ac7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2692ffd640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472aba0b0 0x55d472ac7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2692ffd640 1 -- 172.21.15.44:0/3271728627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d472a1b3c0 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2692ffd640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 0x55d472ac82a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2688005f30 tx=0x7f2688022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc0624cd7ec2c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2672ffd640 1 -- 172.21.15.44:0/3271728627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268802e030 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2672ffd640 1 -- 172.21.15.44:0/3271728627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f268802f070 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26840054b0 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2672ffd640 1 -- 172.21.15.44:0/3271728627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2688011d30 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 10 client.16173 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4729e2e80 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2672ffd640 1 -- 172.21.15.44:0/3271728627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26840054b0 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f2672ffd640 1 -- 172.21.15.44:0/3271728627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2688004750 con 0x55d472ac4950 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 10 client.16173 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 20 client.16173 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472b2ba00 0x55d472b2de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 4 client.16173 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.261+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d4727104e0 con 0x55d472b2ba00 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f26937fe640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472b2ba00 0x55d472b2de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f2672ffd640 10 client.16173 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f26937fe640 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472b2ba00 0x55d472b2de20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9eaa75f317cb9f1 server_cookie=920f6d7c9d7ebcda in_seq=0 out_seq=0 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f2672ffd640 1 -- 172.21.15.44:0/3271728627 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d4727104e0 con 0x55d472b2ba00 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f2672ffd640 10 client.16173 handle_command_reply: tid=0 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 client.16173 shutdown 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 msgr2=0x55d472ac82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 0x55d472ac82a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2688005f30 tx=0x7f2688022040 comp rx=0 tx=0).stop 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 shutdown_connections 2024-09-13T17:28:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d472b2ba00 0x55d472b2de20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d472ac4950 0x55d472ac82a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f2672ffd640 0 client.16173 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d472aba0b0 0x55d472ac7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.262+0000 7f269ca6f740 1 --2- 172.21.15.44:0/3271728627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d472ab9790 0x55d472acb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 >> 172.21.15.44:0/3271728627 conn(0x55d472ab5d00 msgr2=0x55d472ac11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 shutdown_connections 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f269ca6f740 1 -- 172.21.15.44:0/3271728627 wait complete. 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f269ca6f740 20 client.16173 trim_cache size 0 max 16384 2024-09-13T17:28:26.271 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f267c0a1280 msgr2=0x7f267c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f267c0a1280 0x7f267c0a3740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f268c0ec290 tx=0x7f268c0db040 comp rx=0 tx=0).stop 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 msgr2=0x7f269419afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 0x7f269419afc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f268c004af0 tx=0x7f268c030ec0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 shutdown_connections 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f267c0a1280 0x7f267c0a3740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269419ab10 0x7f269419afc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2694103fd0 0x7f2694196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 --2- 172.21.15.44:0/664322656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26940fee80 0x7f26941961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 >> 172.21.15.44:0/664322656 conn(0x7f26940fa930 msgr2=0x7f26940688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 shutdown_connections 2024-09-13T17:28:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:26.263+0000 7f2699d4c640 1 -- 172.21.15.44:0/664322656 wait complete. 2024-09-13T17:28:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:26 smithi100 ceph-mon[35055]: pgmap v54: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-09-13T17:28:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:26 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1048010806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:28:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:26 smithi044 ceph-mon[31175]: pgmap v54: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-09-13T17:28:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:26 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1048010806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:28:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:26 smithi070 ceph-mon[34502]: pgmap v54: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-09-13T17:28:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:26 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1048010806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:28:27.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:28:27.277 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 -- 172.21.15.44:0/3688788987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc100940 msgr2=0x7fc5fc104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 --2- 172.21.15.44:0/3688788987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc100940 0x7fc5fc104980 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009930 tx=0x7fc5ec02a420 comp rx=0 tx=0).stop 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 -- 172.21.15.44:0/3688788987 shutdown_connections 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 --2- 172.21.15.44:0/3688788987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 --2- 172.21.15.44:0/3688788987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc100940 0x7fc5fc104980 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 --2- 172.21.15.44:0/3688788987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5fc0fff90 0x7fc5fc100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 -- 172.21.15.44:0/3688788987 >> 172.21.15.44:0/3688788987 conn(0x7fc5fc0fba20 msgr2=0x7fc5fc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 -- 172.21.15.44:0/3688788987 shutdown_connections 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.395+0000 7fc603812640 1 -- 172.21.15.44:0/3688788987 wait complete. 2024-09-13T17:28:27.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 Processor -- start 2024-09-13T17:28:27.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 -- start start 2024-09-13T17:28:27.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc0fff90 0x7fc5fc1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5fc19abf0 0x7fc5fc19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc19b5e0 con 0x7fc5fc1050e0 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc19b750 con 0x7fc5fc0fff90 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc603812640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc19b8c0 con 0x7fc5fc19abf0 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42322/0 (socket says 172.21.15.44:42322) 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc600dad640 1 -- 172.21.15.44:0/1642415398 learned_addr learned my addr 172.21.15.44:0/1642415398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc6015ae640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc0fff90 0x7fc5fc1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.396+0000 7fc601daf640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5fc19abf0 0x7fc5fc19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc600dad640 1 -- 172.21.15.44:0/1642415398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5fc19abf0 msgr2=0x7fc5fc19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc600dad640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5fc19abf0 0x7fc5fc19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc600dad640 1 -- 172.21.15.44:0/1642415398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc0fff90 msgr2=0x7fc5fc1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc600dad640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc0fff90 0x7fc5fc1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc600dad640 1 -- 172.21.15.44:0/1642415398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec009560 con 0x7fc5fc1050e0 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc600dad640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc196800 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005ab0 tx=0x7fc5ec032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc5f27fc640 1 -- 172.21.15.44:0/1642415398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0044d0 con 0x7fc5fc1050e0 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc5f27fc640 1 -- 172.21.15.44:0/1642415398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec032e10 con 0x7fc5fc1050e0 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc1a2ea0 con 0x7fc5fc1050e0 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc5f27fc640 1 -- 172.21.15.44:0/1642415398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec003850 con 0x7fc5fc1050e0 2024-09-13T17:28:27.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.397+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc1a3260 con 0x7fc5fc1050e0 2024-09-13T17:28:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.399+0000 7fc5f27fc640 1 -- 172.21.15.44:0/1642415398 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc5ec004850 con 0x7fc5fc1050e0 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 Processor -- start 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 -- start start 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563316435c60 con 0x5633164d5980 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633164333f0 con 0x5633164cb0e0 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc606610740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331642ece0 con 0x5633164ca7c0 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164d7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42324/0 (socket says 172.21.15.44:42324) 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc5f27fc640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5e80a1280 0x7fc5e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.400+0000 7fc600dad640 1 -- 172.21.15.44:0/2904997977 learned_addr learned my addr 172.21.15.44:0/2904997977 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc601daf640 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc601daf640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5e80a1280 0x7fc5e80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc5f27fc640 1 -- 172.21.15.44:0/1642415398 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc5ec0edfa0 con 0x7fc5fc1050e0 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc600dad640 1 -- 172.21.15.44:0/2904997977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 msgr2=0x5633164caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc600dad640 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc600dad640 1 -- 172.21.15.44:0/2904997977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 msgr2=0x5633164d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc600dad640 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc600dad640 1 -- 172.21.15.44:0/2904997977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633163ea970 con 0x5633164d5980 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc600dad640 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164d7d70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0c3240 tx=0x7fc5ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f946d70b4e9eeb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc601daf640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5e80a1280 0x7fc5e80a3740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005c40 tx=0x7fc5f4009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc5e77fe640 1 -- 172.21.15.44:0/2904997977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0ba990 con 0x5633164d5980 2024-09-13T17:28:27.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc5e77fe640 1 -- 172.21.15.44:0/2904997977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec0ba7b0 con 0x5633164d5980 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.401+0000 7fc5e77fe640 1 -- 172.21.15.44:0/2904997977 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0c3360 con 0x5633164d5980 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 -- 172.21.15.44:0/2904997977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 msgr2=0x5633164d7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164d7d70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0c3240 tx=0x7fc5ec009260 comp rx=0 tx=0).stop 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 -- 172.21.15.44:0/2904997977 shutdown_connections 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164d7d70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 --2- 172.21.15.44:0/2904997977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 -- 172.21.15.44:0/2904997977 >> 172.21.15.44:0/2904997977 conn(0x5633164c6d50 msgr2=0x5633164c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 -- 172.21.15.44:0/2904997977 shutdown_connections 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 -- 172.21.15.44:0/2904997977 wait complete. 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.402+0000 7fc606610740 1 -- start start 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 Processor -- start 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164ca7c0 0x5633164d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164d5980 0x5633164d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633163ecf80 con 0x5633164cb0e0 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633163ecc20 con 0x5633164ca7c0 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633163b9ab0 con 0x5633164d5980 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc606610740 10 client.0 _ll_register_callbacks cb 0x563316428350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42336/0 (socket says 172.21.15.44:42336) 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc600dad640 1 -- 172.21.15.44:0/2309288348 learned_addr learned my addr 172.21.15.44:0/2309288348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164ca7c0 0x5633164d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.403+0000 7fc601daf640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164d5980 0x5633164d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc600dad640 1 -- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164d5980 msgr2=0x5633164d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc600dad640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164d5980 0x5633164d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc600dad640 1 -- 172.21.15.44:0/2309288348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164ca7c0 msgr2=0x5633164d5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc600dad640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164ca7c0 0x5633164d5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc600dad640 1 -- 172.21.15.44:0/2309288348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc1a3260 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc600dad640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164d25c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0bade0 tx=0x7fc5ec0eb820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bc04e300daddab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc5e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2309288348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0c3340 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2309288348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc5ec0bae50 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563316122590 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2309288348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ec004e60 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 10 client.16185 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56331642d360 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2309288348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563316122590 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2309288348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5ec0ced20 con 0x5633164cb0e0 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 10 client.16185 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:27.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 20 client.16185 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633164e4f10 0x5633164e7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 4 client.16185 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.404+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563316133860 con 0x5633164e4f10 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.405+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633164e4f10 0x5633164e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.405+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633164e4f10 0x5633164e7300 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62041e08f090b4bd server_cookie=a182fae6b6432cfd in_seq=0 out_seq=0 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.405+0000 7fc5e4ff9640 10 client.16185 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2309288348 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563316133860 con 0x5633164e4f10 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc5e4ff9640 10 client.16185 handle_command_reply: tid=0 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 client.16185 shutdown 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 msgr2=0x5633164d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164d25c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0bade0 tx=0x7fc5ec0eb820 comp rx=0 tx=0).stop 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 shutdown_connections 2024-09-13T17:28:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633164e4f10 0x5633164e7300 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164d5980 0x5633164d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164d25c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc5e4ff9640 0 client.16185 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 --2- 172.21.15.44:0/2309288348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164ca7c0 0x5633164d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 >> 172.21.15.44:0/2309288348 conn(0x5633164c6d50 msgr2=0x5633164cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 shutdown_connections 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 1 -- 172.21.15.44:0/2309288348 wait complete. 2024-09-13T17:28:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.406+0000 7fc606610740 20 client.16185 trim_cache size 0 max 16384 2024-09-13T17:28:27.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 Processor -- start 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 -- start start 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164d5980 0x5633164cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633164d0b40 con 0x5633164cb0e0 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633164e7630 con 0x5633164d5980 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc606610740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633164c7950 con 0x5633164ca7c0 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc600dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164dc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42346/0 (socket says 172.21.15.44:42346) 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc600dad640 1 -- 172.21.15.44:0/4123420876 learned_addr learned my addr 172.21.15.44:0/4123420876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc601daf640 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164d5980 0x5633164cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc6015ae640 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc600dad640 1 -- 172.21.15.44:0/4123420876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 msgr2=0x5633164dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc600dad640 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.421+0000 7fc600dad640 1 -- 172.21.15.44:0/4123420876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164d5980 msgr2=0x5633164cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc600dad640 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164d5980 0x5633164cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc600dad640 1 -- 172.21.15.44:0/4123420876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633163cf1c0 con 0x5633164cb0e0 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc600dad640 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164dc910 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ba920 tx=0x7fc5ec0b77c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70988d349c9972c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc5e7fff640 1 -- 172.21.15.44:0/4123420876 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec004e60 con 0x5633164cb0e0 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc5e7fff640 1 -- 172.21.15.44:0/4123420876 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec0bb550 con 0x5633164cb0e0 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc5e7fff640 1 -- 172.21.15.44:0/4123420876 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0d9170 con 0x5633164cb0e0 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 -- 172.21.15.44:0/4123420876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 msgr2=0x5633164dc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164dc910 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ba920 tx=0x7fc5ec0b77c0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 -- 172.21.15.44:0/4123420876 shutdown_connections 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164d5980 0x5633164cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164cb0e0 0x5633164dc910 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 --2- 172.21.15.44:0/4123420876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.422+0000 7fc606610740 1 -- 172.21.15.44:0/4123420876 >> 172.21.15.44:0/4123420876 conn(0x5633164c6d50 msgr2=0x5633164ebc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 -- 172.21.15.44:0/4123420876 shutdown_connections 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 -- 172.21.15.44:0/4123420876 wait complete. 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 -- start start 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 Processor -- start 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164db580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331653b390 con 0x5633164d5980 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331653b500 con 0x5633164cb0e0 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331653b6c0 con 0x5633164ca7c0 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc606610740 10 client.0 _ll_register_callbacks cb 0x7fc5f800d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc601daf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc601daf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164dbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42358/0 (socket says 172.21.15.44:42358) 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.423+0000 7fc601daf640 1 -- 172.21.15.44:0/2719721249 learned_addr learned my addr 172.21.15.44:0/2719721249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc600dad640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164db580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc601daf640 1 -- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 msgr2=0x5633164db040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc601daf640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc601daf640 1 -- 172.21.15.44:0/2719721249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 msgr2=0x5633164db580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc601daf640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164db580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc601daf640 1 -- 172.21.15.44:0/2719721249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56331642d360 con 0x5633164d5980 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc600dad640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc601daf640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164dbac0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4003cf0 tx=0x7fc5f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f22f519c390b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc5e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2719721249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4003ea0 con 0x5633164d5980 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2719721249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc5f40045b0 con 0x5633164d5980 2024-09-13T17:28:27.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5ec0ba7b0 con 0x5633164d5980 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2719721249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f4004770 con 0x5633164d5980 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc606610740 10 client.16191 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.424+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5633163f4990 con 0x5633164d5980 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2719721249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec0ba7b0 con 0x5633164d5980 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2719721249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5f4005520 con 0x5633164d5980 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc606610740 10 client.16191 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc606610740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc606610740 20 client.16191 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc606610740 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56331653cf30 0x56331653f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc606610740 4 client.16191 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563316186eb0 con 0x56331653cf30 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56331653cf30 0x56331653f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc5e4ff9640 10 client.16191 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.425+0000 7fc6015ae640 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56331653cf30 0x56331653f350 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5284fce167d9d8ed server_cookie=9adf6ff59ae3a984 in_seq=0 out_seq=0 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.426+0000 7fc5e4ff9640 1 -- 172.21.15.44:0/2719721249 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563316186eb0 con 0x56331653cf30 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.426+0000 7fc5e4ff9640 10 client.16191 handle_command_reply: tid=0 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.426+0000 7fc606610740 1 client.16191 shutdown 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.426+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 msgr2=0x5633164dbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.426+0000 7fc606610740 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164dbac0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4003cf0 tx=0x7fc5f4004a40 comp rx=0 tx=0).stop 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 shutdown_connections 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56331653cf30 0x56331653f350 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633164d5980 0x5633164dbac0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc5e4ff9640 0 client.16191 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633164cb0e0 0x5633164db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 --2- 172.21.15.44:0/2719721249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633164ca7c0 0x5633164db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 >> 172.21.15.44:0/2719721249 conn(0x5633164c6d50 msgr2=0x5633164cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 shutdown_connections 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 1 -- 172.21.15.44:0/2719721249 wait complete. 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc606610740 20 client.16191 trim_cache size 0 max 16384 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "04e91014-3a15-41af-966c-287ccb4a59e1", 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:28:27.435 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:28:27.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5e80a1280 msgr2=0x7fc5e80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5e80a1280 0x7fc5e80a3740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005c40 tx=0x7fc5f4009500 comp rx=0 tx=0).stop 2024-09-13T17:28:27.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 msgr2=0x7fc5fc196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc196800 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005ab0 tx=0x7fc5ec032c20 comp rx=0 tx=0).stop 2024-09-13T17:28:27.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 shutdown_connections 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5e80a1280 0x7fc5e80a3740 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5fc19abf0 0x7fc5fc19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5fc1050e0 0x7fc5fc196800 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 --2- 172.21.15.44:0/1642415398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc0fff90 0x7fc5fc1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.427+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 >> 172.21.15.44:0/1642415398 conn(0x7fc5fc0fba20 msgr2=0x7fc5fc0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.428+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 shutdown_connections 2024-09-13T17:28:27.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.428+0000 7fc603812640 1 -- 172.21.15.44:0/1642415398 wait complete. 2024-09-13T17:28:27.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:28:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 -- 172.21.15.44:0/4088071357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec10c670 msgr2=0x7f77ec10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 --2- 172.21.15.44:0/4088071357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec10c670 0x7f77ec10ea60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009ca0 tx=0x7f77e402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 -- 172.21.15.44:0/4088071357 shutdown_connections 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 --2- 172.21.15.44:0/4088071357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec10c670 0x7f77ec10ea60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 --2- 172.21.15.44:0/4088071357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77ec0ff470 0x7f77ec10c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 --2- 172.21.15.44:0/4088071357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ec0feac0 0x7f77ec0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 -- 172.21.15.44:0/4088071357 >> 172.21.15.44:0/4088071357 conn(0x7f77ec075c60 msgr2=0x7f77ec0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 -- 172.21.15.44:0/4088071357 shutdown_connections 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 -- 172.21.15.44:0/4088071357 wait complete. 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.544+0000 7f77f386b640 1 Processor -- start 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 -- start start 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ec0feac0 0x7f77ec1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 0x7f77ec1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77ec10c670 0x7f77ec19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec19d2d0 con 0x7f77ec0ff470 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec19d440 con 0x7f77ec10c670 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f386b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec19d600 con 0x7f77ec0feac0 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 0x7f77ec1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 0x7f77ec1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42364/0 (socket says 172.21.15.44:42364) 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 -- 172.21.15.44:0/1134283411 learned_addr learned my addr 172.21.15.44:0/1134283411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f1607640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ec0feac0 0x7f77ec1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f1e08640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77ec10c670 0x7f77ec19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 -- 172.21.15.44:0/1134283411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ec0feac0 msgr2=0x7f77ec1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ec0feac0 0x7f77ec1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 -- 172.21.15.44:0/1134283411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77ec10c670 msgr2=0x7f77ec19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77ec10c670 0x7f77ec19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 -- 172.21.15.44:0/1134283411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4009900 con 0x7f77ec0ff470 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.545+0000 7f77f0e06640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 0x7f77ec1988e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9f0 tx=0x7f77dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.546+0000 7f77e27fc640 1 -- 172.21.15.44:0/1134283411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc009700 con 0x7f77ec0ff470 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.546+0000 7f77e27fc640 1 -- 172.21.15.44:0/1134283411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77dc00cd80 con 0x7f77ec0ff470 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.546+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec19d8e0 con 0x7f77ec0ff470 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.546+0000 7f77e27fc640 1 -- 172.21.15.44:0/1134283411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0187e0 con 0x7f77ec0ff470 2024-09-13T17:28:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.546+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec1a5160 con 0x7f77ec0ff470 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.548+0000 7f77e27fc640 1 -- 172.21.15.44:0/1134283411 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f77dc010430 con 0x7f77ec0ff470 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.548+0000 7f77f666d740 1 Processor -- start 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 -- start start 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac0637f20 con 0x55bac06d7fd0 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac06356b0 con 0x55bac06cd730 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac0630fa0 con 0x55bac06cce10 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06da3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42370/0 (socket says 172.21.15.44:42370) 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f0e06640 1 -- 172.21.15.44:0/129788514 learned_addr learned my addr 172.21.15.44:0/129788514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77e27fc640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f77d80a1280 0x7f77d80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1607640 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f77d80a1280 0x7f77d80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 -- 172.21.15.44:0/129788514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 msgr2=0x55bac06d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 -- 172.21.15.44:0/129788514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 msgr2=0x55bac06da3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06da3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77f1e08640 1 -- 172.21.15.44:0/129788514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bac05eb840 con 0x55bac06cce10 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.549+0000 7f77e27fc640 1 -- 172.21.15.44:0/1134283411 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f77dc014070 con 0x7f77ec0ff470 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.550+0000 7f77f1e08640 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06cd1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77e402ade0 tx=0x7f77e4031c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933c2d285a8329c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.550+0000 7f77f1e08640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f77d80a1280 0x7f77d80a3740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e404b000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.550+0000 7f77d77fe640 1 -- 172.21.15.44:0/129788514 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4003b70 con 0x55bac06cce10 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.550+0000 7f77d77fe640 1 -- 172.21.15.44:0/129788514 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77e4003cd0 con 0x55bac06cce10 2024-09-13T17:28:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.550+0000 7f77d77fe640 1 -- 172.21.15.44:0/129788514 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40042a0 con 0x55bac06cce10 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- 172.21.15.44:0/129788514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 msgr2=0x55bac06cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06cd1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77e402ade0 tx=0x7f77e4031c40 comp rx=0 tx=0).stop 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- 172.21.15.44:0/129788514 shutdown_connections 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06da3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- 172.21.15.44:0/129788514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06cd1f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- 172.21.15.44:0/129788514 >> 172.21.15.44:0/129788514 conn(0x55bac06c9380 msgr2=0x55bac06c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- 172.21.15.44:0/129788514 shutdown_connections 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- 172.21.15.44:0/129788514 wait complete. 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- start start 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 Processor -- start 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac05d22a0 con 0x55bac06cd730 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac0309430 con 0x55bac06d7fd0 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.551+0000 7f77f666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac05b4680 con 0x55bac06cce10 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f666d740 10 client.0 _ll_register_callbacks cb 0x55bac062a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42384/0 (socket says 172.21.15.44:42384) 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 -- 172.21.15.44:0/1114207756 learned_addr learned my addr 172.21.15.44:0/1114207756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f1607640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f1e08640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 -- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 msgr2=0x55bac06d7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 -- 172.21.15.44:0/1114207756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 msgr2=0x55bac06d5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 -- 172.21.15.44:0/1114207756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec10f640 con 0x55bac06cd730 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f0e06640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d4cc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9c0 tx=0x7f77dc099cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c214ee8ad0212b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77d4ff9640 1 -- 172.21.15.44:0/1114207756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0108e0 con 0x55bac06cd730 2024-09-13T17:28:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77d4ff9640 1 -- 172.21.15.44:0/1114207756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f77dc0a2490 con 0x55bac06cd730 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bac03244e0 con 0x55bac06cd730 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77d4ff9640 1 -- 172.21.15.44:0/1114207756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77dc0ca8d0 con 0x55bac06cd730 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f666d740 10 client.16203 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.552+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bac062f620 con 0x55bac06cd730 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77d4ff9640 1 -- 172.21.15.44:0/1114207756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bac03244e0 con 0x55bac06cd730 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77d4ff9640 1 -- 172.21.15.44:0/1114207756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77dc0a2000 con 0x55bac06cd730 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f666d740 10 client.16203 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f666d740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f666d740 20 client.16203 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f666d740 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac06e7dc0 0x55bac06ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f666d740 4 client.16203 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bac03357e0 con 0x55bac06e7dc0 2024-09-13T17:28:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f1607640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac06e7dc0 0x55bac06ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77f1607640 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac06e7dc0 0x55bac06ea1e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e0fe7ac14491ba server_cookie=f701dd06daa84887 in_seq=0 out_seq=0 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.553+0000 7f77d4ff9640 10 client.16203 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77d4ff9640 1 -- 172.21.15.44:0/1114207756 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bac03357e0 con 0x55bac06e7dc0 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77d4ff9640 10 client.16203 handle_command_reply: tid=0 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 client.16203 shutdown 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 msgr2=0x55bac06d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d4cc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9c0 tx=0x7f77dc099cc0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 shutdown_connections 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac06e7dc0 0x55bac06ea1e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77d4ff9640 0 client.16203 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d4cc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 --2- 172.21.15.44:0/1114207756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 >> 172.21.15.44:0/1114207756 conn(0x55bac06c9380 msgr2=0x55bac06cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 shutdown_connections 2024-09-13T17:28:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.554+0000 7f77f666d740 1 -- 172.21.15.44:0/1114207756 wait complete. 2024-09-13T17:28:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.555+0000 7f77f666d740 20 client.16203 trim_cache size 0 max 16384 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.568+0000 7f77f666d740 1 Processor -- start 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.568+0000 7f77f666d740 1 -- start start 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f666d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f666d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f666d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac03357e0 con 0x55bac06cd730 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f666d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac03244e0 con 0x55bac06d7fd0 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac06ca070 con 0x55bac06cce10 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42400/0 (socket says 172.21.15.44:42400) 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 -- 172.21.15.44:0/3901185093 learned_addr learned my addr 172.21.15.44:0/3901185093 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f1607640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f1e08640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 -- 172.21.15.44:0/3901185093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 msgr2=0x55bac06d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 -- 172.21.15.44:0/3901185093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 msgr2=0x55bac06d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 -- 172.21.15.44:0/3901185093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bac05d1790 con 0x55bac06cd730 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f1e08640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f1607640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77f0e06640 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d7550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f77dc09a000 tx=0x7f77dc099d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4696bc6b8dcc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77d7fff640 1 -- 172.21.15.44:0/3901185093 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0a2490 con 0x55bac06cd730 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77d7fff640 1 -- 172.21.15.44:0/3901185093 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77dc0106e0 con 0x55bac06cd730 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.569+0000 7f77d7fff640 1 -- 172.21.15.44:0/3901185093 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0cab10 con 0x55bac06cd730 2024-09-13T17:28:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 -- 172.21.15.44:0/3901185093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 msgr2=0x55bac06d7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d7550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f77dc09a000 tx=0x7f77dc099d30 comp rx=0 tx=0).stop 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 -- 172.21.15.44:0/3901185093 shutdown_connections 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06d7fd0 0x55bac06d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06cd730 0x55bac06d7550 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 --2- 172.21.15.44:0/3901185093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06d4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 -- 172.21.15.44:0/3901185093 >> 172.21.15.44:0/3901185093 conn(0x55bac06c9380 msgr2=0x55bac06d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 -- 172.21.15.44:0/3901185093 shutdown_connections 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 -- 172.21.15.44:0/3901185093 wait complete. 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 -- start start 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.570+0000 7f77f666d740 1 Processor -- start 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06def30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac06d1f50 con 0x55bac06d7fd0 2024-09-13T17:28:27.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac06d20c0 con 0x55bac06cd730 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac06d2280 con 0x55bac06cce10 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f666d740 10 client.0 _ll_register_callbacks cb 0x7f77dc00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42414/0 (socket says 172.21.15.44:42414) 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 -- 172.21.15.44:0/2517720872 learned_addr learned my addr 172.21.15.44:0/2517720872 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1607640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f0e06640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 -- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 msgr2=0x55bac06def30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06def30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 -- 172.21.15.44:0/2517720872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 msgr2=0x55bac06d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 -- 172.21.15.44:0/2517720872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bac062f620 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f0e06640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77f1e08640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06d1a10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f77e4030bd0 tx=0x7f77e4031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d30f7847ed86ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.571+0000 7f77d4ff9640 1 -- 172.21.15.44:0/2517720872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4030a00 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77d4ff9640 1 -- 172.21.15.44:0/2517720872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f77e4003e40 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77e4003cd0 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77d4ff9640 1 -- 172.21.15.44:0/2517720872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77e4030c00 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 10 client.16209 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bac05f6ea0 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77d4ff9640 1 -- 172.21.15.44:0/2517720872 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77e4003cd0 con 0x55bac06d7fd0 2024-09-13T17:28:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77d4ff9640 1 -- 172.21.15.44:0/2517720872 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77e4052070 con 0x55bac06d7fd0 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 20 client.16209 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac073f440 0x55bac0741830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 4 client.16209 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bac0388e30 con 0x55bac073f440 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.572+0000 7f77f1607640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac073f440 0x55bac0741830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77d4ff9640 10 client.16209 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f1607640 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac073f440 0x55bac0741830 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c52660f1a0c3028 server_cookie=8fac1d961ae6ae46 in_seq=0 out_seq=0 2024-09-13T17:28:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77d4ff9640 1 -- 172.21.15.44:0/2517720872 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bac0388e30 con 0x55bac073f440 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77d4ff9640 10 client.16209 handle_command_reply: tid=0 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 client.16209 shutdown 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 msgr2=0x55bac06d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06d1a10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f77e4030bd0 tx=0x7f77e4031f50 comp rx=0 tx=0).stop 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 shutdown_connections 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac073f440 0x55bac0741830 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac06d7fd0 0x55bac06d1a10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac06cd730 0x55bac06d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 --2- 172.21.15.44:0/2517720872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac06cce10 0x55bac06def30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77d4ff9640 0 client.16209 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 >> 172.21.15.44:0/2517720872 conn(0x55bac06c9380 msgr2=0x55bac06cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 shutdown_connections 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 1 -- 172.21.15.44:0/2517720872 wait complete. 2024-09-13T17:28:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.573+0000 7f77f666d740 20 client.16209 trim_cache size 0 max 16384 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: "04e91014-3a15-41af-966c-287ccb4a59e1": { 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "04e91014-3a15-41af-966c-287ccb4a59e1", 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:28:27.583 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:28:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f77d80a1280 msgr2=0x7f77d80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f77d80a1280 0x7f77d80a3740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e404b000 comp rx=0 tx=0).stop 2024-09-13T17:28:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 msgr2=0x7f77ec1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 0x7f77ec1988e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9f0 tx=0x7f77dc00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 shutdown_connections 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f77d80a1280 0x7f77d80a3740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77ec10c670 0x7f77ec19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77ec0ff470 0x7f77ec1988e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 --2- 172.21.15.44:0/1134283411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ec0feac0 0x7f77ec1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 >> 172.21.15.44:0/1134283411 conn(0x7f77ec075c60 msgr2=0x7f77ec0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 shutdown_connections 2024-09-13T17:28:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:27.574+0000 7f77f386b640 1 -- 172.21.15.44:0/1134283411 wait complete. 2024-09-13T17:28:27.586 INFO:tasks.cephfs.filesystem:scrub status for tag:04e91014-3a15-41af-966c-287ccb4a59e1 - {'path': '/', 'tag': '04e91014-3a15-41af-966c-287ccb4a59e1', 'options': 'recursive,force'} 2024-09-13T17:28:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:28 smithi100 ceph-mon[35055]: pgmap v55: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:28 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:28:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:28 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:28:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:28 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:28:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:28 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:28:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:28 smithi044 ceph-mon[31175]: pgmap v55: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:28.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:28 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:28:28.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:28 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:28:28.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:28 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:28:28.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:28 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:28:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:28 smithi070 ceph-mon[34502]: pgmap v55: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:28 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:28:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:28 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:28:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:28 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:28:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:28 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:28:29.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:29 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:28:29.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:29 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:28:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:29 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:28:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:30 smithi100 ceph-mon[35055]: pgmap v56: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:30.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:30 smithi044 ceph-mon[31175]: pgmap v56: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:30 smithi070 ceph-mon[34502]: pgmap v56: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:32.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:32 smithi100 ceph-mon[35055]: pgmap v57: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-13T17:28:32.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:32 smithi044 ceph-mon[31175]: pgmap v57: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-13T17:28:32.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:32 smithi070 ceph-mon[34502]: pgmap v57: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-13T17:28:34.499 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:34 smithi070 ceph-mon[34502]: pgmap v58: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-13T17:28:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:34 smithi100 ceph-mon[35055]: pgmap v58: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-13T17:28:34.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:34 smithi044 ceph-mon[31175]: pgmap v58: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-13T17:28:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:36 smithi100 ceph-mon[35055]: pgmap v59: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:36 smithi044 ceph-mon[31175]: pgmap v59: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:36 smithi070 ceph-mon[34502]: pgmap v59: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-09-13T17:28:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 -- 172.21.15.44:0/701859936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f40ff7d0 msgr2=0x7f04f4103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 --2- 172.21.15.44:0/701859936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f40ff7d0 0x7f04f4103810 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009930 tx=0x7f04e402a420 comp rx=0 tx=0).stop 2024-09-13T17:28:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 -- 172.21.15.44:0/701859936 shutdown_connections 2024-09-13T17:28:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 --2- 172.21.15.44:0/701859936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f4103f70 0x7f04f410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 --2- 172.21.15.44:0/701859936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f40ff7d0 0x7f04f4103810 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 --2- 172.21.15.44:0/701859936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04f40fee20 0x7f04f40ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 -- 172.21.15.44:0/701859936 >> 172.21.15.44:0/701859936 conn(0x7f04f40fa8f0 msgr2=0x7f04f40fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 -- 172.21.15.44:0/701859936 shutdown_connections 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 -- 172.21.15.44:0/701859936 wait complete. 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.997+0000 7f04fbb33640 1 Processor -- start 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 -- start start 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f40fee20 0x7f04f41961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 0x7f04f4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04f419aaf0 0x7f04f419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f419b4e0 con 0x7f04f4103f70 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f419b650 con 0x7f04f419aaf0 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fbb33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f419b7c0 con 0x7f04f40fee20 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 0x7f04f4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 0x7f04f4196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33878/0 (socket says 172.21.15.44:33878) 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 -- 172.21.15.44:0/594031573 learned_addr learned my addr 172.21.15.44:0/594031573 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04f419aaf0 0x7f04f419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f98cf640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f40fee20 0x7f04f41961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 -- 172.21.15.44:0/594031573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f40fee20 msgr2=0x7f04f41961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f40fee20 0x7f04f41961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 -- 172.21.15.44:0/594031573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04f419aaf0 msgr2=0x7f04f419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04f419aaf0 0x7f04f419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.998+0000 7f04f90ce640 1 -- 172.21.15.44:0/594031573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e4009560 con 0x7f04f4103f70 2024-09-13T17:28:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.999+0000 7f04f90ce640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 0x7f04f4196700 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005ab0 tx=0x7f04e4032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.999+0000 7f04eaffd640 1 -- 172.21.15.44:0/594031573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e40044d0 con 0x7f04f4103f70 2024-09-13T17:28:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.999+0000 7f04eaffd640 1 -- 172.21.15.44:0/594031573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04e4032e10 con 0x7f04f4103f70 2024-09-13T17:28:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.999+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f41a2da0 con 0x7f04f4103f70 2024-09-13T17:28:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.999+0000 7f04eaffd640 1 -- 172.21.15.44:0/594031573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4003850 con 0x7f04f4103f70 2024-09-13T17:28:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:37.999+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f41a31c0 con 0x7f04f4103f70 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.001+0000 7f04eaffd640 1 -- 172.21.15.44:0/594031573 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04e4004850 con 0x7f04f4103f70 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 Processor -- start 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 -- start start 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473186e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x564731878fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647317d9c30 con 0x5647318794f0 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647317d73c0 con 0x56473186ec50 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fe85a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647317d2cb0 con 0x56473186e330 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04eaffd640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04e00a1200 0x7f04e00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04f90ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04f90ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33884/0 (socket says 172.21.15.44:33884) 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473186e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04f90ce640 1 -- 172.21.15.44:0/394693675 learned_addr learned my addr 172.21.15.44:0/394693675 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04f98cf640 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x564731878fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.002+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04e00a1200 0x7f04e00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04eaffd640 1 -- 172.21.15.44:0/594031573 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f04e40edf30 con 0x7f04f4103f70 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f90ce640 1 -- 172.21.15.44:0/394693675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 msgr2=0x56473186e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f90ce640 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473186e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f90ce640 1 -- 172.21.15.44:0/394693675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 msgr2=0x564731878fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f90ce640 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x564731878fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f90ce640 1 -- 172.21.15.44:0/394693675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564731779910 con 0x5647318794f0 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473186e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f98cf640 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x564731878fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04f90ce640 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187b8e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f04e40eb270 tx=0x7f04e402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6648b6c0c92855 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04e00a1200 0x7f04e00a36c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009fd0 tx=0x7f04ec009600 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04dffff640 1 -- 172.21.15.44:0/394693675 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4004e60 con 0x5647318794f0 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04dffff640 1 -- 172.21.15.44:0/394693675 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04e40bad70 con 0x5647318794f0 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.003+0000 7f04dffff640 1 -- 172.21.15.44:0/394693675 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e40bafd0 con 0x5647318794f0 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 -- 172.21.15.44:0/394693675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 msgr2=0x56473187b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187b8e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f04e40eb270 tx=0x7f04e402a420 comp rx=0 tx=0).stop 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 -- 172.21.15.44:0/394693675 shutdown_connections 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187b8e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x564731878fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 --2- 172.21.15.44:0/394693675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473186e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 -- 172.21.15.44:0/394693675 >> 172.21.15.44:0/394693675 conn(0x56473186a8c0 msgr2=0x56473186aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 -- 172.21.15.44:0/394693675 shutdown_connections 2024-09-13T17:28:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 -- 172.21.15.44:0/394693675 wait complete. 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.004+0000 7f04fe85a740 1 -- start start 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 Processor -- start 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x564731878db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x5647318760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564731790e80 con 0x5647318794f0 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564731790b20 con 0x56473186ec50 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564731756260 con 0x56473186e330 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fe85a740 10 client.0 _ll_register_callbacks cb 0x56473178bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318765e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33890/0 (socket says 172.21.15.44:33890) 2024-09-13T17:28:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3558816648 learned_addr learned my addr 172.21.15.44:0/3558816648 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04f90ce640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x5647318760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.005+0000 7f04f98cf640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x564731878db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 msgr2=0x564731878db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x564731878db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3558816648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 msgr2=0x5647318760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x5647318760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3558816648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f41a31c0 con 0x5647318794f0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318765e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00abc0 tx=0x7f04ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2dd65665ef8b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04dd7fa640 1 -- 172.21.15.44:0/3558816648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec010da0 con 0x5647318794f0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04dd7fa640 1 -- 172.21.15.44:0/3558816648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f04ec004b20 con 0x5647318794f0 2024-09-13T17:28:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647317fb680 con 0x5647318794f0 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04dd7fa640 1 -- 172.21.15.44:0/3558816648 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ec004ce0 con 0x5647318794f0 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fe85a740 10 client.16221 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5647317d1330 con 0x5647318794f0 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.006+0000 7f04dd7fa640 1 -- 172.21.15.44:0/3558816648 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5647317fb680 con 0x5647318794f0 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04dd7fa640 1 -- 172.21.15.44:0/3558816648 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04ec026030 con 0x5647318794f0 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04fe85a740 10 client.16221 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04fe85a740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04fe85a740 20 client.16221 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564731888c70 0x56473188b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04fe85a740 4 client.16221 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647317c65d0 con 0x564731888c70 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04f98cf640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564731888c70 0x56473188b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04dd7fa640 10 client.16221 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.007+0000 7f04f98cf640 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564731888c70 0x56473188b090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266029974d14f54f server_cookie=d211f95febd19db8 in_seq=0 out_seq=0 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04dd7fa640 1 -- 172.21.15.44:0/3558816648 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5647317c65d0 con 0x564731888c70 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04dd7fa640 10 client.16221 handle_command_reply: tid=0 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 client.16221 shutdown 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 msgr2=0x5647318765e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318765e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00abc0 tx=0x7f04ec004a40 comp rx=0 tx=0).stop 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 shutdown_connections 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564731888c70 0x56473188b090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318765e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x5647318760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04dd7fa640 0 client.16221 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3558816648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x564731878db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 >> 172.21.15.44:0/3558816648 conn(0x56473186a8c0 msgr2=0x564731869b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 shutdown_connections 2024-09-13T17:28:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.008+0000 7f04fe85a740 1 -- 172.21.15.44:0/3558816648 wait complete. 2024-09-13T17:28:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.009+0000 7f04fe85a740 20 client.16221 trim_cache size 0 max 16384 2024-09-13T17:28:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 Processor -- start 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 -- start start 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186e330 0x564731865a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186ec50 0x564731865f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647317fb680 con 0x5647318794f0 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647318874b0 con 0x56473186e330 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fe85a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56473186a620 con 0x56473186ec50 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.021+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33906/0 (socket says 172.21.15.44:33906) 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3984447605 learned_addr learned my addr 172.21.15.44:0/3984447605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04f90ce640 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186ec50 0x564731865f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04f98cf640 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186e330 0x564731865a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3984447605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186ec50 msgr2=0x564731865f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186ec50 0x564731865f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3984447605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186e330 msgr2=0x564731865a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186e330 0x564731865a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 -- 172.21.15.44:0/3984447605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564731773300 con 0x5647318794f0 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318732e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f04ec028dc0 tx=0x7f04ec009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2089d01ded55e5b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04e8e05640 1 -- 172.21.15.44:0/3984447605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec010860 con 0x5647318794f0 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04e8e05640 1 -- 172.21.15.44:0/3984447605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04ec022070 con 0x5647318794f0 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.022+0000 7f04e8e05640 1 -- 172.21.15.44:0/3984447605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec028830 con 0x5647318794f0 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 -- 172.21.15.44:0/3984447605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 msgr2=0x5647318732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318732e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f04ec028dc0 tx=0x7f04ec009ad0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 -- 172.21.15.44:0/3984447605 shutdown_connections 2024-09-13T17:28:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186ec50 0x564731865f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x5647318732e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 --2- 172.21.15.44:0/3984447605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186e330 0x564731865a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 -- 172.21.15.44:0/3984447605 >> 172.21.15.44:0/3984447605 conn(0x56473186a8c0 msgr2=0x564731874950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 -- 172.21.15.44:0/3984447605 shutdown_connections 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 -- 172.21.15.44:0/3984447605 wait complete. 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 -- start start 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.023+0000 7f04fe85a740 1 Processor -- start 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473187eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x56473187f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647318df180 con 0x5647318794f0 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647318df2c0 con 0x56473186ec50 2024-09-13T17:28:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647318df400 con 0x56473186e330 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 10 client.0 _ll_register_callbacks cb 0x7f04e4004b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33922/0 (socket says 172.21.15.44:33922) 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 -- 172.21.15.44:0/1903664014 learned_addr learned my addr 172.21.15.44:0/1903664014 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04f90ce640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x56473187f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04f98cf640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473187eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 -- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 msgr2=0x56473187eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473187eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 -- 172.21.15.44:0/1903664014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 msgr2=0x56473187f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x56473187f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 -- 172.21.15.44:0/1903664014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647317d1330 con 0x5647318794f0 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fa0d0640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187f5b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0109e0 tx=0x7f04ec004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb746dcac3f6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04dd7fa640 1 -- 172.21.15.44:0/1903664014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec028620 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04dd7fa640 1 -- 172.21.15.44:0/1903664014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f04ec028780 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.024+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04e40bad70 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04dd7fa640 1 -- 172.21.15.44:0/1903664014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ec028940 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 10 client.16227 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5647317988e0 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04dd7fa640 1 -- 172.21.15.44:0/1903664014 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04e40bad70 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04dd7fa640 1 -- 172.21.15.44:0/1903664014 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04ec03d070 con 0x5647318794f0 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 10 client.16227 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 20 client.16227 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5647318e0840 0x5647318e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 4 client.16227 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:28:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647314c6300 con 0x5647318e0840 2024-09-13T17:28:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.025+0000 7f04f98cf640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5647318e0840 0x5647318e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.026+0000 7f04dd7fa640 10 client.16227 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.026+0000 7f04f98cf640 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5647318e0840 0x5647318e2c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19fb94c73fdb7479 server_cookie=86a7fcd0e2f01b6f in_seq=0 out_seq=0 2024-09-13T17:28:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.038+0000 7f04dd7fa640 1 -- 172.21.15.44:0/1903664014 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647314c6300 con 0x5647318e0840 2024-09-13T17:28:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.038+0000 7f04dd7fa640 10 client.16227 handle_command_reply: tid=0 2024-09-13T17:28:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.038+0000 7f04fe85a740 1 client.16227 shutdown 2024-09-13T17:28:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 msgr2=0x56473187f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187f5b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0109e0 tx=0x7f04ec004cb0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 shutdown_connections 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5647318e0840 0x5647318e2c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647318794f0 0x56473187f5b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56473186ec50 0x56473187f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04dd7fa640 0 client.16227 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 --2- 172.21.15.44:0/1903664014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56473186e330 0x56473187eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 >> 172.21.15.44:0/1903664014 conn(0x56473186a8c0 msgr2=0x564731874880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 shutdown_connections 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 1 -- 172.21.15.44:0/1903664014 wait complete. 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fe85a740 20 client.16227 trim_cache size 0 max 16384 2024-09-13T17:28:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:28:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:28:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:28:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:28:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04e00a1200 msgr2=0x7f04e00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fbb33640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04e00a1200 0x7f04e00a36c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009fd0 tx=0x7f04ec009600 comp rx=0 tx=0).stop 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 msgr2=0x7f04f4196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.039+0000 7f04fbb33640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 0x7f04f4196700 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005ab0 tx=0x7f04e4032c20 comp rx=0 tx=0).stop 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 shutdown_connections 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04e00a1200 0x7f04e00a36c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04f419aaf0 0x7f04f419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4103f70 0x7f04f4196700 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 --2- 172.21.15.44:0/594031573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f40fee20 0x7f04f41961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 >> 172.21.15.44:0/594031573 conn(0x7f04f40fa8f0 msgr2=0x7f04f40689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 shutdown_connections 2024-09-13T17:28:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:38.040+0000 7f04fbb33640 1 -- 172.21.15.44:0/594031573 wait complete. 2024-09-13T17:28:38.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:38 smithi100 ceph-mon[35055]: pgmap v60: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-13T17:28:38.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:38 smithi044 ceph-mon[31175]: pgmap v60: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-13T17:28:38.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:38 smithi070 ceph-mon[34502]: pgmap v60: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-13T17:28:40.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:40 smithi100 ceph-mon[35055]: pgmap v61: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-09-13T17:28:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:40 smithi044 ceph-mon[31175]: pgmap v61: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-09-13T17:28:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:40 smithi070 ceph-mon[34502]: pgmap v61: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-09-13T17:28:42.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:42 smithi100 ceph-mon[35055]: pgmap v62: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:42.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:42 smithi044 ceph-mon[31175]: pgmap v62: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:42 smithi070 ceph-mon[34502]: pgmap v62: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:44.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:44 smithi100 ceph-mon[35055]: pgmap v63: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:44.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:44 smithi044 ceph-mon[31175]: pgmap v63: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:44.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:44 smithi070 ceph-mon[34502]: pgmap v63: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:46.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:46 smithi100 ceph-mon[35055]: pgmap v64: 129 pgs: 129 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-13T17:28:46.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:46 smithi044 ceph-mon[31175]: pgmap v64: 129 pgs: 129 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-13T17:28:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:46 smithi070 ceph-mon[34502]: pgmap v64: 129 pgs: 129 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-13T17:28:48.525 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:48 smithi100 ceph-mon[35055]: pgmap v65: 129 pgs: 129 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:48.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:48 smithi044 ceph-mon[31175]: pgmap v65: 129 pgs: 129 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:48.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:48 smithi070 ceph-mon[34502]: pgmap v65: 129 pgs: 129 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-13T17:28:50.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:50 smithi100 ceph-mon[35055]: pgmap v66: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-13T17:28:50.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:50 smithi044 ceph-mon[31175]: pgmap v66: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-13T17:28:50.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:50 smithi070 ceph-mon[34502]: pgmap v66: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-13T17:28:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.139+0000 7f55e9a80640 1 -- 172.21.15.44:0/61900009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e40ff8b0 msgr2=0x7f55e40ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.139+0000 7f55e9a80640 1 --2- 172.21.15.44:0/61900009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e40ff8b0 0x7f55e40ffcf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009930 tx=0x7f55d802a420 comp rx=0 tx=0).stop 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 -- 172.21.15.44:0/61900009 shutdown_connections 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 --2- 172.21.15.44:0/61900009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e4100230 0x7f55e410b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 --2- 172.21.15.44:0/61900009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e40ff8b0 0x7f55e40ffcf0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 --2- 172.21.15.44:0/61900009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55e4103cd0 0x7f55e41040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 -- 172.21.15.44:0/61900009 >> 172.21.15.44:0/61900009 conn(0x7f55e4075340 msgr2=0x7f55e4075750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 -- 172.21.15.44:0/61900009 shutdown_connections 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 -- 172.21.15.44:0/61900009 wait complete. 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.140+0000 7f55e9a80640 1 Processor -- start 2024-09-13T17:28:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 -- start start 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e40ff8b0 0x7f55e4196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55e4100230 0x7f55e41966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 0x7f55e419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419b0a0 con 0x7f55e4103cd0 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419b210 con 0x7f55e40ff8b0 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e9a80640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419b3d0 con 0x7f55e4100230 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 0x7f55e419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 0x7f55e419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55980/0 (socket says 172.21.15.44:55980) 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 -- 172.21.15.44:0/905059721 learned_addr learned my addr 172.21.15.44:0/905059721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e40ff8b0 0x7f55e4196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55e4100230 0x7f55e41966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 -- 172.21.15.44:0/905059721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55e4100230 msgr2=0x7f55e41966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55e4100230 0x7f55e41966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 -- 172.21.15.44:0/905059721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e40ff8b0 msgr2=0x7f55e4196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e40ff8b0 0x7f55e4196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e3fff640 1 -- 172.21.15.44:0/905059721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8009560 con 0x7f55e4103cd0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.141+0000 7f55e37fe640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e40ff8b0 0x7f55e4196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.142+0000 7f55e3fff640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 0x7f55e419aaa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00eb10 tx=0x7f55dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.142+0000 7f55e0ff9640 1 -- 172.21.15.44:0/905059721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc004b20 con 0x7f55e4103cd0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.142+0000 7f55e0ff9640 1 -- 172.21.15.44:0/905059721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55dc004c80 con 0x7f55e4103cd0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.142+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e419b6b0 con 0x7f55e4103cd0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.142+0000 7f55e0ff9640 1 -- 172.21.15.44:0/905059721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc005a10 con 0x7f55e4103cd0 2024-09-13T17:28:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.142+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e41a2f30 con 0x7f55e4103cd0 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.144+0000 7f55e0ff9640 1 -- 172.21.15.44:0/905059721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f55dc005020 con 0x7f55e4103cd0 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 Processor -- start 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 -- start start 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e0ff9640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f55cc0a1280 0x7f55cc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19ef740 0x55c0d19efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1956c30 con 0x55c0d19f0060 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d19543c0 con 0x55c0d19fa900 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55ec87e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d194fcb0 con 0x55c0d19ef740 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e0ff9640 1 -- 172.21.15.44:0/905059721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f55dc014070 con 0x7f55e4103cd0 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f55cc0a1280 0x7f55cc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19ef740 0x55c0d19efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54950/0 (socket says 172.21.15.44:54950) 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19ef740 0x55c0d19efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:55748/0 (socket says 172.21.15.44:55748) 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55982/0 (socket says 172.21.15.44:55982) 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.145+0000 7f55e3fff640 1 -- 172.21.15.44:0/2596476966 learned_addr learned my addr 172.21.15.44:0/2596476966 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e3fff640 1 -- 172.21.15.44:0/2596476966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19ef740 msgr2=0x55c0d19efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19ef740 0x55c0d19efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e3fff640 1 -- 172.21.15.44:0/2596476966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 msgr2=0x55c0d19fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e3fff640 1 -- 172.21.15.44:0/2596476966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0d18f6910 con 0x55c0d19fa900 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19fccf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55dc096da0 tx=0x7f55dc09a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b82157ea49aa1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f55cc0a1280 0x7f55cc0a3740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006010 tx=0x7f55d8004210 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55c9ffb640 1 -- 172.21.15.44:0/2596476966 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc09aa30 con 0x55c0d19fa900 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55c9ffb640 1 -- 172.21.15.44:0/2596476966 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55dc09a330 con 0x55c0d19fa900 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.146+0000 7f55c9ffb640 1 -- 172.21.15.44:0/2596476966 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0cabf0 con 0x55c0d19fa900 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 -- 172.21.15.44:0/2596476966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 msgr2=0x55c0d19fccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19fccf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55dc096da0 tx=0x7f55dc09a230 comp rx=0 tx=0).stop 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 -- 172.21.15.44:0/2596476966 shutdown_connections 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19fccf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2596476966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19ef740 0x55c0d19efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 -- 172.21.15.44:0/2596476966 >> 172.21.15.44:0/2596476966 conn(0x55c0d19ebcd0 msgr2=0x55c0d19ec0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 -- 172.21.15.44:0/2596476966 shutdown_connections 2024-09-13T17:28:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 -- 172.21.15.44:0/2596476966 wait complete. 2024-09-13T17:28:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.147+0000 7f55ec87e740 1 -- start start 2024-09-13T17:28:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 Processor -- start 2024-09-13T17:28:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d190de80 con 0x55c0d19f0060 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d190db20 con 0x55c0d19ef740 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d18d3260 con 0x55c0d19fa900 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55ec87e740 10 client.0 _ll_register_callbacks cb 0x55c0d1908df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54958/0 (socket says 172.21.15.44:54958) 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55994/0 (socket says 172.21.15.44:55994) 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e37fe640 1 -- 172.21.15.44:0/2527964238 learned_addr learned my addr 172.21.15.44:0/2527964238 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 msgr2=0x55c0d19f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2527964238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 msgr2=0x55c0d19fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2527964238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e41a2f30 con 0x55c0d19f0060 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19f7630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f55d802a940 tx=0x7f55d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6302aa78fc3771c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.148+0000 7f55c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2527964238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8003ef0 con 0x55c0d19f0060 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2527964238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f55d803db60 con 0x55c0d19f0060 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0d1978680 con 0x55c0d19f0060 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2527964238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d803dd20 con 0x55c0d19f0060 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 10 client.16239 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c0d194e330 con 0x55c0d19f0060 2024-09-13T17:28:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2527964238 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c0d1978680 con 0x55c0d19f0060 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2527964238 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55d803b400 con 0x55c0d19f0060 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 10 client.16239 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 20 client.16239 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a0a270 0x55c0d1a0c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 4 client.16239 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.149+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0d19435d0 con 0x55c0d1a0a270 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.150+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a0a270 0x55c0d1a0c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.150+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a0a270 0x55c0d1a0c690 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c84c9023140ce51 server_cookie=b3b8120b24739db7 in_seq=0 out_seq=0 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.150+0000 7f55c2ffd640 10 client.16239 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2527964238 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c0d19435d0 con 0x55c0d1a0a270 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55c2ffd640 10 client.16239 handle_command_reply: tid=0 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 client.16239 shutdown 2024-09-13T17:28:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 msgr2=0x55c0d19f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19f7630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f55d802a940 tx=0x7f55d8000bc0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 shutdown_connections 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a0a270 0x55c0d1a0c690 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19f7630 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55c2ffd640 0 client.16239 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2527964238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 >> 172.21.15.44:0/2527964238 conn(0x55c0d19ebcd0 msgr2=0x55c0d19e6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 shutdown_connections 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 1 -- 172.21.15.44:0/2527964238 wait complete. 2024-09-13T17:28:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.151+0000 7f55ec87e740 20 client.16239 trim_cache size 0 max 16384 2024-09-13T17:28:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 Processor -- start 2024-09-13T17:28:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 -- start start 2024-09-13T17:28:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 0x55c0d1a015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19f0060 0x55c0d1a01b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1908d10 con 0x55c0d19ef740 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d19fd070 con 0x55c0d19fa900 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55ec87e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d19f5b60 con 0x55c0d19f0060 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 0x55c0d1a015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 0x55c0d1a015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56004/0 (socket says 172.21.15.44:56004) 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 -- 172.21.15.44:0/2710713656 learned_addr learned my addr 172.21.15.44:0/2710713656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19f0060 0x55c0d1a01b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 -- 172.21.15.44:0/2710713656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19f0060 msgr2=0x55c0d1a01b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19f0060 0x55c0d1a01b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 -- 172.21.15.44:0/2710713656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 msgr2=0x55c0d19f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 -- 172.21.15.44:0/2710713656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0d18f0300 con 0x55c0d19ef740 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19f0060 0x55c0d1a01b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.164+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 0x55c0d1a015d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55c0d1a0cfb0 tx=0x7f55d400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b559668b2b5930b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ca7fc640 1 -- 172.21.15.44:0/2710713656 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d401d4f0 con 0x55c0d19ef740 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ca7fc640 1 -- 172.21.15.44:0/2710713656 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55d4017040 con 0x55c0d19ef740 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ca7fc640 1 -- 172.21.15.44:0/2710713656 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4010c50 con 0x55c0d19ef740 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 -- 172.21.15.44:0/2710713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 msgr2=0x55c0d1a015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 0x55c0d1a015d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55c0d1a0cfb0 tx=0x7f55d400b390 comp rx=0 tx=0).stop 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 -- 172.21.15.44:0/2710713656 shutdown_connections 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19f0060 0x55c0d1a01b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19fa900 0x55c0d19f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2710713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19ef740 0x55c0d1a015d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 -- 172.21.15.44:0/2710713656 >> 172.21.15.44:0/2710713656 conn(0x55c0d19ebcd0 msgr2=0x55c0d19e2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 -- 172.21.15.44:0/2710713656 shutdown_connections 2024-09-13T17:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.165+0000 7f55ec87e740 1 -- 172.21.15.44:0/2710713656 wait complete. 2024-09-13T17:28:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 -- start start 2024-09-13T17:28:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 Processor -- start 2024-09-13T17:28:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1a60850 con 0x55c0d19f0060 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1a60990 con 0x55c0d19ef740 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1a60b50 con 0x55c0d19fa900 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55ec87e740 10 client.0 _ll_register_callbacks cb 0x7f55dc00cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56010/0 (socket says 172.21.15.44:56010) 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2332761308 learned_addr learned my addr 172.21.15.44:0/2332761308 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.166+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 msgr2=0x55c0d19fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19fecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2332761308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 msgr2=0x55c0d19fe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e2ffd640 1 -- 172.21.15.44:0/2332761308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d400bdf0 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55e2ffd640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fe770 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55c0d19f8c20 tx=0x7f55d80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e93d6f99de8921 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2332761308 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d803ba30 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2332761308 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f55d803b400 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55dc09a330 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2332761308 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d803b5c0 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55ec87e740 10 client.16245 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c0d19158e0 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.167+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2332761308 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55dc09a330 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2332761308 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55d804b500 con 0x55c0d19f0060 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55ec87e740 10 client.16245 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55ec87e740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55ec87e740 20 client.16245 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a61ef0 0x55c0d1a642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55ec87e740 4 client.16245 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c0d1643300 con 0x55c0d1a61ef0 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a61ef0 0x55c0d1a642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55c2ffd640 10 client.16245 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.168+0000 7f55e37fe640 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a61ef0 0x55c0d1a642e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceba7c1f2f6f2280 server_cookie=13ae541c656f3b94 in_seq=0 out_seq=0 2024-09-13T17:28:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55c2ffd640 1 -- 172.21.15.44:0/2332761308 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c0d1643300 con 0x55c0d1a61ef0 2024-09-13T17:28:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55c2ffd640 10 client.16245 handle_command_reply: tid=0 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 client.16245 shutdown 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 msgr2=0x55c0d19fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fe770 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55c0d19f8c20 tx=0x7f55d80035f0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 shutdown_connections 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0d1a61ef0 0x55c0d1a642e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d19fa900 0x55c0d19fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0d19f0060 0x55c0d19fe770 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55c2ffd640 0 client.16245 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 --2- 172.21.15.44:0/2332761308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0d19ef740 0x55c0d19fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 >> 172.21.15.44:0/2332761308 conn(0x55c0d19ebcd0 msgr2=0x55c0d19f6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 shutdown_connections 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 1 -- 172.21.15.44:0/2332761308 wait complete. 2024-09-13T17:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.181+0000 7f55ec87e740 20 client.16245 trim_cache size 0 max 16384 2024-09-13T17:28:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:28:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:28:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:28:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f55cc0a1280 msgr2=0x7f55cc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f55cc0a1280 0x7f55cc0a3740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006010 tx=0x7f55d8004210 comp rx=0 tx=0).stop 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 msgr2=0x7f55e419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 0x7f55e419aaa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00eb10 tx=0x7f55dc00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 shutdown_connections 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f55cc0a1280 0x7f55cc0a3740 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55e4100230 0x7f55e41966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55e4103cd0 0x7f55e419aaa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 --2- 172.21.15.44:0/905059721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55e40ff8b0 0x7f55e4196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 >> 172.21.15.44:0/905059721 conn(0x7f55e4075340 msgr2=0x7f55e40fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 shutdown_connections 2024-09-13T17:28:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:28:51.182+0000 7f55e9a80640 1 -- 172.21.15.44:0/905059721 wait complete. 2024-09-13T17:28:51.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:51 smithi100 ceph-mon[35055]: pgmap v67: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:51.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:51 smithi044 ceph-mon[31175]: pgmap v67: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:51.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:51 smithi070 ceph-mon[34502]: pgmap v67: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:52.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:52 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:28:52.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:52 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:28:52.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:52 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: pgmap v68: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:28:53.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:28:53.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:53 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: pgmap v68: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:28:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:53 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: pgmap v68: 129 pgs: 129 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:28:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:28:53.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:53 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:28:55.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:55 smithi070 ceph-mon[34502]: pgmap v69: 129 pgs: 129 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-13T17:28:55.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:55 smithi100 ceph-mon[35055]: pgmap v69: 129 pgs: 129 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-13T17:28:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:55 smithi044 ceph-mon[31175]: pgmap v69: 129 pgs: 129 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-13T17:28:57.588 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2366512883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b80ff830 msgr2=0x7f04b8103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2366512883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b80ff830 0x7f04b8103870 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009930 tx=0x7f04a802a420 comp rx=0 tx=0).stop 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2366512883 shutdown_connections 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2366512883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8103fd0 0x7f04b810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2366512883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b80ff830 0x7f04b8103870 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2366512883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b80fee80 0x7f04b80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.705+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2366512883 >> 172.21.15.44:0/2366512883 conn(0x7f04b80fa930 msgr2=0x7f04b80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2366512883 shutdown_connections 2024-09-13T17:28:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2366512883 wait complete. 2024-09-13T17:28:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 Processor -- start 2024-09-13T17:28:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 -- start start 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b80fee80 0x7f04b81962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b8103fd0 0x7f04b81967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 0x7f04b819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b819b5d0 con 0x7f04b819abe0 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b819b740 con 0x7f04b8103fd0 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.706+0000 7f04bf1d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b819b8b0 con 0x7f04b80fee80 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 0x7f04b819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 0x7f04b819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56024/0 (socket says 172.21.15.44:56024) 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 -- 172.21.15.44:0/2370461467 learned_addr learned my addr 172.21.15.44:0/2370461467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04affff640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b8103fd0 0x7f04b81967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bcf70640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b80fee80 0x7f04b81962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 -- 172.21.15.44:0/2370461467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b80fee80 msgr2=0x7f04b81962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b80fee80 0x7f04b81962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 -- 172.21.15.44:0/2370461467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b8103fd0 msgr2=0x7f04b81967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b8103fd0 0x7f04b81967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 -- 172.21.15.44:0/2370461467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8009560 con 0x7f04b819abe0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bcf70640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b80fee80 0x7f04b81962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bd771640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 0x7f04b819b090 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f04b000eaa0 tx=0x7f04b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04adffb640 1 -- 172.21.15.44:0/2370461467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b000cdb0 con 0x7f04b819abe0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04adffb640 1 -- 172.21.15.44:0/2370461467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b0005020 con 0x7f04b819abe0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b81a2e90 con 0x7f04b819abe0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.707+0000 7f04adffb640 1 -- 172.21.15.44:0/2370461467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00116d0 con 0x7f04b819abe0 2024-09-13T17:28:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.708+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b81a3310 con 0x7f04b819abe0 2024-09-13T17:28:57.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.710+0000 7f04adffb640 1 -- 172.21.15.44:0/2370461467 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04b0005190 con 0x7f04b819abe0 2024-09-13T17:28:57.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.710+0000 7f04c1fd2740 1 Processor -- start 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.710+0000 7f04c1fd2740 1 -- start start 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.710+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a063a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04adffb640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a00a1280 0x7f04a00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c09fc3f20 con 0x558c0a059730 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c09fc16b0 con 0x558c0a063fd0 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c09fbcfa0 con 0x558c0a058e10 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a063a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bd771640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a00a1280 0x7f04a00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a063a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56028/0 (socket says 172.21.15.44:56028) 2024-09-13T17:28:57.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04affff640 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 -- 172.21.15.44:0/2237292954 learned_addr learned my addr 172.21.15.44:0/2237292954 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04affff640 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04adffb640 1 -- 172.21.15.44:0/2370461467 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f04b00cd150 con 0x7f04b819abe0 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 -- 172.21.15.44:0/2237292954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 msgr2=0x558c0a0591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 -- 172.21.15.44:0/2237292954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 msgr2=0x558c0a0663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 -- 172.21.15.44:0/2237292954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c09f77840 con 0x558c0a059730 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04affff640 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bcf70640 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a063a90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009d50 tx=0x7f04b40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802f1d0d297e1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.711+0000 7f04bd771640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a00a1280 0x7f04a00a3740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04b0099be0 tx=0x7f04b0000f40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f049effd640 1 -- 172.21.15.44:0/2237292954 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4012650 con 0x558c0a059730 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f049effd640 1 -- 172.21.15.44:0/2237292954 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b4009e40 con 0x558c0a059730 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f049effd640 1 -- 172.21.15.44:0/2237292954 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400fbb0 con 0x558c0a059730 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2237292954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 msgr2=0x558c0a063a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a063a90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009d50 tx=0x7f04b40117b0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2237292954 shutdown_connections 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a063a90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2237292954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2237292954 >> 172.21.15.44:0/2237292954 conn(0x558c0a055380 msgr2=0x558c0a055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2237292954 shutdown_connections 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.712+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2237292954 wait complete. 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 -- start start 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 Processor -- start 2024-09-13T17:28:57.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a0639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a059730 0x558c0a060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a063fd0 0x558c0a061290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c09f5e2a0 con 0x558c0a058e10 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c09c95430 con 0x558c0a059730 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c09f40680 con 0x558c0a063fd0 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04c1fd2740 10 client.0 _ll_register_callbacks cb 0x558c09fb6610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a0639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a0639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56042/0 (socket says 172.21.15.44:56042) 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04affff640 1 -- 172.21.15.44:0/86305460 learned_addr learned my addr 172.21.15.44:0/86305460 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.713+0000 7f04bcf70640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a059730 0x558c0a060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a063fd0 0x558c0a061290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 -- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a063fd0 msgr2=0x558c0a061290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a063fd0 0x558c0a061290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 -- 172.21.15.44:0/86305460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a059730 msgr2=0x558c0a060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a059730 0x558c0a060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 -- 172.21.15.44:0/86305460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b4009980 con 0x558c0a058e10 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04affff640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a0639d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04b8069730 tx=0x7f04a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca781a51fc5a6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f0497fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f0497fff640 1 -- 172.21.15.44:0/86305460 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8031d70 con 0x558c0a058e10 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f0497fff640 1 -- 172.21.15.44:0/86305460 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f04a80035f0 con 0x558c0a058e10 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c09cb04e0 con 0x558c0a058e10 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f0497fff640 1 -- 172.21.15.44:0/86305460 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a80037b0 con 0x558c0a058e10 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04c1fd2740 10 client.16257 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558c09fbb620 con 0x558c0a058e10 2024-09-13T17:28:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.714+0000 7f0497fff640 1 -- 172.21.15.44:0/86305460 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558c09cb04e0 con 0x558c0a058e10 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f0497fff640 1 -- 172.21.15.44:0/86305460 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04a803d970 con 0x558c0a058e10 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04c1fd2740 10 client.16257 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04c1fd2740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04c1fd2740 20 client.16257 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a073ce0 0x558c0a0760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04c1fd2740 4 client.16257 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c09cc17e0 con 0x558c0a073ce0 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04bcf70640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a073ce0 0x558c0a0760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f04bcf70640 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a073ce0 0x558c0a0760d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d196ba6cb75c85 server_cookie=dee2c573f89ae4a5 in_seq=0 out_seq=0 2024-09-13T17:28:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.715+0000 7f0497fff640 10 client.16257 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:57.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f0497fff640 1 -- 172.21.15.44:0/86305460 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558c09cc17e0 con 0x558c0a073ce0 2024-09-13T17:28:57.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f0497fff640 10 client.16257 handle_command_reply: tid=0 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 client.16257 shutdown 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 msgr2=0x558c0a0639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a0639d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04b8069730 tx=0x7f04a8009260 comp rx=0 tx=0).stop 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 shutdown_connections 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a073ce0 0x558c0a0760d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a063fd0 0x558c0a061290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f0497fff640 0 client.16257 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a059730 0x558c0a060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/86305460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a0639d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 >> 172.21.15.44:0/86305460 conn(0x558c0a055380 msgr2=0x558c0a054510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 shutdown_connections 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 1 -- 172.21.15.44:0/86305460 wait complete. 2024-09-13T17:28:57.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.716+0000 7f04c1fd2740 20 client.16257 trim_cache size 0 max 16384 2024-09-13T17:28:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 Processor -- start 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 -- start start 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a061270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a059730 0x558c0a06f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0a058b90 con 0x558c0a058e10 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0a05f1f0 con 0x558c0a063fd0 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0a076400 con 0x558c0a059730 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a061270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a061270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56046/0 (socket says 172.21.15.44:56046) 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 -- 172.21.15.44:0/2794269323 learned_addr learned my addr 172.21.15.44:0/2794269323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04bcf70640 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a059730 0x558c0a06f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 -- 172.21.15.44:0/2794269323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a059730 msgr2=0x558c0a06f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a059730 0x558c0a06f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 -- 172.21.15.44:0/2794269323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 msgr2=0x558c0a06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.731+0000 7f04affff640 1 -- 172.21.15.44:0/2794269323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c09f5d790 con 0x558c0a058e10 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04affff640 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a061270 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f04a802a940 tx=0x7f04a802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92412efda20c26a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f049f7fe640 1 -- 172.21.15.44:0/2794269323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8031d70 con 0x558c0a058e10 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f049f7fe640 1 -- 172.21.15.44:0/2794269323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04a80044f0 con 0x558c0a058e10 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f049f7fe640 1 -- 172.21.15.44:0/2794269323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a803c9e0 con 0x558c0a058e10 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2794269323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 msgr2=0x558c0a061270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a061270 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f04a802a940 tx=0x7f04a802a420 comp rx=0 tx=0).stop 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2794269323 shutdown_connections 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a059730 0x558c0a06f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/2794269323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a058e10 0x558c0a061270 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2794269323 >> 172.21.15.44:0/2794269323 conn(0x558c0a055380 msgr2=0x558c0a07aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2794269323 shutdown_connections 2024-09-13T17:28:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.732+0000 7f04c1fd2740 1 -- 172.21.15.44:0/2794269323 wait complete. 2024-09-13T17:28:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 -- start start 2024-09-13T17:28:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 Processor -- start 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0a0c2720 con 0x558c0a059730 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0a0c2890 con 0x558c0a063fd0 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0a0c2a50 con 0x558c0a058e10 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04c1fd2740 10 client.0 _ll_register_callbacks cb 0x7f04a8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04bcf70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04bcf70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a0c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56058/0 (socket says 172.21.15.44:56058) 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.733+0000 7f04bcf70640 1 -- 172.21.15.44:0/1731640066 learned_addr learned my addr 172.21.15.44:0/1731640066 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04bcf70640 1 -- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 msgr2=0x558c0a0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04affff640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04bcf70640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04bcf70640 1 -- 172.21.15.44:0/1731640066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 msgr2=0x558c0a0c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04bcf70640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04bcf70640 1 -- 172.21.15.44:0/1731640066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c09fbb620 con 0x558c0a059730 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04affff640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04bcf70640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a0c6f40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009d50 tx=0x7f04b4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fea2d4e8ee0e83a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f0497fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f0497fff640 1 -- 172.21.15.44:0/1731640066 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40055e0 con 0x558c0a059730 2024-09-13T17:28:57.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f0497fff640 1 -- 172.21.15.44:0/1731640066 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f04b400f8a0 con 0x558c0a059730 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b4009e40 con 0x558c0a059730 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f0497fff640 1 -- 172.21.15.44:0/1731640066 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b400fa60 con 0x558c0a059730 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04c1fd2740 10 client.16263 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558c09f82ea0 con 0x558c0a059730 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f0497fff640 1 -- 172.21.15.44:0/1731640066 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b4009e40 con 0x558c0a059730 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f0497fff640 1 -- 172.21.15.44:0/1731640066 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04b4022040 con 0x558c0a059730 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04c1fd2740 10 client.16263 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04c1fd2740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.734+0000 7f04c1fd2740 20 client.16263 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.735+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a0cf990 0x558c0a0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.735+0000 7f04c1fd2740 4 client.16263 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.735+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c09d14e30 con 0x558c0a0cf990 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.735+0000 7f04affff640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a0cf990 0x558c0a0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.735+0000 7f04affff640 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a0cf990 0x558c0a0d1db0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782dee95be5f7ce1 server_cookie=b139d1afd1066fdb in_seq=0 out_seq=0 2024-09-13T17:28:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.735+0000 7f0497fff640 10 client.16263 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f0497fff640 1 -- 172.21.15.44:0/1731640066 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558c09d14e30 con 0x558c0a0cf990 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f0497fff640 10 client.16263 handle_command_reply: tid=0 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 client.16263 shutdown 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 msgr2=0x558c0a0c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a0c6f40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009d50 tx=0x7f04b4009cb0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 shutdown_connections 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558c0a0cf990 0x558c0a0d1db0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558c0a063fd0 0x558c0a0c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c0a059730 0x558c0a0c6f40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f0497fff640 0 client.16263 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 --2- 172.21.15.44:0/1731640066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0a058e10 0x558c0a0c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 >> 172.21.15.44:0/1731640066 conn(0x558c0a055380 msgr2=0x558c0a055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 shutdown_connections 2024-09-13T17:28:57.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 1 -- 172.21.15.44:0/1731640066 wait complete. 2024-09-13T17:28:57.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04c1fd2740 20 client.16263 trim_cache size 0 max 16384 2024-09-13T17:28:57.743 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:28:57.743 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:28:57.743 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:28:57.743 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a00a1280 msgr2=0x7f04a00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a00a1280 0x7f04a00a3740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04b0099be0 tx=0x7f04b0000f40 comp rx=0 tx=0).stop 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 msgr2=0x7f04b819b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.736+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 0x7f04b819b090 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f04b000eaa0 tx=0x7f04b000ef60 comp rx=0 tx=0).stop 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 shutdown_connections 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a00a1280 0x7f04a00a3740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04b819abe0 0x7f04b819b090 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b8103fd0 0x7f04b81967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 --2- 172.21.15.44:0/2370461467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b80fee80 0x7f04b81962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 >> 172.21.15.44:0/2370461467 conn(0x7f04b80fa930 msgr2=0x7f04b80fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 shutdown_connections 2024-09-13T17:28:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.737+0000 7f04bf1d4640 1 -- 172.21.15.44:0/2370461467 wait complete. 2024-09-13T17:28:57.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:28:57.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:28:57.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 -- 172.21.15.44:0/4015348204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078410a380 msgr2=0x7f078410c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 --2- 172.21.15.44:0/4015348204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078410a380 0x7f078410c7f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f077c009900 tx=0x7f077c02a480 comp rx=0 tx=0).stop 2024-09-13T17:28:57.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 -- 172.21.15.44:0/4015348204 shutdown_connections 2024-09-13T17:28:57.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 --2- 172.21.15.44:0/4015348204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078410a380 0x7f078410c7f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 --2- 172.21.15.44:0/4015348204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07841018d0 0x7f0784109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 --2- 172.21.15.44:0/4015348204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784100fb0 0x7f0784101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 -- 172.21.15.44:0/4015348204 >> 172.21.15.44:0/4015348204 conn(0x7f07840fa900 msgr2=0x7f07840fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:57.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 -- 172.21.15.44:0/4015348204 shutdown_connections 2024-09-13T17:28:57.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.852+0000 7f0788d45640 1 -- 172.21.15.44:0/4015348204 wait complete. 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 Processor -- start 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 -- start start 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784100fb0 0x7f07840fec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 0x7f07840ff140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078410a380 0x7f07840ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841a4590 con 0x7f07841018d0 2024-09-13T17:28:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841a4700 con 0x7f078410a380 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0788d45640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841a48c0 con 0x7f0784100fb0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0781d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 0x7f07840ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0781d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 0x7f07840ff140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56078/0 (socket says 172.21.15.44:56078) 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0781d9b640 1 -- 172.21.15.44:0/3783708014 learned_addr learned my addr 172.21.15.44:0/3783708014 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0782d9d640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078410a380 0x7f07840ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f078259c640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784100fb0 0x7f07840fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.853+0000 7f0781d9b640 1 -- 172.21.15.44:0/3783708014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784100fb0 msgr2=0x7f07840fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0781d9b640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784100fb0 0x7f07840fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0781d9b640 1 -- 172.21.15.44:0/3783708014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078410a380 msgr2=0x7f07840ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0781d9b640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078410a380 0x7f07840ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0781d9b640 1 -- 172.21.15.44:0/3783708014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009560 con 0x7f07841018d0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0782d9d640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078410a380 0x7f07840ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0781d9b640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 0x7f07840ff140 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f077800eab0 tx=0x7f077800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077800ce40 con 0x7f07841018d0 2024-09-13T17:28:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0778011900 con 0x7f07841018d0 2024-09-13T17:28:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07841a4ba0 con 0x7f07841018d0 2024-09-13T17:28:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780108c0 con 0x7f07841018d0 2024-09-13T17:28:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.854+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07841a50b0 con 0x7f07841018d0 2024-09-13T17:28:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.855+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0784068e30 con 0x7f07841018d0 2024-09-13T17:28:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.856+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0778011470 con 0x7f07841018d0 2024-09-13T17:28:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.857+0000 7f07737fe640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f076c0a1200 0x7f076c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.857+0000 7f078259c640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f076c0a1200 0x7f076c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.857+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f07780cd540 con 0x7f07841018d0 2024-09-13T17:28:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.858+0000 7f078259c640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f076c0a1200 0x7f076c0a36c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0774009bd0 tx=0x7f0774005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:57.860+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0778096610 con 0x7f07841018d0 2024-09-13T17:28:58.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.068+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0784101390 con 0x7f07841018d0 2024-09-13T17:28:58.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.069+0000 7f07737fe640 1 -- 172.21.15.44:0/3783708014 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f0778099c30 con 0x7f07841018d0 2024-09-13T17:28:58.070 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:28:58.070 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:28:58.071 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f076c0a1200 msgr2=0x7f076c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f076c0a1200 0x7f076c0a36c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0774009bd0 tx=0x7f0774005f90 comp rx=0 tx=0).stop 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 msgr2=0x7f07840ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 0x7f07840ff140 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f077800eab0 tx=0x7f077800ef70 comp rx=0 tx=0).stop 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 shutdown_connections 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f076c0a1200 0x7f076c0a36c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078410a380 0x7f07840ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07841018d0 0x7f07840ff140 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 --2- 172.21.15.44:0/3783708014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784100fb0 0x7f07840fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 >> 172.21.15.44:0/3783708014 conn(0x7f07840fa900 msgr2=0x7f07840fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 shutdown_connections 2024-09-13T17:28:58.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.071+0000 7f0788d45640 1 -- 172.21.15.44:0/3783708014 wait complete. 2024-09-13T17:28:58.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:28:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 -- 172.21.15.44:0/3701160077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e80ff800 msgr2=0x7f43e8103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 --2- 172.21.15.44:0/3701160077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e80ff800 0x7f43e8103840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009930 tx=0x7f43d802a420 comp rx=0 tx=0).stop 2024-09-13T17:28:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 -- 172.21.15.44:0/3701160077 shutdown_connections 2024-09-13T17:28:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 --2- 172.21.15.44:0/3701160077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e8103fa0 0x7f43e810baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 --2- 172.21.15.44:0/3701160077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e80ff800 0x7f43e8103840 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 --2- 172.21.15.44:0/3701160077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43e80fee50 0x7f43e80ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 -- 172.21.15.44:0/3701160077 >> 172.21.15.44:0/3701160077 conn(0x7f43e80fa8c0 msgr2=0x7f43e80fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 -- 172.21.15.44:0/3701160077 shutdown_connections 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.187+0000 7f43f01c3640 1 -- 172.21.15.44:0/3701160077 wait complete. 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 Processor -- start 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 -- start start 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e80fee50 0x7f43e81970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43e8103fa0 0x7f43e8197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 0x7f43e81a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81a27d0 con 0x7f43e8197b60 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81a2940 con 0x7f43e80fee50 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43f01c3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81a2b00 con 0x7f43e8103fa0 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 0x7f43e81a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 0x7f43e81a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56102/0 (socket says 172.21.15.44:56102) 2024-09-13T17:28:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 -- 172.21.15.44:0/451321662 learned_addr learned my addr 172.21.15.44:0/451321662 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ed75e640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43e8103fa0 0x7f43e8197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43edf5f640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e80fee50 0x7f43e81970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 -- 172.21.15.44:0/451321662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43e8103fa0 msgr2=0x7f43e8197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43e8103fa0 0x7f43e8197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 -- 172.21.15.44:0/451321662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e80fee50 msgr2=0x7f43e81970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e80fee50 0x7f43e81970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.188+0000 7f43ee760640 1 -- 172.21.15.44:0/451321662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8009560 con 0x7f43e8197b60 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43edf5f640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e80fee50 0x7f43e81970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43ee760640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 0x7f43e81a2160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f43e000eb10 tx=0x7f43e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43deffd640 1 -- 172.21.15.44:0/451321662 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0004b20 con 0x7f43e8197b60 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e81a2de0 con 0x7f43e8197b60 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43deffd640 1 -- 172.21.15.44:0/451321662 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43e0004c80 con 0x7f43e8197b60 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43deffd640 1 -- 172.21.15.44:0/451321662 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0005a10 con 0x7f43e8197b60 2024-09-13T17:28:58.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.189+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e81a3270 con 0x7f43e8197b60 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.191+0000 7f43deffd640 1 -- 172.21.15.44:0/451321662 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f43e0005020 con 0x7f43e8197b60 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.191+0000 7f43f2fbf740 1 Processor -- start 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 -- start start 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 0x5632b9bb9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bc4950 0x5632b9bc6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9b23cc0 con 0x5632b9bb9790 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9b21450 con 0x5632b9bba0b0 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9b1cd40 con 0x5632b9bc4950 2024-09-13T17:28:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 0x5632b9bb9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 0x5632b9bb9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56108/0 (socket says 172.21.15.44:56108) 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 -- 172.21.15.44:0/1075292132 learned_addr learned my addr 172.21.15.44:0/1075292132 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43deffd640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43d40a1230 0x7f43d40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43edf5f640 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 -- 172.21.15.44:0/1075292132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bc4950 msgr2=0x5632b9bc6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bc4950 0x5632b9bc6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 -- 172.21.15.44:0/1075292132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 msgr2=0x5632b9bc4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ee760640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43d40a1230 0x7f43d40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43ed75e640 1 -- 172.21.15.44:0/1075292132 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b9ae4620 con 0x5632b9bb9790 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.192+0000 7f43deffd640 1 -- 172.21.15.44:0/451321662 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f43e0014070 con 0x7f43e8197b60 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43ed75e640 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 0x5632b9bb9b70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f43d80057c0 tx=0x7f43d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c73c012f4bf9d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43d3fff640 1 -- 172.21.15.44:0/1075292132 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8004020 con 0x5632b9bb9790 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43ee760640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43d40a1230 0x7f43d40a36f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43e00cafe0 tx=0x7f43e0096040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43d3fff640 1 -- 172.21.15.44:0/1075292132 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43d8031960 con 0x5632b9bb9790 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43d3fff640 1 -- 172.21.15.44:0/1075292132 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80039f0 con 0x5632b9bb9790 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43f2fbf740 1 -- 172.21.15.44:0/1075292132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 msgr2=0x5632b9bb9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 0x5632b9bb9b70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f43d80057c0 tx=0x7f43d8005fd0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43f2fbf740 1 -- 172.21.15.44:0/1075292132 shutdown_connections 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.193+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bc4950 0x5632b9bc6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/1075292132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb9790 0x5632b9bb9b70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 -- 172.21.15.44:0/1075292132 >> 172.21.15.44:0/1075292132 conn(0x5632b9bb5d00 msgr2=0x5632b9bb60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 -- 172.21.15.44:0/1075292132 shutdown_connections 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 -- 172.21.15.44:0/1075292132 wait complete. 2024-09-13T17:28:58.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 -- start start 2024-09-13T17:28:58.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.194+0000 7f43f2fbf740 1 Processor -- start 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bc3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 0x5632b9bc1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9adb180 con 0x5632b9bc4950 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b97f5430 con 0x5632b9bba0b0 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9aa05e0 con 0x5632b9bb9790 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43f2fbf740 10 client.0 _ll_register_callbacks cb 0x5632b9b0caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 0x5632b9bc1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 0x5632b9bc1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56118/0 (socket says 172.21.15.44:56118) 2024-09-13T17:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 -- 172.21.15.44:0/2786555410 learned_addr learned my addr 172.21.15.44:0/2786555410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43edf5f640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bc3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 -- 172.21.15.44:0/2786555410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 msgr2=0x5632b9bc3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bc3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 -- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 msgr2=0x5632b9bc1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 -- 172.21.15.44:0/2786555410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e810c330 con 0x5632b9bc4950 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 0x5632b9bc1740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f43d8031f90 tx=0x7f43d8003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dbf4e8c9e97675 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.195+0000 7f43d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.197+0000 7f43d17fa640 1 -- 172.21.15.44:0/2786555410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d803c640 con 0x5632b9bc4950 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.197+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632b9b46b80 con 0x5632b9bc4950 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.197+0000 7f43d17fa640 1 -- 172.21.15.44:0/2786555410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f43d803c7a0 con 0x5632b9bc4950 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.197+0000 7f43d17fa640 1 -- 172.21.15.44:0/2786555410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d803c960 con 0x5632b9bc4950 2024-09-13T17:28:58.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.197+0000 7f43f2fbf740 10 client.16281 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.197+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5632b9b1b3c0 con 0x5632b9bc4950 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43d17fa640 1 -- 172.21.15.44:0/2786555410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5632b9b46b80 con 0x5632b9bc4950 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43d17fa640 1 -- 172.21.15.44:0/2786555410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43d8044070 con 0x5632b9bc4950 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43f2fbf740 10 client.16281 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43f2fbf740 10 client.16281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43f2fbf740 20 client.16281 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bd3e10 0x5632b9bd6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43f2fbf740 4 client.16281 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632b9b0f1f0 con 0x5632b9bd3e10 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.198+0000 7f43edf5f640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bd3e10 0x5632b9bd6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43edf5f640 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bd3e10 0x5632b9bd6230 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36d49bdea538e701 server_cookie=fcd35ff7bf397b29 in_seq=0 out_seq=0 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43d17fa640 10 client.16281 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43d17fa640 1 -- 172.21.15.44:0/2786555410 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5632b9b0f1f0 con 0x5632b9bd3e10 2024-09-13T17:28:58.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43d17fa640 10 client.16281 handle_command_reply: tid=0 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 client.16281 shutdown 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 msgr2=0x5632b9bc1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 0x5632b9bc1740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f43d8031f90 tx=0x7f43d8003690 comp rx=0 tx=0).stop 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 shutdown_connections 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bd3e10 0x5632b9bd6230 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bc4950 0x5632b9bc1740 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bc1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2786555410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bc3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43d17fa640 0 client.16281 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.199+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 >> 172.21.15.44:0/2786555410 conn(0x5632b9bb5d00 msgr2=0x5632b9bbba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.200+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 shutdown_connections 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.200+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2786555410 wait complete. 2024-09-13T17:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.200+0000 7f43f2fbf740 20 client.16281 trim_cache size 0 max 16384 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 Processor -- start 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 -- start start 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9bcba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bbfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bbfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9bb68b0 con 0x5632b9bb5d00 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9bc1000 con 0x5632b9bba0b0 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9bc44c0 con 0x5632b9bb9790 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9bcba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9bcba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56120/0 (socket says 172.21.15.44:56120) 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 -- 172.21.15.44:0/2358206902 learned_addr learned my addr 172.21.15.44:0/2358206902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43edf5f640 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bbfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 -- 172.21.15.44:0/2358206902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 msgr2=0x5632b9bbfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bbfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 -- 172.21.15.44:0/2358206902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 msgr2=0x5632b9bbfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bbfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 -- 172.21.15.44:0/2358206902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b9abd770 con 0x5632b9bb5d00 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43ed75e640 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9bcba60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f43d80092a0 tx=0x7f43d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e41a3f21607c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43dcff9640 1 -- 172.21.15.44:0/2358206902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8031b80 con 0x5632b9bb5d00 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43dcff9640 1 -- 172.21.15.44:0/2358206902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43d80468d0 con 0x5632b9bb5d00 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.211+0000 7f43dcff9640 1 -- 172.21.15.44:0/2358206902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d803ca60 con 0x5632b9bb5d00 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2358206902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 msgr2=0x5632b9bcba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9bcba60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f43d80092a0 tx=0x7f43d8005fd0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2358206902 shutdown_connections 2024-09-13T17:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bb9790 0x5632b9bbfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bba0b0 0x5632b9bbfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/2358206902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9bcba60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2358206902 >> 172.21.15.44:0/2358206902 conn(0x5632b9bc4040 msgr2=0x5632b9bc11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2358206902 shutdown_connections 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 -- 172.21.15.44:0/2358206902 wait complete. 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 -- start start 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.212+0000 7f43f2fbf740 1 Processor -- start 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9c22470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bb9790 0x5632b9c229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bba0b0 0x5632b9c26610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9c26b80 con 0x5632b9bb5d00 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9c26cf0 con 0x5632b9bb9790 2024-09-13T17:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b9c26eb0 con 0x5632b9bba0b0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43f2fbf740 10 client.0 _ll_register_callbacks cb 0x7f43e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9c22470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9c22470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56126/0 (socket says 172.21.15.44:56126) 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 -- 172.21.15.44:0/3648626037 learned_addr learned my addr 172.21.15.44:0/3648626037 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43edf5f640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bb9790 0x5632b9c229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bba0b0 0x5632b9c26610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 -- 172.21.15.44:0/3648626037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bba0b0 msgr2=0x5632b9c26610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bba0b0 0x5632b9c26610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 -- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bb9790 msgr2=0x5632b9c229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bb9790 0x5632b9c229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 -- 172.21.15.44:0/3648626037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632b9b1b3c0 con 0x5632b9bb5d00 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43ed75e640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9c22470 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009930 tx=0x7f43d803c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b659a622960cecb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.213+0000 7f43d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43d17fa640 1 -- 172.21.15.44:0/3648626037 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8031c40 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43d17fa640 1 -- 172.21.15.44:0/3648626037 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f43d8000e40 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43d8031960 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43d17fa640 1 -- 172.21.15.44:0/3648626037 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d803c7e0 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5632b9ae2e80 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43d17fa640 1 -- 172.21.15.44:0/3648626037 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43d8031960 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43d17fa640 1 -- 172.21.15.44:0/3648626037 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43d8044040 con 0x5632b9bb5d00 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 20 client.16287 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bc4950 0x5632b9c31450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 4 client.16287 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5632b98104e0 con 0x5632b9bc4950 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.214+0000 7f43edf5f640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bc4950 0x5632b9c31450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43d17fa640 10 client.16287 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43edf5f640 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bc4950 0x5632b9c31450 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1617f8c7a73c587b server_cookie=80ea8378c512fa1c in_seq=0 out_seq=0 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43d17fa640 1 -- 172.21.15.44:0/3648626037 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5632b98104e0 con 0x5632b9bc4950 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43d17fa640 10 client.16287 handle_command_reply: tid=0 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 client.16287 shutdown 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 msgr2=0x5632b9c22470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9c22470 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009930 tx=0x7f43d803c640 comp rx=0 tx=0).stop 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 shutdown_connections 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5632b9bc4950 0x5632b9c31450 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632b9bba0b0 0x5632b9c26610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5632b9bb9790 0x5632b9c229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43d17fa640 0 client.16287 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 --2- 172.21.15.44:0/3648626037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632b9bb5d00 0x5632b9c22470 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 >> 172.21.15.44:0/3648626037 conn(0x5632b9bc4040 msgr2=0x5632b9bbfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 shutdown_connections 2024-09-13T17:28:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 1 -- 172.21.15.44:0/3648626037 wait complete. 2024-09-13T17:28:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.215+0000 7f43f2fbf740 20 client.16287 trim_cache size 0 max 16384 2024-09-13T17:28:58.223 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43d40a1230 msgr2=0x7f43d40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43d40a1230 0x7f43d40a36f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43e00cafe0 tx=0x7f43e0096040 comp rx=0 tx=0).stop 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 msgr2=0x7f43e81a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 0x7f43e81a2160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f43e000eb10 tx=0x7f43e000c2d0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 shutdown_connections 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43d40a1230 0x7f43d40a36f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43e8103fa0 0x7f43e8197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e8197b60 0x7f43e81a2160 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 --2- 172.21.15.44:0/451321662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43e80fee50 0x7f43e81970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 >> 172.21.15.44:0/451321662 conn(0x7f43e80fa8c0 msgr2=0x7f43e80fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 shutdown_connections 2024-09-13T17:28:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.216+0000 7f43f01c3640 1 -- 172.21.15.44:0/451321662 wait complete. 2024-09-13T17:28:58.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:28:58.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 -- 172.21.15.44:0/4078435440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf600fedc0 msgr2=0x7fcf600ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/4078435440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf600fedc0 0x7fcf600ff1a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009930 tx=0x7fcf5402a420 comp rx=0 tx=0).stop 2024-09-13T17:28:58.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 -- 172.21.15.44:0/4078435440 shutdown_connections 2024-09-13T17:28:58.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/4078435440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600ff770 0x7fcf601037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/4078435440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf60103f10 0x7fcf6010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/4078435440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf600fedc0 0x7fcf600ff1a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 -- 172.21.15.44:0/4078435440 >> 172.21.15.44:0/4078435440 conn(0x7fcf600fa870 msgr2=0x7fcf600fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 -- 172.21.15.44:0/4078435440 shutdown_connections 2024-09-13T17:28:58.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 -- 172.21.15.44:0/4078435440 wait complete. 2024-09-13T17:28:58.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 Processor -- start 2024-09-13T17:28:58.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.331+0000 7fcf65d9d640 1 -- start start 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600fedc0 0x7fcf601961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 0x7fcf60196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6019aaf0 0x7fcf6019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6019b4e0 con 0x7fcf60103f10 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6019b650 con 0x7fcf6019aaf0 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6019b7c0 con 0x7fcf600fedc0 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 0x7fcf60196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 0x7fcf60196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56140/0 (socket says 172.21.15.44:56140) 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 -- 172.21.15.44:0/3014218477 learned_addr learned my addr 172.21.15.44:0/3014218477 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf6559c640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6019aaf0 0x7fcf6019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600fedc0 0x7fcf601961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 -- 172.21.15.44:0/3014218477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600fedc0 msgr2=0x7fcf601961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600fedc0 0x7fcf601961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 -- 172.21.15.44:0/3014218477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6019aaf0 msgr2=0x7fcf6019afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6019aaf0 0x7fcf6019afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 -- 172.21.15.44:0/3014218477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54009560 con 0x7fcf60103f10 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600fedc0 0x7fcf601961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 0x7fcf60196700 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00d740 tx=0x7fcf4c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5dffb640 1 -- 172.21.15.44:0/3014218477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c009c30 con 0x7fcf60103f10 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5dffb640 1 -- 172.21.15.44:0/3014218477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf4c00de50 con 0x7fcf60103f10 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf5dffb640 1 -- 172.21.15.44:0/3014218477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004d30 con 0x7fcf60103f10 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.332+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf601a2da0 con 0x7fcf60103f10 2024-09-13T17:28:58.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.333+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf601a31a0 con 0x7fcf60103f10 2024-09-13T17:28:58.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf5dffb640 1 -- 172.21.15.44:0/3014218477 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcf4c004e90 con 0x7fcf60103f10 2024-09-13T17:28:58.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 Processor -- start 2024-09-13T17:28:58.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 -- start start 2024-09-13T17:28:58.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033eb970 0x564e033ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033ec290 0x564e033f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 0x564e033f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03355cc0 con 0x564e033f6b30 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03353450 con 0x564e033ec290 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0334ed40 con 0x564e033eb970 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.335+0000 7fcf5dffb640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcf500a1280 0x7fcf500a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 0x564e033f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf6559c640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcf500a1280 0x7fcf500a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033ec290 0x564e033f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 0x564e033f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56154/0 (socket says 172.21.15.44:56154) 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 -- 172.21.15.44:0/2427976452 learned_addr learned my addr 172.21.15.44:0/2427976452 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5dffb640 1 -- 172.21.15.44:0/3014218477 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcf4c0cc9b0 con 0x7fcf60103f10 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf6559c640 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033eb970 0x564e033ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 -- 172.21.15.44:0/2427976452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033eb970 msgr2=0x564e033ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033eb970 0x564e033ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 -- 172.21.15.44:0/2427976452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033ec290 msgr2=0x564e033f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033ec290 0x564e033f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 -- 172.21.15.44:0/2427976452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e03316620 con 0x564e033f6b30 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf6559c640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcf500a1280 0x7fcf500a3740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf58007310 tx=0x7fcf58013ac0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 0x564e033f8f20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0961b0 tx=0x7fcf4c0c3c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564cf32252ac1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf4affd640 1 -- 172.21.15.44:0/2427976452 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0990a0 con 0x564e033f6b30 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf4affd640 1 -- 172.21.15.44:0/2427976452 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf4c068a70 con 0x564e033f6b30 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.336+0000 7fcf4affd640 1 -- 172.21.15.44:0/2427976452 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0b90b0 con 0x564e033f6b30 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2427976452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 msgr2=0x564e033f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 0x564e033f8f20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0961b0 tx=0x7fcf4c0c3c80 comp rx=0 tx=0).stop 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2427976452 shutdown_connections 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033f6b30 0x564e033f8f20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033ec290 0x564e033f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2427976452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033eb970 0x564e033ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2427976452 >> 172.21.15.44:0/2427976452 conn(0x564e033e7ee0 msgr2=0x564e033e82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2427976452 shutdown_connections 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2427976452 wait complete. 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.337+0000 7fcf69dfb740 1 -- start start 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 Processor -- start 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033f3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0330d180 con 0x564e033eb970 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03027430 con 0x564e033f6b30 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e032d25e0 con 0x564e033ec290 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf69dfb740 10 client.0 _ll_register_callbacks cb 0x564e0333eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56156/0 (socket says 172.21.15.44:56156) 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 -- 172.21.15.44:0/224346810 learned_addr learned my addr 172.21.15.44:0/224346810 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf6559c640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 -- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 msgr2=0x564e033f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 -- 172.21.15.44:0/224346810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 msgr2=0x564e033f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf64d9b640 1 -- 172.21.15.44:0/224346810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf601a31a0 con 0x564e033eb970 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.338+0000 7fcf6559c640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f6160 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcf5402aa10 tx=0x7fcf540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc27c103bb45a559 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf3ffff640 1 -- 172.21.15.44:0/224346810 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54003c90 con 0x564e033eb970 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf3ffff640 1 -- 172.21.15.44:0/224346810 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcf54003df0 con 0x564e033eb970 2024-09-13T17:28:58.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e03378b80 con 0x564e033eb970 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf3ffff640 1 -- 172.21.15.44:0/224346810 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf54046470 con 0x564e033eb970 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 10 client.16299 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e0334d3c0 con 0x564e033eb970 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf3ffff640 1 -- 172.21.15.44:0/224346810 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564e03378b80 con 0x564e033eb970 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf3ffff640 1 -- 172.21.15.44:0/224346810 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf5403d610 con 0x564e033eb970 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 10 client.16299 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 20 client.16299 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e034064e0 0x564e034088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 4 client.16299 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.339+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e033411f0 con 0x564e034064e0 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.340+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e034064e0 0x564e034088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.340+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e034064e0 0x564e034088d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ed6689c2182b406 server_cookie=5d7d8164bf7f7cb1 in_seq=0 out_seq=0 2024-09-13T17:28:58.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.340+0000 7fcf3ffff640 10 client.16299 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:58.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf3ffff640 1 -- 172.21.15.44:0/224346810 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564e033411f0 con 0x564e034064e0 2024-09-13T17:28:58.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf3ffff640 10 client.16299 handle_command_reply: tid=0 2024-09-13T17:28:58.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 client.16299 shutdown 2024-09-13T17:28:58.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 msgr2=0x564e033f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f6160 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcf5402aa10 tx=0x7fcf540038a0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 shutdown_connections 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e034064e0 0x564e034088d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/224346810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f6160 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf3ffff640 0 client.16299 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 >> 172.21.15.44:0/224346810 conn(0x564e033e7ee0 msgr2=0x564e033e7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 shutdown_connections 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 1 -- 172.21.15.44:0/224346810 wait complete. 2024-09-13T17:28:58.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.341+0000 7fcf69dfb740 20 client.16299 trim_cache size 0 max 16384 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 Processor -- start 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 -- start start 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e033e37d0 con 0x564e033eb970 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03406300 con 0x564e033f6b30 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e033e3d00 con 0x564e033ec290 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56162/0 (socket says 172.21.15.44:56162) 2024-09-13T17:28:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.352+0000 7fcf64d9b640 1 -- 172.21.15.44:0/4075650342 learned_addr learned my addr 172.21.15.44:0/4075650342 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf6559c640 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf64d9b640 1 -- 172.21.15.44:0/4075650342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 msgr2=0x564e033fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033fdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf64d9b640 1 -- 172.21.15.44:0/4075650342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 msgr2=0x564e033ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033ee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf64d9b640 1 -- 172.21.15.44:0/4075650342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e032ef770 con 0x564e033eb970 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f3960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcf54005ab0 tx=0x7fcf54005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d263e975f51ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf4b7fe640 1 -- 172.21.15.44:0/4075650342 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5403d650 con 0x564e033eb970 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf4b7fe640 1 -- 172.21.15.44:0/4075650342 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf54041070 con 0x564e033eb970 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.353+0000 7fcf4b7fe640 1 -- 172.21.15.44:0/4075650342 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf540466d0 con 0x564e033eb970 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 -- 172.21.15.44:0/4075650342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 msgr2=0x564e033f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f3960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcf54005ab0 tx=0x7fcf54005a20 comp rx=0 tx=0).stop 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 -- 172.21.15.44:0/4075650342 shutdown_connections 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e033fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e033ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/4075650342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e033f3960 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 -- 172.21.15.44:0/4075650342 >> 172.21.15.44:0/4075650342 conn(0x564e033e7ee0 msgr2=0x564e033f1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 -- 172.21.15.44:0/4075650342 shutdown_connections 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 -- 172.21.15.44:0/4075650342 wait complete. 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 -- start start 2024-09-13T17:28:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 Processor -- start 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e0345a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e0345a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.354+0000 7fcf69dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e03454920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03454ff0 con 0x564e033eb970 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03455160 con 0x564e033f6b30 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf69dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03455320 con 0x564e033ec290 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf69dfb740 10 client.0 _ll_register_callbacks cb 0x7fcf4c022f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e0345a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e0345a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56178/0 (socket says 172.21.15.44:56178) 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 -- 172.21.15.44:0/2087559904 learned_addr learned my addr 172.21.15.44:0/2087559904 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf6559c640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e03454920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e0345a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 -- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 msgr2=0x564e0345a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e0345a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 -- 172.21.15.44:0/2087559904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 msgr2=0x564e03454920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e03454920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 -- 172.21.15.44:0/2087559904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e0334d3c0 con 0x564e033eb970 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e0345a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf64d9b640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e0345a340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcf540042d0 tx=0x7fcf54003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4891b92d0be2df0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:58.360 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:58 smithi044 ceph-mon[31175]: pgmap v70: 129 pgs: 129 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:28:58.360 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:28:58 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3783708014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:28:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf3ffff640 1 -- 172.21.15.44:0/2087559904 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5403dc60 con 0x564e033eb970 2024-09-13T17:28:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf3ffff640 1 -- 172.21.15.44:0/2087559904 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcf54000e40 con 0x564e033eb970 2024-09-13T17:28:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf4c068a70 con 0x564e033eb970 2024-09-13T17:28:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.355+0000 7fcf3ffff640 1 -- 172.21.15.44:0/2087559904 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf54046470 con 0x564e033eb970 2024-09-13T17:28:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 10 client.16305 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e03314e80 con 0x564e033eb970 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf3ffff640 1 -- 172.21.15.44:0/2087559904 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf54064020 con 0x564e033eb970 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf3ffff640 1 -- 172.21.15.44:0/2087559904 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf5405c4c0 con 0x564e033eb970 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 10 client.16305 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 20 client.16305 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e03461d80 0x564e03464170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 4 client.16305 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e030424e0 con 0x564e03461d80 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.356+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e03461d80 0x564e03464170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf3ffff640 10 client.16305 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf5ffff640 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e03461d80 0x564e03464170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b19037b3a033b7b6 server_cookie=b9992d0588539fa8 in_seq=0 out_seq=0 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf3ffff640 1 -- 172.21.15.44:0/2087559904 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e030424e0 con 0x564e03461d80 2024-09-13T17:28:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf3ffff640 10 client.16305 handle_command_reply: tid=0 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 client.16305 shutdown 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 msgr2=0x564e0345a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e0345a340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcf540042d0 tx=0x7fcf54003ad0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 shutdown_connections 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564e03461d80 0x564e03464170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e033ec290 0x564e0345a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e033f6b30 0x564e03454920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 --2- 172.21.15.44:0/2087559904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e033eb970 0x564e0345a340 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf3ffff640 0 client.16305 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 >> 172.21.15.44:0/2087559904 conn(0x564e033e7ee0 msgr2=0x564e033ee450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 shutdown_connections 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 1 -- 172.21.15.44:0/2087559904 wait complete. 2024-09-13T17:28:58.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.357+0000 7fcf69dfb740 20 client.16305 trim_cache size 0 max 16384 2024-09-13T17:28:58.365 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:28:58.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcf500a1280 msgr2=0x7fcf500a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcf500a1280 0x7fcf500a3740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf58007310 tx=0x7fcf58013ac0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 msgr2=0x7fcf60196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 0x7fcf60196700 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00d740 tx=0x7fcf4c00dc00 comp rx=0 tx=0).stop 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 shutdown_connections 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcf500a1280 0x7fcf500a3740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf6019aaf0 0x7fcf6019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf60103f10 0x7fcf60196700 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 --2- 172.21.15.44:0/3014218477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf600fedc0 0x7fcf601961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 >> 172.21.15.44:0/3014218477 conn(0x7fcf600fa870 msgr2=0x7fcf60068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 shutdown_connections 2024-09-13T17:28:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.358+0000 7fcf65d9d640 1 -- 172.21.15.44:0/3014218477 wait complete. 2024-09-13T17:28:58.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:28:58.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:58 smithi070 ceph-mon[34502]: pgmap v70: 129 pgs: 129 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:28:58.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:28:58 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3783708014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 -- 172.21.15.44:0/3757214972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 msgr2=0x7f9950109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 --2- 172.21.15.44:0/3757214972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950109d10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9940009900 tx=0x7f994002a480 comp rx=0 tx=0).stop 2024-09-13T17:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 -- 172.21.15.44:0/3757214972 shutdown_connections 2024-09-13T17:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 --2- 172.21.15.44:0/3757214972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995010a2e0 0x7f995010c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 --2- 172.21.15.44:0/3757214972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950109d10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 --2- 172.21.15.44:0/3757214972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9950100f10 0x7f99501012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 -- 172.21.15.44:0/3757214972 >> 172.21.15.44:0/3757214972 conn(0x7f99500fa8c0 msgr2=0x7f99500fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 -- 172.21.15.44:0/3757214972 shutdown_connections 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.474+0000 7f9957eb1640 1 -- 172.21.15.44:0/3757214972 wait complete. 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 Processor -- start 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 -- start start 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9950100f10 0x7f99501961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f995010a2e0 0x7f995019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019b0d0 con 0x7f9950101830 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019b240 con 0x7f995010a2e0 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9957eb1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019b400 con 0x7f9950100f10 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56200/0 (socket says 172.21.15.44:56200) 2024-09-13T17:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 -- 172.21.15.44:0/445059078 learned_addr learned my addr 172.21.15.44:0/445059078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995644e640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f995010a2e0 0x7f995019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9955c4d640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9950100f10 0x7f99501961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 -- 172.21.15.44:0/445059078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9950100f10 msgr2=0x7f99501961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9950100f10 0x7f99501961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 -- 172.21.15.44:0/445059078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f995010a2e0 msgr2=0x7f995019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f995010a2e0 0x7f995019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 -- 172.21.15.44:0/445059078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940009560 con 0x7f9950101830 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f9955c4d640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9950100f10 0x7f99501961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.475+0000 7f995544c640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950196730 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f99400057c0 tx=0x7f9940032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.476+0000 7f9946ffd640 1 -- 172.21.15.44:0/445059078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940005000 con 0x7f9950101830 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.476+0000 7f9946ffd640 1 -- 172.21.15.44:0/445059078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99400046c0 con 0x7f9950101830 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.476+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995019b680 con 0x7f9950101830 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.476+0000 7f9946ffd640 1 -- 172.21.15.44:0/445059078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940003850 con 0x7f9950101830 2024-09-13T17:28:58.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.476+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99501a2f60 con 0x7f9950101830 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f9946ffd640 1 -- 172.21.15.44:0/445059078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9940005160 con 0x7f9950101830 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 Processor -- start 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 -- start start 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de3ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de4dd60 0x55d86de50150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86dda8cc0 con 0x55d86de3ef90 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86dda6450 con 0x55d86de4dd60 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995abd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86dda1d40 con 0x55d86de3e670 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f9955c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f9955c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de4d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56204/0 (socket says 172.21.15.44:56204) 2024-09-13T17:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f9955c4d640 1 -- 172.21.15.44:0/960641826 learned_addr learned my addr 172.21.15.44:0/960641826 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.479+0000 7f995644e640 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de3ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f995544c640 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de4dd60 0x55d86de50150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9946ffd640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f993c0a1280 0x7f993c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9955c4d640 1 -- 172.21.15.44:0/960641826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 msgr2=0x55d86de3ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9955c4d640 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de3ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9955c4d640 1 -- 172.21.15.44:0/960641826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de4dd60 msgr2=0x55d86de50150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9955c4d640 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de4dd60 0x55d86de50150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9955c4d640 1 -- 172.21.15.44:0/960641826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86dd69620 con 0x55d86de3ef90 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f995644e640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f993c0a1280 0x7f993c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9946ffd640 1 -- 172.21.15.44:0/445059078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f99400edb00 con 0x7f9950101830 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f9955c4d640 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de4d820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f994c009d80 tx=0x7f994c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856300b137fb1a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f993bfff640 1 -- 172.21.15.44:0/960641826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c01b670 con 0x55d86de3ef90 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f993bfff640 1 -- 172.21.15.44:0/960641826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f994c009e70 con 0x55d86de3ef90 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.480+0000 7f993bfff640 1 -- 172.21.15.44:0/960641826 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c012b60 con 0x55d86de3ef90 2024-09-13T17:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 -- 172.21.15.44:0/960641826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 msgr2=0x55d86de4d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de4d820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f994c009d80 tx=0x7f994c0117b0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995644e640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f993c0a1280 0x7f993c0a3740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9948005fd0 tx=0x7f9948004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 -- 172.21.15.44:0/960641826 shutdown_connections 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de4dd60 0x55d86de50150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de4d820 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 --2- 172.21.15.44:0/960641826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de3ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 -- 172.21.15.44:0/960641826 >> 172.21.15.44:0/960641826 conn(0x55d86de3ad00 msgr2=0x55d86de3b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 -- 172.21.15.44:0/960641826 shutdown_connections 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 -- 172.21.15.44:0/960641826 wait complete. 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 -- start start 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.481+0000 7f995abd2740 1 Processor -- start 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de4d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3ef90 0x55d86de4a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de4acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86dd60180 con 0x55d86de4dd60 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86da7a430 con 0x55d86de3ef90 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86dd255e0 con 0x55d86de3e670 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 10 client.0 _ll_register_callbacks cb 0x55d86dd91aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de4acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de4acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56214/0 (socket says 172.21.15.44:56214) 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 -- 172.21.15.44:0/1868913228 learned_addr learned my addr 172.21.15.44:0/1868913228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995544c640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3ef90 0x55d86de4a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f9955c4d640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de4d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 -- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 msgr2=0x55d86de4d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de4d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 -- 172.21.15.44:0/1868913228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3ef90 msgr2=0x55d86de4a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3ef90 0x55d86de4a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 -- 172.21.15.44:0/1868913228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c009980 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995644e640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de4acb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9948010830 tx=0x7f9948004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbc44ed03b4eb24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f99397fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f99397fa640 1 -- 172.21.15.44:0/1868913228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480119c0 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f99397fa640 1 -- 172.21.15.44:0/1868913228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9948011b20 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.482+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d86ddcbb80 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f99397fa640 1 -- 172.21.15.44:0/1868913228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9948011ce0 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 10 client.16317 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d86dda03c0 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f99397fa640 1 -- 172.21.15.44:0/1868913228 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d86ddcbb80 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f99397fa640 1 -- 172.21.15.44:0/1868913228 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f994801e3f0 con 0x55d86de4dd60 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 10 client.16317 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 20 client.16317 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86de5d830 0x55d86de5fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 4 client.16317 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d86dd941f0 con 0x55d86de5d830 2024-09-13T17:28:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.483+0000 7f9955c4d640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86de5d830 0x55d86de5fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f99397fa640 10 client.16317 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f9955c4d640 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86de5d830 0x55d86de5fc50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb36072ba34d187a server_cookie=407d5ed5cbc5f2f6 in_seq=0 out_seq=0 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f99397fa640 1 -- 172.21.15.44:0/1868913228 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d86dd941f0 con 0x55d86de5d830 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f99397fa640 10 client.16317 handle_command_reply: tid=0 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f995abd2740 1 client.16317 shutdown 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 msgr2=0x55d86de4acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.484+0000 7f995abd2740 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de4acb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9948010830 tx=0x7f9948004ab0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 shutdown_connections 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86de5d830 0x55d86de5fc50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de4acb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3ef90 0x55d86de4a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f99397fa640 0 client.16317 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 --2- 172.21.15.44:0/1868913228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3e670 0x55d86de4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 >> 172.21.15.44:0/1868913228 conn(0x55d86de3ad00 msgr2=0x55d86de40a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 shutdown_connections 2024-09-13T17:28:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 1 -- 172.21.15.44:0/1868913228 wait complete. 2024-09-13T17:28:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.485+0000 7f995abd2740 20 client.16317 trim_cache size 0 max 16384 2024-09-13T17:28:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 Processor -- start 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 -- start start 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de3a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3ef90 0x55d86de54a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de54f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86de368e0 con 0x55d86de4dd60 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86de48ca0 con 0x55d86de3e670 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995abd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86de3ed80 con 0x55d86de3ef90 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995644e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de54f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995644e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de54f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56216/0 (socket says 172.21.15.44:56216) 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995644e640 1 -- 172.21.15.44:0/1091485063 learned_addr learned my addr 172.21.15.44:0/1091485063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f995544c640 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3ef90 0x55d86de54a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.496+0000 7f9955c4d640 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de3a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995644e640 1 -- 172.21.15.44:0/1091485063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3ef90 msgr2=0x55d86de54a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995644e640 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3ef90 0x55d86de54a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995644e640 1 -- 172.21.15.44:0/1091485063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 msgr2=0x55d86de3a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995644e640 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de3a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995644e640 1 -- 172.21.15.44:0/1091485063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86dd42770 con 0x55d86de4dd60 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995644e640 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de54f80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9948013d80 tx=0x7f9948013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327888c544d550a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f9944ff9640 1 -- 172.21.15.44:0/1091485063 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480131a0 con 0x55d86de4dd60 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f9944ff9640 1 -- 172.21.15.44:0/1091485063 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f994801ea70 con 0x55d86de4dd60 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f9944ff9640 1 -- 172.21.15.44:0/1091485063 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948027830 con 0x55d86de4dd60 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.497+0000 7f995abd2740 1 -- 172.21.15.44:0/1091485063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 msgr2=0x55d86de54f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de54f80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9948013d80 tx=0x7f9948013f40 comp rx=0 tx=0).stop 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- 172.21.15.44:0/1091485063 shutdown_connections 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de3ef90 0x55d86de54a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de4dd60 0x55d86de54f80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- 172.21.15.44:0/1091485063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de3a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- 172.21.15.44:0/1091485063 >> 172.21.15.44:0/1091485063 conn(0x55d86de3ad00 msgr2=0x55d86de49020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- 172.21.15.44:0/1091485063 shutdown_connections 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- 172.21.15.44:0/1091485063 wait complete. 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- start start 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 Processor -- start 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de54f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de51420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de4dd60 0x55d86de51bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86de522b0 con 0x55d86de3ef90 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86deb3d50 con 0x55d86de3e670 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.498+0000 7f995abd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86deb3e90 con 0x55d86de4dd60 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995abd2740 10 client.0 _ll_register_callbacks cb 0x7f994c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:58.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de51420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de51420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56220/0 (socket says 172.21.15.44:56220) 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 -- 172.21.15.44:0/2551147219 learned_addr learned my addr 172.21.15.44:0/2551147219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f9955c4d640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de54f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995644e640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de4dd60 0x55d86de51bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 -- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de4dd60 msgr2=0x55d86de51bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de4dd60 0x55d86de51bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 -- 172.21.15.44:0/2551147219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 msgr2=0x55d86de54f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de54f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 -- 172.21.15.44:0/2551147219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d86dda03c0 con 0x55d86de3ef90 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f9955c4d640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de54f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995644e640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de4dd60 0x55d86de51bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995544c640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de51420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f99400b6df0 tx=0x7f99400ba310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2e76b640318b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f99397fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f99397fa640 1 -- 172.21.15.44:0/2551147219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400c2d70 con 0x55d86de3ef90 2024-09-13T17:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f99397fa640 1 -- 172.21.15.44:0/2551147219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f99400f3030 con 0x55d86de3ef90 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f994c009e70 con 0x55d86de3ef90 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f99397fa640 1 -- 172.21.15.44:0/2551147219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99400ba8f0 con 0x55d86de3ef90 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995abd2740 10 client.16323 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d86dd67e80 con 0x55d86de3ef90 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.499+0000 7f99397fa640 1 -- 172.21.15.44:0/2551147219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f994c009e70 con 0x55d86de3ef90 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f99397fa640 1 -- 172.21.15.44:0/2551147219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f99400bab50 con 0x55d86de3ef90 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f995abd2740 10 client.16323 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f995abd2740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f995abd2740 20 client.16323 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f995abd2740 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86deb5490 0x55d86deb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f995abd2740 4 client.16323 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d86da954e0 con 0x55d86deb5490 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f9955c4d640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86deb5490 0x55d86deb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f9955c4d640 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86deb5490 0x55d86deb78b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fefbc2630942f2f3 server_cookie=b2c5401b9cd55f03 in_seq=0 out_seq=0 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.500+0000 7f99397fa640 10 client.16323 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f99397fa640 1 -- 172.21.15.44:0/2551147219 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d86da954e0 con 0x55d86deb5490 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f99397fa640 10 client.16323 handle_command_reply: tid=0 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 client.16323 shutdown 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 msgr2=0x55d86de51420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de51420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f99400b6df0 tx=0x7f99400ba310 comp rx=0 tx=0).stop 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 shutdown_connections 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d86deb5490 0x55d86deb78b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f99397fa640 0 client.16323 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d86de4dd60 0x55d86de51bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d86de3ef90 0x55d86de51420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 --2- 172.21.15.44:0/2551147219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d86de3e670 0x55d86de54f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 >> 172.21.15.44:0/2551147219 conn(0x55d86de3ad00 msgr2=0x55d86de457c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 shutdown_connections 2024-09-13T17:28:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 1 -- 172.21.15.44:0/2551147219 wait complete. 2024-09-13T17:28:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f995abd2740 20 client.16323 trim_cache size 0 max 16384 2024-09-13T17:28:58.509 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f993c0a1280 msgr2=0x7f993c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f9957eb1640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f993c0a1280 0x7f993c0a3740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9948005fd0 tx=0x7f9948004a40 comp rx=0 tx=0).stop 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 msgr2=0x7f9950196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f9957eb1640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950196730 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f99400057c0 tx=0x7f9940032be0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.501+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 shutdown_connections 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f993c0a1280 0x7f993c0a3740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f995010a2e0 0x7f995019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950101830 0x7f9950196730 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 --2- 172.21.15.44:0/445059078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9950100f10 0x7f99501961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 >> 172.21.15.44:0/445059078 conn(0x7f99500fa8c0 msgr2=0x7f99500fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 shutdown_connections 2024-09-13T17:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:58.502+0000 7f9957eb1640 1 -- 172.21.15.44:0/445059078 wait complete. 2024-09-13T17:28:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:58 smithi100 ceph-mon[35055]: pgmap v70: 129 pgs: 129 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:28:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:28:58 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3783708014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:28:59.467 INFO:tasks.workunit.client.0.smithi044.stdout:1/600: open fd = 3 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:2/600: open fd = 3 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:3/600: open fd = 3 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.468 INFO:tasks.workunit.client.0.smithi044.stdout:4/600: open fd = 3 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:5/600: open fd = 3 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:6/600: open fd = 3 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:7/600: open fd = 3 2024-09-13T17:28:59.469 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:8/600: open fd = 3 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:9/600: open fd = 3 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:10/600: open fd = 3 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.470 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:11/600: open fd = 3 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:12/600: open fd = 3 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:13/600: open fd = 3 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:14/600: open fd = 3 2024-09-13T17:28:59.471 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:15/600: open fd = 3 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:16/600: open fd = 3 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:17/600: open fd = 3 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.472 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:18/600: open fd = 3 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:19/600: open fd = 3 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:20/600: open fd = 3 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:21/600: open fd = 3 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.473 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:22/600: open fd = 3 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:23/600: open fd = 3 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:24/600: open fd = 3 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.474 INFO:tasks.workunit.client.0.smithi044.stdout:25/600: open fd = 3 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:26/600: open fd = 3 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:27/600: open fd = 3 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:28/600: open fd = 3 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.475 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:29/600: open fd = 3 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:30/600: open fd = 3 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:31/600: open fd = 3 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.476 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:32/600: open fd = 3 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:33/600: open fd = 3 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:34/600: open fd = 3 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:35/600: open fd = 3 2024-09-13T17:28:59.477 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:36/600: open fd = 3 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:37/600: open fd = 3 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:38/600: open fd = 3 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.478 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:39/600: open fd = 3 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:40/600: open fd = 3 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:41/600: open fd = 3 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.479 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:42/600: open fd = 3 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:43/600: open fd = 3 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:44/600: open fd = 3 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.480 INFO:tasks.workunit.client.0.smithi044.stdout:45/600: open fd = 3 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:46/600: open fd = 3 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:47/600: open fd = 3 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:48/600: open fd = 3 2024-09-13T17:28:59.481 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:49/600: open fd = 3 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:50/600: open fd = 3 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.482 INFO:tasks.workunit.client.0.smithi044.stdout:51/600: open fd = 3 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:52/600: open fd = 3 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:53/600: open fd = 3 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.483 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:54/600: open fd = 3 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:55/600: open fd = 3 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:56/600: open fd = 3 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.484 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:57/600: open fd = 3 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:58/600: open fd = 3 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:59/600: open fd = 3 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.485 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:60/600: open fd = 3 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:61/600: open fd = 3 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:62/600: open fd = 3 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.486 INFO:tasks.workunit.client.0.smithi044.stdout:63/600: open fd = 3 2024-09-13T17:28:59.487 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.487 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:28:59.487 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:28:59.514 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:28:59.514 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:28:59.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.634+0000 7f5a4079c640 1 -- 172.21.15.44:0/49013862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38101800 msgr2=0x7f5a38109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.634+0000 7f5a4079c640 1 --2- 172.21.15.44:0/49013862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38101800 0x7f5a38109ce0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009900 tx=0x7f5a2802a480 comp rx=0 tx=0).stop 2024-09-13T17:28:59.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 -- 172.21.15.44:0/49013862 shutdown_connections 2024-09-13T17:28:59.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 --2- 172.21.15.44:0/49013862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3810a330 0x7f5a3810c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 --2- 172.21.15.44:0/49013862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38101800 0x7f5a38109ce0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 --2- 172.21.15.44:0/49013862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a38100ee0 0x7f5a381012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 -- 172.21.15.44:0/49013862 >> 172.21.15.44:0/49013862 conn(0x7f5a380fa8d0 msgr2=0x7f5a380fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 -- 172.21.15.44:0/49013862 shutdown_connections 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 -- 172.21.15.44:0/49013862 wait complete. 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 Processor -- start 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 -- start start 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 0x7f5a381961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.635+0000 7f5a4079c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101800 0x7f5a38196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a4079c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3810a330 0x7f5a3819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a4079c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3819b110 con 0x7f5a38100ee0 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a4079c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3819b280 con 0x7f5a3810a330 2024-09-13T17:28:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a4079c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3819b440 con 0x7f5a38101800 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 0x7f5a381961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 0x7f5a381961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33492/0 (socket says 172.21.15.44:33492) 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 -- 172.21.15.44:0/1025041734 learned_addr learned my addr 172.21.15.44:0/1025041734 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3810a330 0x7f5a3819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101800 0x7f5a38196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 -- 172.21.15.44:0/1025041734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101800 msgr2=0x7f5a38196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101800 0x7f5a38196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 -- 172.21.15.44:0/1025041734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3810a330 msgr2=0x7f5a3819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3810a330 0x7f5a3819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 -- 172.21.15.44:0/1025041734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28009560 con 0x7f5a38100ee0 2024-09-13T17:28:59.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 0x7f5a381961e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d950 tx=0x7f5a3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a2f7fe640 1 -- 172.21.15.44:0/1025041734 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34009c30 con 0x7f5a38100ee0 2024-09-13T17:28:59.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a2f7fe640 1 -- 172.21.15.44:0/1025041734 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a34009d90 con 0x7f5a38100ee0 2024-09-13T17:28:59.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3819b720 con 0x7f5a38100ee0 2024-09-13T17:28:59.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.636+0000 7f5a2f7fe640 1 -- 172.21.15.44:0/1025041734 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400f620 con 0x7f5a38100ee0 2024-09-13T17:28:59.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.637+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a381a2fa0 con 0x7f5a38100ee0 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.639+0000 7f5a2f7fe640 1 -- 172.21.15.44:0/1025041734 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5a340109f0 con 0x7f5a38100ee0 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.639+0000 7f5a434c3740 1 Processor -- start 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.639+0000 7f5a434c3740 1 -- start start 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e361056a0 0x562e36105a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36105fc0 0x562e36114850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 0x562e36117180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a434c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e36070c60 con 0x562e36114d90 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a434c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3606e3f0 con 0x562e36105fc0 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a434c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e36069ce0 con 0x562e361056a0 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 0x562e36117180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3ed39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e361056a0 0x562e36105a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 0x562e36117180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33506/0 (socket says 172.21.15.44:33506) 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 -- 172.21.15.44:0/1300238014 learned_addr learned my addr 172.21.15.44:0/1300238014 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a2f7fe640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5a240a1200 0x7f5a240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36105fc0 0x562e36114850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5a240a1200 0x7f5a240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a2f7fe640 1 -- 172.21.15.44:0/1025041734 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5a340ccc00 con 0x7f5a38100ee0 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 -- 172.21.15.44:0/1300238014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e361056a0 msgr2=0x562e36105a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e361056a0 0x562e36105a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 -- 172.21.15.44:0/1300238014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36105fc0 msgr2=0x562e36114850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36105fc0 0x562e36114850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3dd37640 1 -- 172.21.15.44:0/1300238014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e36025970 con 0x562e36114d90 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.640+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e361056a0 0x562e36105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:59.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.641+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 0x562e36117180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005940 tx=0x7f5a28005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20473219c37a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.641+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5a240a1200 0x7f5a240a36c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005fd0 tx=0x7f5a30004ab0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.641+0000 7f5a2cff9640 1 -- 172.21.15.44:0/1300238014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004030 con 0x562e36114d90 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.641+0000 7f5a2cff9640 1 -- 172.21.15.44:0/1300238014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a28031960 con 0x562e36114d90 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.641+0000 7f5a2cff9640 1 -- 172.21.15.44:0/1300238014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280039e0 con 0x562e36114d90 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 -- 172.21.15.44:0/1300238014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 msgr2=0x562e36117180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 0x562e36117180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005940 tx=0x7f5a28005e80 comp rx=0 tx=0).stop 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 -- 172.21.15.44:0/1300238014 shutdown_connections 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36114d90 0x562e36117180 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36105fc0 0x562e36114850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1300238014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e361056a0 0x562e36105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 -- 172.21.15.44:0/1300238014 >> 172.21.15.44:0/1300238014 conn(0x562e36101d50 msgr2=0x562e36102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 -- 172.21.15.44:0/1300238014 shutdown_connections 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 -- 172.21.15.44:0/1300238014 wait complete. 2024-09-13T17:28:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 -- start start 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 Processor -- start 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.642+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e361119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e36111f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e36027f80 con 0x562e361056a0 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e36027c20 con 0x562e36114d90 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e35ff4ab0 con 0x562e36105fc0 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 10 client.0 _ll_register_callbacks cb 0x562e36063350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33520/0 (socket says 172.21.15.44:33520) 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 -- 172.21.15.44:0/1074733936 learned_addr learned my addr 172.21.15.44:0/1074733936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e361119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e36111f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 -- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 msgr2=0x562e361119b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e361119b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 -- 172.21.15.44:0/1074733936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 msgr2=0x562e36111f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e36111f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 -- 172.21.15.44:0/1074733936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3810d430 con 0x562e361056a0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e36111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a3e538640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114710 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ca0e0 tx=0x7f5a3406bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a202e83851b18f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a21ffb640 1 -- 172.21.15.44:0/1074733936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340999f0 con 0x562e361056a0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a21ffb640 1 -- 172.21.15.44:0/1074733936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5a3409d820 con 0x562e361056a0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.643+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e35d5d590 con 0x562e361056a0 2024-09-13T17:28:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a21ffb640 1 -- 172.21.15.44:0/1074733936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a340a1e70 con 0x562e361056a0 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 10 client.16335 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562e36068360 con 0x562e361056a0 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a21ffb640 1 -- 172.21.15.44:0/1074733936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562e35d5d590 con 0x562e361056a0 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a21ffb640 1 -- 172.21.15.44:0/1074733936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a34098950 con 0x562e361056a0 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 10 client.16335 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 20 client.16335 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e36124a40 0x562e36126e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 4 client.16335 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e35d6e860 con 0x562e36124a40 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.644+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e36124a40 0x562e36126e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e36124a40 0x562e36126e30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1f71abe443a53b server_cookie=bde17c3045bd877b in_seq=0 out_seq=0 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a21ffb640 10 client.16335 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a21ffb640 1 -- 172.21.15.44:0/1074733936 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562e35d6e860 con 0x562e36124a40 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a21ffb640 10 client.16335 handle_command_reply: tid=0 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 client.16335 shutdown 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 msgr2=0x562e36114710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114710 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ca0e0 tx=0x7f5a3406bad0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 shutdown_connections 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e36124a40 0x562e36126e30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e361119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e36111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a21ffb640 0 client.16335 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.645+0000 7f5a434c3740 1 --2- 172.21.15.44:0/1074733936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114710 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.646+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 >> 172.21.15.44:0/1074733936 conn(0x562e36101d50 msgr2=0x562e36101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.646+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 shutdown_connections 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.646+0000 7f5a434c3740 1 -- 172.21.15.44:0/1074733936 wait complete. 2024-09-13T17:28:59.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.646+0000 7f5a434c3740 20 client.16335 trim_cache size 0 max 16384 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 Processor -- start 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 -- start start 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e3611bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e3611bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e35d5d5c0 con 0x562e361056a0 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e36112b50 con 0x562e36114d90 2024-09-13T17:28:59.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a434c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3610c7f0 con 0x562e36105fc0 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33532/0 (socket says 172.21.15.44:33532) 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.660+0000 7f5a3e538640 1 -- 172.21.15.44:0/2496121257 learned_addr learned my addr 172.21.15.44:0/2496121257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e3611bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3e538640 1 -- 172.21.15.44:0/2496121257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 msgr2=0x562e3611bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3e538640 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e3611bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3e538640 1 -- 172.21.15.44:0/2496121257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 msgr2=0x562e3611bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3e538640 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e3611bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3e538640 1 -- 172.21.15.44:0/2496121257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e3600a1c0 con 0x562e361056a0 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a3e538640 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114220 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ca630 tx=0x7f5a34095df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d912f75d7417a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a2d7fa640 1 -- 172.21.15.44:0/2496121257 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34099350 con 0x562e361056a0 2024-09-13T17:28:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a2d7fa640 1 -- 172.21.15.44:0/2496121257 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a340997c0 con 0x562e361056a0 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.661+0000 7f5a2d7fa640 1 -- 172.21.15.44:0/2496121257 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340a1c50 con 0x562e361056a0 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- 172.21.15.44:0/2496121257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 msgr2=0x562e36114220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114220 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ca630 tx=0x7f5a34095df0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- 172.21.15.44:0/2496121257 shutdown_connections 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36105fc0 0x562e3611bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e36114d90 0x562e3611bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2496121257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e361056a0 0x562e36114220 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- 172.21.15.44:0/2496121257 >> 172.21.15.44:0/2496121257 conn(0x562e36101d50 msgr2=0x562e36110260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- 172.21.15.44:0/2496121257 shutdown_connections 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- 172.21.15.44:0/2496121257 wait complete. 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- start start 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 Processor -- start 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e361056a0 0x562e3610e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 0x562e3610ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36114d90 0x562e3610f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3617af50 con 0x562e36105fc0 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3617b0c0 con 0x562e361056a0 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3617b280 con 0x562e36114d90 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a434c3740 10 client.0 _ll_register_callbacks cb 0x7f5a34010ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.662+0000 7f5a3dd37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 0x562e3610ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3e538640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e361056a0 0x562e3610e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 0x562e3610ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33542/0 (socket says 172.21.15.44:33542) 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 -- 172.21.15.44:0/2790175058 learned_addr learned my addr 172.21.15.44:0/2790175058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3ed39640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36114d90 0x562e3610f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 -- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36114d90 msgr2=0x562e3610f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36114d90 0x562e3610f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 -- 172.21.15.44:0/2790175058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e361056a0 msgr2=0x562e3610e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e361056a0 0x562e3610e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 -- 172.21.15.44:0/2790175058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e36068360 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3e538640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e361056a0 0x562e3610e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a3dd37640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 0x562e3610ec00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005dc0 tx=0x7f5a280317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a33d46378aca891 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a21ffb640 1 -- 172.21.15.44:0/2790175058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004440 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a21ffb640 1 -- 172.21.15.44:0/2790175058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5a28055030 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a28031960 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a21ffb640 1 -- 172.21.15.44:0/2790175058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a280503a0 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a434c3740 10 client.16341 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.663+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562e3602f990 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a21ffb640 1 -- 172.21.15.44:0/2790175058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a28031960 con 0x562e36105fc0 2024-09-13T17:28:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a21ffb640 1 -- 172.21.15.44:0/2790175058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a28042020 con 0x562e36105fc0 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a434c3740 10 client.16341 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a434c3740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a434c3740 20 client.16341 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e3617c730 0x562e3617eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a434c3740 4 client.16341 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562e35dc1eb0 con 0x562e3617c730 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a3e538640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e3617c730 0x562e3617eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a21ffb640 10 client.16341 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.664+0000 7f5a3e538640 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e3617c730 0x562e3617eb50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51855470caf6698 server_cookie=3c96a46f99b199d3 in_seq=0 out_seq=0 2024-09-13T17:28:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a21ffb640 1 -- 172.21.15.44:0/2790175058 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562e35dc1eb0 con 0x562e3617c730 2024-09-13T17:28:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a21ffb640 10 client.16341 handle_command_reply: tid=0 2024-09-13T17:28:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 client.16341 shutdown 2024-09-13T17:28:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 msgr2=0x562e3610ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 0x562e3610ec00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005dc0 tx=0x7f5a280317d0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 shutdown_connections 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562e3617c730 0x562e3617eb50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e36114d90 0x562e3610f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.665+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e36105fc0 0x562e3610ec00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a434c3740 1 --2- 172.21.15.44:0/2790175058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e361056a0 0x562e3610e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 >> 172.21.15.44:0/2790175058 conn(0x562e36101d50 msgr2=0x562e36107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a21ffb640 0 client.16341 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 shutdown_connections 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a434c3740 1 -- 172.21.15.44:0/2790175058 wait complete. 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a434c3740 20 client.16341 trim_cache size 0 max 16384 2024-09-13T17:28:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:28:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:28:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "efae9cbf-d3b8-4e22-9347-3d576d9dfedc", 2024-09-13T17:28:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:28:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:28:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5a240a1200 msgr2=0x7f5a240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5a240a1200 0x7f5a240a36c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005fd0 tx=0x7f5a30004ab0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 msgr2=0x7f5a381961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 0x7f5a381961e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d950 tx=0x7f5a3400de10 comp rx=0 tx=0).stop 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 shutdown_connections 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5a240a1200 0x7f5a240a36c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101800 0x7f5a38196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3810a330 0x7f5a3819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 --2- 172.21.15.44:0/1025041734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a38100ee0 0x7f5a381961e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 >> 172.21.15.44:0/1025041734 conn(0x7f5a380fa8d0 msgr2=0x7f5a380fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 shutdown_connections 2024-09-13T17:28:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.666+0000 7f5a4079c640 1 -- 172.21.15.44:0/1025041734 wait complete. 2024-09-13T17:28:59.678 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:28:59.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 -- 172.21.15.44:0/252769494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca410a310 msgr2=0x7f5ca410c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 --2- 172.21.15.44:0/252769494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca410a310 0x7f5ca410c790 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009ca0 tx=0x7f5c9c02a950 comp rx=0 tx=0).stop 2024-09-13T17:28:59.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 -- 172.21.15.44:0/252769494 shutdown_connections 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 --2- 172.21.15.44:0/252769494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca410a310 0x7f5ca410c790 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 --2- 172.21.15.44:0/252769494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca4101860 0x7f5ca4109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 --2- 172.21.15.44:0/252769494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4100f40 0x7f5ca4101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 -- 172.21.15.44:0/252769494 >> 172.21.15.44:0/252769494 conn(0x7f5ca4075c60 msgr2=0x7f5ca40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 -- 172.21.15.44:0/252769494 shutdown_connections 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 -- 172.21.15.44:0/252769494 wait complete. 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 Processor -- start 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.782+0000 7f5caa872640 1 -- start start 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 0x7f5ca40feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4101860 0x7f5ca40ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca410a310 0x7f5ca40ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41a4410 con 0x7f5ca4100f40 2024-09-13T17:28:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41a4580 con 0x7f5ca410a310 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41a4740 con 0x7f5ca4101860 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 0x7f5ca40feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 0x7f5ca40feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33564/0 (socket says 172.21.15.44:33564) 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 -- 172.21.15.44:0/1381448263 learned_addr learned my addr 172.21.15.44:0/1381448263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca410a310 0x7f5ca40ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4101860 0x7f5ca40ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 -- 172.21.15.44:0/1381448263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4101860 msgr2=0x7f5ca40ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4101860 0x7f5ca40ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 -- 172.21.15.44:0/1381448263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca410a310 msgr2=0x7f5ca40ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca410a310 0x7f5ca40ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 -- 172.21.15.44:0/1381448263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009900 con 0x7f5ca4100f40 2024-09-13T17:28:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 0x7f5ca40feb90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400c920 tx=0x7f5c9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca1ffb640 1 -- 172.21.15.44:0/1381448263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94007c70 con 0x7f5ca4100f40 2024-09-13T17:28:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca1ffb640 1 -- 172.21.15.44:0/1381448263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c94007dd0 con 0x7f5ca4100f40 2024-09-13T17:28:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca41a4a20 con 0x7f5ca4100f40 2024-09-13T17:28:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5ca1ffb640 1 -- 172.21.15.44:0/1381448263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94018730 con 0x7f5ca4100f40 2024-09-13T17:28:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.783+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca41a4f80 con 0x7f5ca4100f40 2024-09-13T17:28:59.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5ca1ffb640 1 -- 172.21.15.44:0/1381448263 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c9400f450 con 0x7f5ca4100f40 2024-09-13T17:28:59.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 Processor -- start 2024-09-13T17:28:59.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 -- start start 2024-09-13T17:28:59.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 0x557e6b6ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b658f20 con 0x557e6b6ede10 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b6566b0 con 0x557e6b6ee730 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5cad672740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b651fa0 con 0x557e6b6f8fd0 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5ca8e0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 0x557e6b6ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5ca8e0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 0x557e6b6ee1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33576/0 (socket says 172.21.15.44:33576) 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/738937901 learned_addr learned my addr 172.21.15.44:0/738937901 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.786+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca1ffb640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c900a1280 0x7f5c900a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/738937901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 msgr2=0x557e6b6fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/738937901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 msgr2=0x557e6b6f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/738937901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6b60c840 con 0x557e6b6ede10 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca1ffb640 1 -- 172.21.15.44:0/1381448263 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5c940cd1a0 con 0x7f5ca4100f40 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c900a1280 0x7f5c900a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 0x557e6b6ee1f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c035000 tx=0x7f5c9c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c430b7efa704d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5c9a7fc640 1 -- 172.21.15.44:0/738937901 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c003e80 con 0x557e6b6ede10 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5c9a7fc640 1 -- 172.21.15.44:0/738937901 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c9c040070 con 0x557e6b6ede10 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.787+0000 7f5c9a7fc640 1 -- 172.21.15.44:0/738937901 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c004460 con 0x557e6b6ede10 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 -- 172.21.15.44:0/738937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 msgr2=0x557e6b6ee1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c900a1280 0x7f5c900a3740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c04e000 tx=0x7f5c9c04b000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 0x557e6b6ee1f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c035000 tx=0x7f5c9c02f470 comp rx=0 tx=0).stop 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 -- 172.21.15.44:0/738937901 shutdown_connections 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 --2- 172.21.15.44:0/738937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ede10 0x557e6b6ee1f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 -- 172.21.15.44:0/738937901 >> 172.21.15.44:0/738937901 conn(0x557e6b6ea380 msgr2=0x557e6b6ea760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 -- 172.21.15.44:0/738937901 shutdown_connections 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 -- 172.21.15.44:0/738937901 wait complete. 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.788+0000 7f5cad672740 1 -- start start 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 Processor -- start 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b6f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b5f32a0 con 0x557e6b6ee730 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b32a430 con 0x557e6b6f8fd0 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b5d5680 con 0x557e6b6ede10 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5cad672740 10 client.0 _ll_register_callbacks cb 0x557e6b64b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:59.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33592/0 (socket says 172.21.15.44:33592) 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 -- 172.21.15.44:0/501640667 learned_addr learned my addr 172.21.15.44:0/501640667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b6f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 -- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 msgr2=0x557e6b6f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b6f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 -- 172.21.15.44:0/501640667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6f8fd0 msgr2=0x557e6b6f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 -- 172.21.15.44:0/501640667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca410d410 con 0x557e6b6ee730 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b6f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f5b40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d9a0 tx=0x7f5c8c00de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6414e801a11066 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.789+0000 7f5c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5c837fe640 1 -- 172.21.15.44:0/501640667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c003ea0 con 0x557e6b6ee730 2024-09-13T17:28:59.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e6b3454e0 con 0x557e6b6ee730 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5c837fe640 1 -- 172.21.15.44:0/501640667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5c8c005220 con 0x557e6b6ee730 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5c837fe640 1 -- 172.21.15.44:0/501640667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c8c0053e0 con 0x557e6b6ee730 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 10 client.16353 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557e6b650620 con 0x557e6b6ee730 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5c837fe640 1 -- 172.21.15.44:0/501640667 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557e6b3454e0 con 0x557e6b6ee730 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5c837fe640 1 -- 172.21.15.44:0/501640667 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c8c011690 con 0x557e6b6ee730 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 10 client.16353 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 20 client.16353 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b708d70 0x557e6b70b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 4 client.16353 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.790+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e6b3567e0 con 0x557e6b708d70 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.791+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b708d70 0x557e6b70b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.791+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b708d70 0x557e6b70b190 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cee5898d450473 server_cookie=8d601e888d6b0984 in_seq=0 out_seq=0 2024-09-13T17:28:59.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.791+0000 7f5c837fe640 10 client.16353 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5c837fe640 1 -- 172.21.15.44:0/501640667 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557e6b3567e0 con 0x557e6b708d70 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5c837fe640 10 client.16353 handle_command_reply: tid=0 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 client.16353 shutdown 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 msgr2=0x557e6b6f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f5b40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d9a0 tx=0x7f5c8c00de60 comp rx=0 tx=0).stop 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 shutdown_connections 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b708d70 0x557e6b70b190 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f5b40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 --2- 172.21.15.44:0/501640667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b6f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5c837fe640 0 client.16353 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 >> 172.21.15.44:0/501640667 conn(0x557e6b6ea380 msgr2=0x557e6b6f00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 shutdown_connections 2024-09-13T17:28:59.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 1 -- 172.21.15.44:0/501640667 wait complete. 2024-09-13T17:28:59.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.792+0000 7f5cad672740 20 client.16353 trim_cache size 0 max 16384 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 Processor -- start 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 -- start start 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ede10 0x557e6b6e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b7053b0 con 0x557e6b6ee730 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b6f4e70 con 0x557e6b6ede10 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5cad672740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b7103a0 con 0x557e6b6f8fd0 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33604/0 (socket says 172.21.15.44:33604) 2024-09-13T17:28:59.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.806+0000 7f5c9bfff640 1 -- 172.21.15.44:0/507218161 learned_addr learned my addr 172.21.15.44:0/507218161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ede10 0x557e6b6e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9bfff640 1 -- 172.21.15.44:0/507218161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 msgr2=0x557e6b6f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9bfff640 1 -- 172.21.15.44:0/507218161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ede10 msgr2=0x557e6b6e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ede10 0x557e6b6e9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9bfff640 1 -- 172.21.15.44:0/507218161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6b5f2790 con 0x557e6b6ee730 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ede10 0x557e6b6e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f43f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c000c00 tx=0x7f5c8c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8defd35bc07d4815 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9affd640 1 -- 172.21.15.44:0/507218161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c003ea0 con 0x557e6b6ee730 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9affd640 1 -- 172.21.15.44:0/507218161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c8c005070 con 0x557e6b6ee730 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.807+0000 7f5c9affd640 1 -- 172.21.15.44:0/507218161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0048c0 con 0x557e6b6ee730 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- 172.21.15.44:0/507218161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 msgr2=0x557e6b6f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f43f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c000c00 tx=0x7f5c8c0039c0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- 172.21.15.44:0/507218161 shutdown_connections 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6f8fd0 0x557e6b6f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6ee730 0x557e6b6f43f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- 172.21.15.44:0/507218161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ede10 0x557e6b6e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- 172.21.15.44:0/507218161 >> 172.21.15.44:0/507218161 conn(0x557e6b6ea380 msgr2=0x557e6b6f0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- 172.21.15.44:0/507218161 shutdown_connections 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- 172.21.15.44:0/507218161 wait complete. 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- start start 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 Processor -- start 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b7001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b6ff380 con 0x557e6b6f8fd0 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b6ff4f0 con 0x557e6b6ee730 2024-09-13T17:28:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b75f2b0 con 0x557e6b6ede10 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.808+0000 7f5cad672740 10 client.0 _ll_register_callbacks cb 0x7f5c8c004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33618/0 (socket says 172.21.15.44:33618) 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/2598394907 learned_addr learned my addr 172.21.15.44:0/2598394907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b7001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 msgr2=0x557e6b7001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b7001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/2598394907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 msgr2=0x557e6b6fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 -- 172.21.15.44:0/2598394907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c01f040 con 0x557e6b6f8fd0 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5c9bfff640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:28:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5ca8e0f640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fec80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009430 tx=0x7f5c9c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b49c0b0a0eca3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5c837fe640 1 -- 172.21.15.44:0/2598394907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c057040 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5c837fe640 1 -- 172.21.15.44:0/2598394907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5c9c004400 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c9c040070 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5c837fe640 1 -- 172.21.15.44:0/2598394907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9c003e80 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5cad672740 10 client.16359 fetch_fsmap learned FSMap version 25 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.809+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557e6b617ea0 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5c837fe640 1 -- 172.21.15.44:0/2598394907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c9c040070 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5c837fe640 1 -- 172.21.15.44:0/2598394907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c9c050020 con 0x557e6b6f8fd0 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5cad672740 10 client.16359 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5cad672740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5cad672740 20 client.16359 populate_metadata read hostname 'smithi044' 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5cad672740 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b760810 0x557e6b762c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5cad672740 4 client.16359 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:28:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e6b3a9e30 con 0x557e6b760810 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b760810 0x557e6b762c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5c837fe640 10 client.16359 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.810+0000 7f5ca3fff640 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b760810 0x557e6b762c30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8713b5b42556b1 server_cookie=14c390cd13a11935 in_seq=0 out_seq=0 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5c837fe640 1 -- 172.21.15.44:0/2598394907 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557e6b3a9e30 con 0x557e6b760810 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5c837fe640 10 client.16359 handle_command_reply: tid=0 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 client.16359 shutdown 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 msgr2=0x557e6b6fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fec80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009430 tx=0x7f5c9c004040 comp rx=0 tx=0).stop 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 shutdown_connections 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557e6b760810 0x557e6b762c30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e6b6f8fd0 0x557e6b6fec80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557e6b6ee730 0x557e6b6fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 --2- 172.21.15.44:0/2598394907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e6b6ede10 0x557e6b7001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5c837fe640 0 client.16359 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 >> 172.21.15.44:0/2598394907 conn(0x557e6b6ea380 msgr2=0x557e6b6f0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 shutdown_connections 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 1 -- 172.21.15.44:0/2598394907 wait complete. 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.811+0000 7f5cad672740 20 client.16359 trim_cache size 0 max 16384 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "efae9cbf-d3b8-4e22-9347-3d576d9dfedc": { 2024-09-13T17:28:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:28:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "efae9cbf-d3b8-4e22-9347-3d576d9dfedc", 2024-09-13T17:28:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:28:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:28:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:28:59.820 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c900a1280 msgr2=0x7f5c900a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c900a1280 0x7f5c900a3740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c04e000 tx=0x7f5c9c04b000 comp rx=0 tx=0).stop 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 msgr2=0x7f5ca40feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 0x7f5ca40feb90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400c920 tx=0x7f5c9400cde0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 shutdown_connections 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c900a1280 0x7f5c900a3740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4101860 0x7f5ca40ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca410a310 0x7f5ca40ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 --2- 172.21.15.44:0/1381448263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ca4100f40 0x7f5ca40feb90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 >> 172.21.15.44:0/1381448263 conn(0x7f5ca4075c60 msgr2=0x7f5ca40fb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 shutdown_connections 2024-09-13T17:28:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:28:59.812+0000 7f5caa872640 1 -- 172.21.15.44:0/1381448263 wait complete. 2024-09-13T17:28:59.823 INFO:tasks.cephfs.filesystem:scrub status for tag:efae9cbf-d3b8-4e22-9347-3d576d9dfedc - {'path': '/', 'tag': 'efae9cbf-d3b8-4e22-9347-3d576d9dfedc', 'options': 'recursive,force'} 2024-09-13T17:29:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:00 smithi100 ceph-mon[35055]: pgmap v71: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-13T17:29:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:00 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:29:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:00 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:00 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:29:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:00 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:00 smithi044 ceph-mon[31175]: pgmap v71: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-13T17:29:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:00 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:29:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:00 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:00 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:29:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:00 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:00 smithi070 ceph-mon[34502]: pgmap v71: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-13T17:29:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:00 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:29:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:00 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:00 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:29:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:00 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:02 smithi100 ceph-mon[35055]: pgmap v72: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:29:02.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:02 smithi044 ceph-mon[31175]: pgmap v72: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:29:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:02 smithi070 ceph-mon[34502]: pgmap v72: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:29:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 -- 172.21.15.44:0/4143944927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c28103f70 msgr2=0x7f3c2810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 --2- 172.21.15.44:0/4143944927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c28103f70 0x7f3c2810bac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009cd0 tx=0x7f3c2002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 -- 172.21.15.44:0/4143944927 shutdown_connections 2024-09-13T17:29:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 --2- 172.21.15.44:0/4143944927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c280ff7d0 0x7f3c28103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 --2- 172.21.15.44:0/4143944927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c28103f70 0x7f3c2810bac0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 --2- 172.21.15.44:0/4143944927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c280fee20 0x7f3c280ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 -- 172.21.15.44:0/4143944927 >> 172.21.15.44:0/4143944927 conn(0x7f3c280fa8f0 msgr2=0x7f3c280fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 -- 172.21.15.44:0/4143944927 shutdown_connections 2024-09-13T17:29:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 -- 172.21.15.44:0/4143944927 wait complete. 2024-09-13T17:29:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 Processor -- start 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.282+0000 7f3c30432640 1 -- start start 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c30432640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c280fee20 0x7f3c28198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c30432640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c28103f70 0x7f3c28198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c30432640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 0x7f3c2819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c30432640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2819d640 con 0x7f3c2819cc50 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c30432640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2819d7b0 con 0x7f3c28103f70 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c30432640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2819d920 con 0x7f3c280fee20 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 0x7f3c2819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 0x7f3c2819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33650/0 (socket says 172.21.15.44:33650) 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3654774848 learned_addr learned my addr 172.21.15.44:0/3654774848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2d9cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c28103f70 0x7f3c28198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c280fee20 0x7f3c28198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3654774848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c280fee20 msgr2=0x7f3c28198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c280fee20 0x7f3c28198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3654774848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c28103f70 msgr2=0x7f3c28198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c28103f70 0x7f3c28198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3654774848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c20009900 con 0x7f3c2819cc50 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.283+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 0x7f3c2819d100 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3c20004af0 tx=0x7f3c20030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.284+0000 7f3c1f7fe640 1 -- 172.21.15.44:0/3654774848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20003970 con 0x7f3c2819cc50 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.284+0000 7f3c1f7fe640 1 -- 172.21.15.44:0/3654774848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c20004040 con 0x7f3c2819cc50 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.284+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c281a4f00 con 0x7f3c2819cc50 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.284+0000 7f3c1f7fe640 1 -- 172.21.15.44:0/3654774848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20045650 con 0x7f3c2819cc50 2024-09-13T17:29:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.284+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c281a5380 con 0x7f3c2819cc50 2024-09-13T17:29:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.286+0000 7f3c1f7fe640 1 -- 172.21.15.44:0/3654774848 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3c2000a040 con 0x7f3c2819cc50 2024-09-13T17:29:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.286+0000 7f3c33230740 1 Processor -- start 2024-09-13T17:29:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 -- start start 2024-09-13T17:29:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c1f7fe640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3c140a1200 0x7f3c140a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 0x55c10a1d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1de4f0 0x55c10a1e08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a13ec30 con 0x55c10a1d3c50 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a13c3c0 con 0x55c10a1d3330 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c33230740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a137cb0 con 0x55c10a1de4f0 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3c140a1200 0x7f3c140a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c1f7fe640 1 -- 172.21.15.44:0/3654774848 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3c200ee3d0 con 0x7f3c2819cc50 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 0x55c10a1d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 0x55c10a1d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:48998/0 (socket says 172.21.15.44:48998) 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/3304629102 learned_addr learned my addr 172.21.15.44:0/3304629102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1de4f0 0x55c10a1e08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/3304629102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1de4f0 msgr2=0x55c10a1e08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1de4f0 0x55c10a1e08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/3304629102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 msgr2=0x55c10a1ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.287+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/3304629102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c10a0de910 con 0x55c10a1d3330 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.288+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3c140a1200 0x7f3c140a36c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c281998c0 tx=0x7f3c1800bac0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.288+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 0x55c10a1d3710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400bd10 tx=0x7f3c240107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9b7f1874fd0b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.288+0000 7f3c1ce05640 1 -- 172.21.15.44:0/3304629102 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24004030 con 0x55c10a1d3330 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.288+0000 7f3c1ce05640 1 -- 172.21.15.44:0/3304629102 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c24015070 con 0x55c10a1d3330 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.288+0000 7f3c1ce05640 1 -- 172.21.15.44:0/3304629102 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c240116a0 con 0x55c10a1d3330 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 -- 172.21.15.44:0/3304629102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 msgr2=0x55c10a1d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 0x55c10a1d3710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400bd10 tx=0x7f3c240107a0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 -- 172.21.15.44:0/3304629102 shutdown_connections 2024-09-13T17:29:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1de4f0 0x55c10a1e08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 --2- 172.21.15.44:0/3304629102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3330 0x55c10a1d3710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 -- 172.21.15.44:0/3304629102 >> 172.21.15.44:0/3304629102 conn(0x55c10a1cf8c0 msgr2=0x55c10a1cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 -- 172.21.15.44:0/3304629102 shutdown_connections 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 -- 172.21.15.44:0/3304629102 wait complete. 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 -- start start 2024-09-13T17:29:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.289+0000 7f3c33230740 1 Processor -- start 2024-09-13T17:29:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a0f5e80 con 0x55c10a1d3c50 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a0f5b20 con 0x55c10a1de4f0 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a0bb260 con 0x55c10a1d3330 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c33230740 10 client.0 _ll_register_callbacks cb 0x55c10a0f0df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1db170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33662/0 (socket says 172.21.15.44:33662) 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 -- 172.21.15.44:0/1708838863 learned_addr learned my addr 172.21.15.44:0/1708838863 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 -- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 msgr2=0x55c10a1dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 -- 172.21.15.44:0/1708838863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 msgr2=0x55c10a1db740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1db740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 -- 172.21.15.44:0/1708838863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2400b940 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1db170 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3c18012440 tx=0x7f3c1800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8e4b1de794cdd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.290+0000 7f3c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c11ffb640 1 -- 172.21.15.44:0/1708838863 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18010ea0 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c10a160680 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c11ffb640 1 -- 172.21.15.44:0/1708838863 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3c1802a410 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c11ffb640 1 -- 172.21.15.44:0/1708838863 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c1802a5d0 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 10 client.16371 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c10a136330 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c11ffb640 1 -- 172.21.15.44:0/1708838863 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c10a160680 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c11ffb640 1 -- 172.21.15.44:0/1708838863 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3c18020600 con 0x55c10a1d3c50 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 10 client.16371 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 20 client.16371 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a1edf20 0x55c10a1f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 4 client.16371 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.291+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c10a12b5d0 con 0x55c10a1edf20 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.292+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a1edf20 0x55c10a1f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.292+0000 7f3c11ffb640 10 client.16371 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.292+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a1edf20 0x55c10a1f0340 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520bb2313391efeb server_cookie=21a6c86d92c69df8 in_seq=0 out_seq=0 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c11ffb640 1 -- 172.21.15.44:0/1708838863 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c10a12b5d0 con 0x55c10a1edf20 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c11ffb640 10 client.16371 handle_command_reply: tid=0 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 client.16371 shutdown 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 msgr2=0x55c10a1db170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1db170 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3c18012440 tx=0x7f3c1800b410 comp rx=0 tx=0).stop 2024-09-13T17:29:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 shutdown_connections 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a1edf20 0x55c10a1f0340 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1db740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3c50 0x55c10a1db170 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c11ffb640 0 client.16371 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 --2- 172.21.15.44:0/1708838863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 >> 172.21.15.44:0/1708838863 conn(0x55c10a1cf8c0 msgr2=0x55c10a1ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 shutdown_connections 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 1 -- 172.21.15.44:0/1708838863 wait complete. 2024-09-13T17:29:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.293+0000 7f3c33230740 20 client.16371 trim_cache size 0 max 16384 2024-09-13T17:29:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 Processor -- start 2024-09-13T17:29:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 -- start start 2024-09-13T17:29:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 0x55c10a1cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3c50 0x55c10a1e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a1eac00 con 0x55c10a1d3330 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a1d96d0 con 0x55c10a1de4f0 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c33230740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a1ea490 con 0x55c10a1d3c50 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 0x55c10a1cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 0x55c10a1cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33672/0 (socket says 172.21.15.44:33672) 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/2363211327 learned_addr learned my addr 172.21.15.44:0/2363211327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3c50 0x55c10a1e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/2363211327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3c50 msgr2=0x55c10a1e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3c50 0x55c10a1e51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/2363211327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 msgr2=0x55c10a1e56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1e56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 -- 172.21.15.44:0/2363211327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c10a0d8300 con 0x55c10a1d3330 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.306+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 0x55c10a1cf300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400bd10 tx=0x7f3c24003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20b9996c6f51c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c1d606640 1 -- 172.21.15.44:0/2363211327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24020070 con 0x55c10a1d3330 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c1d606640 1 -- 172.21.15.44:0/2363211327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c24021040 con 0x55c10a1d3330 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c1d606640 1 -- 172.21.15.44:0/2363211327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24028340 con 0x55c10a1d3330 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 -- 172.21.15.44:0/2363211327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 msgr2=0x55c10a1cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 0x55c10a1cf300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400bd10 tx=0x7f3c24003c00 comp rx=0 tx=0).stop 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 -- 172.21.15.44:0/2363211327 shutdown_connections 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3c50 0x55c10a1e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1de4f0 0x55c10a1e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 --2- 172.21.15.44:0/2363211327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1d3330 0x55c10a1cf300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 -- 172.21.15.44:0/2363211327 >> 172.21.15.44:0/2363211327 conn(0x55c10a1cf8c0 msgr2=0x55c10a1f4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 -- 172.21.15.44:0/2363211327 shutdown_connections 2024-09-13T17:29:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 -- 172.21.15.44:0/2363211327 wait complete. 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.307+0000 7f3c33230740 1 -- start start 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 Processor -- start 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3c50 0x55c10a1e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 0x55c10a1e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a1e2560 con 0x55c10a1de4f0 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a1e26d0 con 0x55c10a1d3c50 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10a244420 con 0x55c10a1d3330 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c33230740 10 client.0 _ll_register_callbacks cb 0x7f3c2400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 0x55c10a1e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2d9cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3c50 0x55c10a1e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 0x55c10a1e1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33688/0 (socket says 172.21.15.44:33688) 2024-09-13T17:29:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3905643959 learned_addr learned my addr 172.21.15.44:0/3905643959 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 msgr2=0x55c10a1e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3905643959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3c50 msgr2=0x55c10a1e1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3c50 0x55c10a1e1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 -- 172.21.15.44:0/3905643959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c10a0de910 con 0x55c10a1de4f0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2d9cd640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3c50 0x55c10a1e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c2e9cf640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 0x55c10a1e1e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3c200b7710 tx=0x7f3c2008cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606a00719cdfe073 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.308+0000 7f3c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c11ffb640 1 -- 172.21.15.44:0/3905643959 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c200c35e0 con 0x55c10a1de4f0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c11ffb640 1 -- 172.21.15.44:0/3905643959 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3c200a7070 con 0x55c10a1de4f0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c24015070 con 0x55c10a1de4f0 2024-09-13T17:29:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c11ffb640 1 -- 172.21.15.44:0/3905643959 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c200b7920 con 0x55c10a1de4f0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 10 client.16377 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c10a0fd8e0 con 0x55c10a1de4f0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c11ffb640 1 -- 172.21.15.44:0/3905643959 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c24015070 con 0x55c10a1de4f0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c11ffb640 1 -- 172.21.15.44:0/3905643959 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3c200c3150 con 0x55c10a1de4f0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 10 client.16377 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 20 client.16377 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a2458b0 0x55c10a247ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 4 client.16377 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.309+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c109e2b300 con 0x55c10a2458b0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.310+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a2458b0 0x55c10a247ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.310+0000 7f3c2e1ce640 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a2458b0 0x55c10a247ca0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf7c8cd2b63a075 server_cookie=e57503da34cf6884 in_seq=0 out_seq=0 2024-09-13T17:29:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.310+0000 7f3c11ffb640 10 client.16377 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c11ffb640 1 -- 172.21.15.44:0/3905643959 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c109e2b300 con 0x55c10a2458b0 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c11ffb640 10 client.16377 handle_command_reply: tid=0 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 client.16377 shutdown 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 msgr2=0x55c10a1e1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 0x55c10a1e1e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3c200b7710 tx=0x7f3c2008cf50 comp rx=0 tx=0).stop 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 shutdown_connections 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c10a2458b0 0x55c10a247ca0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c10a1de4f0 0x55c10a1e1e90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c10a1d3c50 0x55c10a1e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c11ffb640 0 client.16377 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 --2- 172.21.15.44:0/3905643959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10a1d3330 0x55c10a1e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 >> 172.21.15.44:0/3905643959 conn(0x55c10a1cf8c0 msgr2=0x55c10a1d6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 shutdown_connections 2024-09-13T17:29:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 1 -- 172.21.15.44:0/3905643959 wait complete. 2024-09-13T17:29:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.323+0000 7f3c33230740 20 client.16377 trim_cache size 0 max 16384 2024-09-13T17:29:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:29:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:29:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:29:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3c140a1200 msgr2=0x7f3c140a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3c140a1200 0x7f3c140a36c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c281998c0 tx=0x7f3c1800bac0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 msgr2=0x7f3c2819d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 0x7f3c2819d100 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3c20004af0 tx=0x7f3c20030ec0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 shutdown_connections 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3c140a1200 0x7f3c140a36c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2819cc50 0x7f3c2819d100 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c28103f70 0x7f3c28198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 --2- 172.21.15.44:0/3654774848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c280fee20 0x7f3c28198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 >> 172.21.15.44:0/3654774848 conn(0x7f3c280fa8f0 msgr2=0x7f3c280fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 shutdown_connections 2024-09-13T17:29:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:04.324+0000 7f3c30432640 1 -- 172.21.15.44:0/3654774848 wait complete. 2024-09-13T17:29:04.499 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:04 smithi070 ceph-mon[34502]: pgmap v73: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:29:04.499 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:04 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:29:04.523 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:04 smithi100 ceph-mon[35055]: pgmap v73: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:29:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:04 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:29:04.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:04 smithi044 ceph-mon[31175]: pgmap v73: 129 pgs: 129 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-13T17:29:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:04 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:29:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:06 smithi100 ceph-mon[35055]: pgmap v74: 129 pgs: 129 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 KiB/s wr, 33 op/s 2024-09-13T17:29:06.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:06 smithi044 ceph-mon[31175]: pgmap v74: 129 pgs: 129 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 KiB/s wr, 33 op/s 2024-09-13T17:29:06.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:06 smithi070 ceph-mon[34502]: pgmap v74: 129 pgs: 129 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 KiB/s wr, 33 op/s 2024-09-13T17:29:08.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:08 smithi100 ceph-mon[35055]: pgmap v75: 129 pgs: 129 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-09-13T17:29:08.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:08 smithi044 ceph-mon[31175]: pgmap v75: 129 pgs: 129 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-09-13T17:29:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:08 smithi070 ceph-mon[34502]: pgmap v75: 129 pgs: 129 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-09-13T17:29:10.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:10 smithi100 ceph-mon[35055]: pgmap v76: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 56 KiB/s wr, 34 op/s 2024-09-13T17:29:10.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:10 smithi044 ceph-mon[31175]: pgmap v76: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 56 KiB/s wr, 34 op/s 2024-09-13T17:29:10.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:10 smithi070 ceph-mon[34502]: pgmap v76: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 56 KiB/s wr, 34 op/s 2024-09-13T17:29:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:12 smithi100 ceph-mon[35055]: pgmap v77: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-09-13T17:29:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:12 smithi044 ceph-mon[31175]: pgmap v77: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-09-13T17:29:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:12 smithi070 ceph-mon[34502]: pgmap v77: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-09-13T17:29:14.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:14 smithi100 ceph-mon[35055]: pgmap v78: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-09-13T17:29:14.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:14 smithi044 ceph-mon[31175]: pgmap v78: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-09-13T17:29:14.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:14 smithi070 ceph-mon[34502]: pgmap v78: 129 pgs: 129 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-09-13T17:29:16.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:16 smithi100 ceph-mon[35055]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 34 op/s 2024-09-13T17:29:16.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:16 smithi044 ceph-mon[31175]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 34 op/s 2024-09-13T17:29:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:16 smithi070 ceph-mon[34502]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 34 op/s 2024-09-13T17:29:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3563402617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84105040 msgr2=0x7fcb8410cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3563402617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84105040 0x7fcb8410cb90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009cd0 tx=0x7fcb7c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3563402617 shutdown_connections 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3563402617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84105040 0x7fcb8410cb90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3563402617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb841008a0 0x7fcb841048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3563402617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb840ffef0 0x7fcb841002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3563402617 >> 172.21.15.44:0/3563402617 conn(0x7fcb840fb960 msgr2=0x7fcb840fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3563402617 shutdown_connections 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.423+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3563402617 wait complete. 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 Processor -- start 2024-09-13T17:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 -- start start 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb840ffef0 0x7fcb840727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb84105040 0x7fcb84072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 0x7fcb841a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841a4940 con 0x7fcb84073230 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841a4ab0 con 0x7fcb84105040 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb8a71f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841a4c70 con 0x7fcb840ffef0 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb88cbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 0x7fcb841a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb88cbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 0x7fcb841a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44082/0 (socket says 172.21.15.44:44082) 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb84105040 0x7fcb84072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb88cbc640 1 -- 172.21.15.44:0/3138860293 learned_addr learned my addr 172.21.15.44:0/3138860293 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.424+0000 7fcb83fff640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb840ffef0 0x7fcb840727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb88cbc640 1 -- 172.21.15.44:0/3138860293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb840ffef0 msgr2=0x7fcb840727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb88cbc640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb840ffef0 0x7fcb840727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb88cbc640 1 -- 172.21.15.44:0/3138860293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb84105040 msgr2=0x7fcb84072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb88cbc640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb84105040 0x7fcb84072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb88cbc640 1 -- 172.21.15.44:0/3138860293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c009900 con 0x7fcb84073230 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb88cbc640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 0x7fcb841a4370 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c004af0 tx=0x7fcb7c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb817fa640 1 -- 172.21.15.44:0/3138860293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c003970 con 0x7fcb84073230 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb817fa640 1 -- 172.21.15.44:0/3138860293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb7c004040 con 0x7fcb84073230 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb817fa640 1 -- 172.21.15.44:0/3138860293 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c045650 con 0x7fcb84073230 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb841a4ef0 con 0x7fcb84073230 2024-09-13T17:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.425+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb841a53c0 con 0x7fcb84073230 2024-09-13T17:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb817fa640 1 -- 172.21.15.44:0/3138860293 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb7c00a040 con 0x7fcb84073230 2024-09-13T17:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 Processor -- start 2024-09-13T17:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 -- start start 2024-09-13T17:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c3f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c49fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 0x55af20c4c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20baac30 con 0x55af20c4a4f0 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20ba83c0 con 0x55af20c3fc50 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb8d440740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20ba3cb0 con 0x55af20c3f330 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 0x55af20c4c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 0x55af20c4c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44088/0 (socket says 172.21.15.44:44088) 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb837fe640 1 -- 172.21.15.44:0/3110795992 learned_addr learned my addr 172.21.15.44:0/3110795992 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.428+0000 7fcb83fff640 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c49fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c3f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb817fa640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb6c0a12b0 0x7fcb6c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 -- 172.21.15.44:0/3110795992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 msgr2=0x55af20c3f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c3f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 -- 172.21.15.44:0/3110795992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 msgr2=0x55af20c49fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c49fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 -- 172.21.15.44:0/3110795992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af20b4a910 con 0x55af20c4a4f0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb88cbc640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb6c0a12b0 0x7fcb6c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb817fa640 1 -- 172.21.15.44:0/3138860293 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcb7c0ee2b0 con 0x7fcb84073230 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 0x55af20c4c8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcb78012700 tx=0x7fcb7800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c78b3c5bf04a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb6a7fc640 1 -- 172.21.15.44:0/3110795992 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78012c80 con 0x55af20c4a4f0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb6a7fc640 1 -- 172.21.15.44:0/3110795992 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb780044c0 con 0x55af20c4a4f0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.429+0000 7fcb6a7fc640 1 -- 172.21.15.44:0/3110795992 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb780035f0 con 0x55af20c4a4f0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 -- 172.21.15.44:0/3110795992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 msgr2=0x55af20c4c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 0x55af20c4c8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcb78012700 tx=0x7fcb7800b6e0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb88cbc640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb6c0a12b0 0x7fcb6c0a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0ebce0 tx=0x7fcb7c08a410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 -- 172.21.15.44:0/3110795992 shutdown_connections 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c4a4f0 0x55af20c4c8e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c49fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3110795992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 -- 172.21.15.44:0/3110795992 >> 172.21.15.44:0/3110795992 conn(0x55af20c3b8c0 msgr2=0x55af20c3bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 -- 172.21.15.44:0/3110795992 shutdown_connections 2024-09-13T17:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 -- 172.21.15.44:0/3110795992 wait complete. 2024-09-13T17:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.430+0000 7fcb8d440740 1 -- start start 2024-09-13T17:29:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 Processor -- start 2024-09-13T17:29:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c49f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c47200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c47810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20b61e80 con 0x55af20c3fc50 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20b61b20 con 0x55af20c4a4f0 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20b27260 con 0x55af20c3f330 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb8d440740 10 client.0 _ll_register_callbacks cb 0x55af20b5cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c47200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c47200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44094/0 (socket says 172.21.15.44:44094) 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 -- 172.21.15.44:0/659207394 learned_addr learned my addr 172.21.15.44:0/659207394 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb837fe640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c49f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb837fe640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c47810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 -- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 msgr2=0x55af20c49f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c49f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 -- 172.21.15.44:0/659207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 msgr2=0x55af20c47810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c47810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb83fff640 1 -- 172.21.15.44:0/659207394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7801a040 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.431+0000 7fcb837fe640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c47810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb83fff640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c47200 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400bad0 tx=0x7fcb7400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749496aa3c0e8c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb637fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb637fe640 1 -- 172.21.15.44:0/659207394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74016070 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb637fe640 1 -- 172.21.15.44:0/659207394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcb74005020 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af20bcc680 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb637fe640 1 -- 172.21.15.44:0/659207394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb74003e80 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb8d440740 10 client.16389 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55af20ba2330 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb637fe640 1 -- 172.21.15.44:0/659207394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af20bcc680 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb637fe640 1 -- 172.21.15.44:0/659207394 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb7401b470 con 0x55af20c3fc50 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb8d440740 10 client.16389 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb8d440740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.432+0000 7fcb8d440740 20 client.16389 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.433+0000 7fcb8d440740 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20c59fd0 0x55af20c5c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.433+0000 7fcb8d440740 4 client.16389 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.433+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af20b975d0 con 0x55af20c59fd0 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.433+0000 7fcb837fe640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20c59fd0 0x55af20c5c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.433+0000 7fcb637fe640 10 client.16389 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.433+0000 7fcb837fe640 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20c59fd0 0x55af20c5c3f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3b76698cc9fda5 server_cookie=60eff87f51887d63 in_seq=0 out_seq=0 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb637fe640 1 -- 172.21.15.44:0/659207394 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55af20b975d0 con 0x55af20c59fd0 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb637fe640 10 client.16389 handle_command_reply: tid=0 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 client.16389 shutdown 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 msgr2=0x55af20c47200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c47200 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400bad0 tx=0x7fcb7400bed0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 shutdown_connections 2024-09-13T17:29:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20c59fd0 0x55af20c5c3f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c47810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb637fe640 0 client.16389 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c47200 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 --2- 172.21.15.44:0/659207394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c49f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 >> 172.21.15.44:0/659207394 conn(0x55af20c3b8c0 msgr2=0x55af20c3ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 shutdown_connections 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 1 -- 172.21.15.44:0/659207394 wait complete. 2024-09-13T17:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.434+0000 7fcb8d440740 20 client.16389 trim_cache size 0 max 16384 2024-09-13T17:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 Processor -- start 2024-09-13T17:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 -- start start 2024-09-13T17:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 0x55af20c3b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c4a4f0 0x55af20c45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20c4cfa0 con 0x55af20c3f330 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20c402a0 con 0x55af20c3fc50 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb8d440740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20c3ca10 con 0x55af20c4a4f0 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.447+0000 7fcb837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 0x55af20c3b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 0x55af20c3b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44104/0 (socket says 172.21.15.44:44104) 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 -- 172.21.15.44:0/3872098313 learned_addr learned my addr 172.21.15.44:0/3872098313 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb83fff640 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c4a4f0 0x55af20c45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 -- 172.21.15.44:0/3872098313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c4a4f0 msgr2=0x55af20c45ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c4a4f0 0x55af20c45ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 -- 172.21.15.44:0/3872098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 msgr2=0x55af20c45ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c45ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 -- 172.21.15.44:0/3872098313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af20b44300 con 0x55af20c3f330 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb837fe640 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 0x55af20c3b300 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcb78012700 tx=0x7fcb78014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad017ac3c478e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb6affd640 1 -- 172.21.15.44:0/3872098313 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78023040 con 0x55af20c3f330 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb6affd640 1 -- 172.21.15.44:0/3872098313 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb7802b830 con 0x55af20c3f330 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.448+0000 7fcb6affd640 1 -- 172.21.15.44:0/3872098313 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78005290 con 0x55af20c3f330 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 -- 172.21.15.44:0/3872098313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 msgr2=0x55af20c3b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 0x55af20c3b300 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcb78012700 tx=0x7fcb78014040 comp rx=0 tx=0).stop 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 -- 172.21.15.44:0/3872098313 shutdown_connections 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c4a4f0 0x55af20c45ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c3fc50 0x55af20c45ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 --2- 172.21.15.44:0/3872098313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3f330 0x55af20c3b300 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 -- 172.21.15.44:0/3872098313 >> 172.21.15.44:0/3872098313 conn(0x55af20c3b8c0 msgr2=0x55af20c55ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 -- 172.21.15.44:0/3872098313 shutdown_connections 2024-09-13T17:29:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 -- 172.21.15.44:0/3872098313 wait complete. 2024-09-13T17:29:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 -- start start 2024-09-13T17:29:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.449+0000 7fcb8d440740 1 Processor -- start 2024-09-13T17:29:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c51290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20c4e200 con 0x55af20c3fc50 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20c4e370 con 0x55af20c4a4f0 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af20c4e530 con 0x55af20c3f330 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb8d440740 10 client.0 _ll_register_callbacks cb 0x7fcb7800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c4d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44106/0 (socket says 172.21.15.44:44106) 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 -- 172.21.15.44:0/1744023174 learned_addr learned my addr 172.21.15.44:0/1744023174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb837fe640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb837fe640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c51290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 -- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 msgr2=0x55af20c51290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c51290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 -- 172.21.15.44:0/1744023174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 msgr2=0x55af20c4dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c4dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 -- 172.21.15.44:0/1744023174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af20b4a910 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb837fe640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c51290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb83fff640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c4d780 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcb7401b7d0 tx=0x7fcb7400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9f04cf916ef608 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.450+0000 7fcb637fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb637fe640 1 -- 172.21.15.44:0/1744023174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74016040 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb637fe640 1 -- 172.21.15.44:0/1744023174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcb74000e40 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb780044c0 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb637fe640 1 -- 172.21.15.44:0/1744023174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb7401bc00 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 10 client.16395 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55af20b698e0 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb637fe640 1 -- 172.21.15.44:0/1744023174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb780044c0 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb637fe640 1 -- 172.21.15.44:0/1744023174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb7402e020 con 0x55af20c3fc50 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 20 client.16395 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20cb18b0 0x55af20cb3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 4 client.16395 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.451+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af20897300 con 0x55af20cb18b0 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.452+0000 7fcb837fe640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20cb18b0 0x55af20cb3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.452+0000 7fcb837fe640 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20cb18b0 0x55af20cb3cd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef87e630c8176500 server_cookie=24323c222b4b7041 in_seq=0 out_seq=0 2024-09-13T17:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.452+0000 7fcb637fe640 10 client.16395 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.465+0000 7fcb637fe640 1 -- 172.21.15.44:0/1744023174 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af20897300 con 0x55af20cb18b0 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.465+0000 7fcb637fe640 10 client.16395 handle_command_reply: tid=0 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.465+0000 7fcb8d440740 1 client.16395 shutdown 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 msgr2=0x55af20c4d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c4d780 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcb7401b7d0 tx=0x7fcb7400b5d0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 shutdown_connections 2024-09-13T17:29:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af20cb18b0 0x55af20cb3cd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af20c4a4f0 0x55af20c4dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af20c3fc50 0x55af20c4d780 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 --2- 172.21.15.44:0/1744023174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af20c3f330 0x55af20c51290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb637fe640 0 client.16395 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 >> 172.21.15.44:0/1744023174 conn(0x55af20c3b8c0 msgr2=0x55af20c55ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 shutdown_connections 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 1 -- 172.21.15.44:0/1744023174 wait complete. 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8d440740 20 client.16395 trim_cache size 0 max 16384 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:29:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:29:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:29:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb6c0a12b0 msgr2=0x7fcb6c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb6c0a12b0 0x7fcb6c0a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0ebce0 tx=0x7fcb7c08a410 comp rx=0 tx=0).stop 2024-09-13T17:29:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 msgr2=0x7fcb841a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.466+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 0x7fcb841a4370 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c004af0 tx=0x7fcb7c030ec0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 shutdown_connections 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb6c0a12b0 0x7fcb6c0a3770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb84073230 0x7fcb841a4370 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb84105040 0x7fcb84072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 --2- 172.21.15.44:0/3138860293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb840ffef0 0x7fcb840727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 >> 172.21.15.44:0/3138860293 conn(0x7fcb840fb960 msgr2=0x7fcb840fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 shutdown_connections 2024-09-13T17:29:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:17.467+0000 7fcb8a71f640 1 -- 172.21.15.44:0/3138860293 wait complete. 2024-09-13T17:29:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:18 smithi100 ceph-mon[35055]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-13T17:29:18.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:18 smithi044 ceph-mon[31175]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-13T17:29:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:18 smithi070 ceph-mon[34502]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-13T17:29:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:20 smithi100 ceph-mon[35055]: pgmap v81: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-09-13T17:29:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:20 smithi044 ceph-mon[31175]: pgmap v81: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-09-13T17:29:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:20 smithi070 ceph-mon[34502]: pgmap v81: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-09-13T17:29:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:22 smithi100 ceph-mon[35055]: pgmap v82: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:22 smithi044 ceph-mon[31175]: pgmap v82: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:22 smithi070 ceph-mon[34502]: pgmap v82: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:24 smithi100 ceph-mon[35055]: pgmap v83: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:24 smithi044 ceph-mon[31175]: pgmap v83: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:24 smithi070 ceph-mon[34502]: pgmap v83: 129 pgs: 129 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:26 smithi100 ceph-mon[35055]: pgmap v84: 129 pgs: 129 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-13T17:29:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:26 smithi044 ceph-mon[31175]: pgmap v84: 129 pgs: 129 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-13T17:29:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:26 smithi070 ceph-mon[34502]: pgmap v84: 129 pgs: 129 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-13T17:29:28.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:28 smithi100 ceph-mon[35055]: pgmap v85: 129 pgs: 129 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:28 smithi044 ceph-mon[31175]: pgmap v85: 129 pgs: 129 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:28 smithi070 ceph-mon[34502]: pgmap v85: 129 pgs: 129 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-13T17:29:29.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 -- 172.21.15.44:0/3070041802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 msgr2=0x7fc4180ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/3070041802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4180ff020 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc414009930 tx=0x7fc41402a420 comp rx=0 tx=0).stop 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 -- 172.21.15.44:0/3070041802 shutdown_connections 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/3070041802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ff5f0 0x7fc418103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/3070041802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 0x7fc418104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/3070041802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4180ff020 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 -- 172.21.15.44:0/3070041802 >> 172.21.15.44:0/3070041802 conn(0x7fc4180fa930 msgr2=0x7fc4180fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 -- 172.21.15.44:0/3070041802 shutdown_connections 2024-09-13T17:29:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.940+0000 7fc41ffbc640 1 -- 172.21.15.44:0/3070041802 wait complete. 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 Processor -- start 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 -- start start 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4181a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ff5f0 0x7fc4181a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 0x7fc41819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819cc00 con 0x7fc4180fec40 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819cd70 con 0x7fc4181041c0 2024-09-13T17:29:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41ffbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819cf30 con 0x7fc4180ff5f0 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.941+0000 7fc41dd58640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4181a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4181a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39576/0 (socket says 172.21.15.44:39576) 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 -- 172.21.15.44:0/4000214969 learned_addr learned my addr 172.21.15.44:0/4000214969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41d557640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ff5f0 0x7fc4181a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41e559640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 0x7fc41819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 -- 172.21.15.44:0/4000214969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ff5f0 msgr2=0x7fc4181a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ff5f0 0x7fc4181a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 -- 172.21.15.44:0/4000214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 msgr2=0x7fc41819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 0x7fc41819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 -- 172.21.15.44:0/4000214969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414009560 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41e559640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 0x7fc41819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41dd58640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4181a2500 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc414005ab0 tx=0x7fc414032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc40effd640 1 -- 172.21.15.44:0/4000214969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414032e10 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc40effd640 1 -- 172.21.15.44:0/4000214969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4140046c0 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41819d1b0 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc40effd640 1 -- 172.21.15.44:0/4000214969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414003850 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.942+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4181a94f0 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.945+0000 7fc40effd640 1 -- 172.21.15.44:0/4000214969 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc414004830 con 0x7fc4180fec40 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.945+0000 7fc422dbe740 1 Processor -- start 2024-09-13T17:29:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.945+0000 7fc422dbe740 1 -- start start 2024-09-13T17:29:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.945+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d6f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d79a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc422dbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304cd9f20 con 0x55a304d79fd0 2024-09-13T17:29:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc422dbe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304cd76b0 con 0x55a304d6f730 2024-09-13T17:29:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc422dbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304cd2fa0 con 0x55a304d6ee10 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39580/0 (socket says 172.21.15.44:39580) 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 -- 172.21.15.44:0/1598001871 learned_addr learned my addr 172.21.15.44:0/1598001871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc40effd640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc4040a1280 0x7fc4040a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41e559640 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d6f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d79a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 -- 172.21.15.44:0/1598001871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 msgr2=0x55a304d6f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d6f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 -- 172.21.15.44:0/1598001871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 msgr2=0x55a304d79a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d79a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41e559640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc4040a1280 0x7fc4040a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 -- 172.21.15.44:0/1598001871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a304c8d840 con 0x55a304d79fd0 2024-09-13T17:29:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc40effd640 1 -- 172.21.15.44:0/4000214969 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc4140edc20 con 0x7fc4180fec40 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d79a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.946+0000 7fc41d557640 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7c3c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc40800bcb0 tx=0x7fc4080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657440afd0b9456 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc403fff640 1 -- 172.21.15.44:0/1598001871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408004030 con 0x55a304d79fd0 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc41e559640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc4040a1280 0x7fc4040a3740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc410005fd0 tx=0x7fc410004ab0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc403fff640 1 -- 172.21.15.44:0/1598001871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc408015070 con 0x55a304d79fd0 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc403fff640 1 -- 172.21.15.44:0/1598001871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408011440 con 0x55a304d79fd0 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 -- 172.21.15.44:0/1598001871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 msgr2=0x55a304d7c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7c3c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc40800bcb0 tx=0x7fc4080107a0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 -- 172.21.15.44:0/1598001871 shutdown_connections 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7c3c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d79a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1598001871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d6f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.947+0000 7fc422dbe740 1 -- 172.21.15.44:0/1598001871 >> 172.21.15.44:0/1598001871 conn(0x55a304d6b380 msgr2=0x55a304d6b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 -- 172.21.15.44:0/1598001871 shutdown_connections 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 -- 172.21.15.44:0/1598001871 wait complete. 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 -- start start 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 Processor -- start 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6ee10 0x55a304d79950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 0x55a304d76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d79fd0 0x55a304d771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304c742a0 con 0x55a304d6f730 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3049ab430 con 0x55a304d6ee10 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304c56680 con 0x55a304d79fd0 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.948+0000 7fc422dbe740 10 client.0 _ll_register_callbacks cb 0x55a304ccc610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 0x55a304d76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 0x55a304d76c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39596/0 (socket says 172.21.15.44:39596) 2024-09-13T17:29:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 -- 172.21.15.44:0/1275274370 learned_addr learned my addr 172.21.15.44:0/1275274370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6ee10 0x55a304d79950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41e559640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d79fd0 0x55a304d771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 -- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d79fd0 msgr2=0x55a304d771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d79fd0 0x55a304d771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 -- 172.21.15.44:0/1275274370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6ee10 msgr2=0x55a304d79950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6ee10 0x55a304d79950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 -- 172.21.15.44:0/1275274370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40800b8e0 con 0x55a304d6f730 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41e559640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d79fd0 0x55a304d771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc41d557640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 0x55a304d76c40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc408000c00 tx=0x7fc408003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc761c536adc99f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc4017fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc4017fa640 1 -- 172.21.15.44:0/1275274370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408021070 con 0x55a304d6f730 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc4017fa640 1 -- 172.21.15.44:0/1275274370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc408017070 con 0x55a304d6f730 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3049c64e0 con 0x55a304d6f730 2024-09-13T17:29:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc4017fa640 1 -- 172.21.15.44:0/1275274370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc40800bee0 con 0x55a304d6f730 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc422dbe740 10 client.16407 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.949+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a304cd1620 con 0x55a304d6f730 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc4017fa640 1 -- 172.21.15.44:0/1275274370 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a3049c64e0 con 0x55a304d6f730 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc4017fa640 1 -- 172.21.15.44:0/1275274370 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc408003cd0 con 0x55a304d6f730 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc422dbe740 10 client.16407 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc422dbe740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc422dbe740 20 client.16407 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304d89c60 0x55a304d8c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc422dbe740 4 client.16407 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3049d77e0 con 0x55a304d89c60 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.950+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304d89c60 0x55a304d8c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.951+0000 7fc4017fa640 10 client.16407 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.951+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304d89c60 0x55a304d8c080 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f502609a8a22c99d server_cookie=e75b1b70ff64811d in_seq=0 out_seq=0 2024-09-13T17:29:29.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.951+0000 7fc4017fa640 1 -- 172.21.15.44:0/1275274370 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a3049d77e0 con 0x55a304d89c60 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.951+0000 7fc4017fa640 10 client.16407 handle_command_reply: tid=0 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.951+0000 7fc422dbe740 1 client.16407 shutdown 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 msgr2=0x55a304d76c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 0x55a304d76c40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc408000c00 tx=0x7fc408003820 comp rx=0 tx=0).stop 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 shutdown_connections 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304d89c60 0x55a304d8c080 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d79fd0 0x55a304d771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6f730 0x55a304d76c40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1275274370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6ee10 0x55a304d79950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc4017fa640 0 client.16407 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 >> 172.21.15.44:0/1275274370 conn(0x55a304d6b380 msgr2=0x55a304d71090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 shutdown_connections 2024-09-13T17:29:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 1 -- 172.21.15.44:0/1275274370 wait complete. 2024-09-13T17:29:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.952+0000 7fc422dbe740 20 client.16407 trim_cache size 0 max 16384 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.966+0000 7fc422dbe740 1 Processor -- start 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.966+0000 7fc422dbe740 1 -- start start 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 0x55a304d6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6f730 0x55a304d909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d79fd0 0x55a304d71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc422dbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304d89a90 con 0x55a304d6ee10 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc422dbe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304d6ec70 con 0x55a304d79fd0 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc422dbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304d6be70 con 0x55a304d6f730 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 0x55a304d6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 0x55a304d6ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39598/0 (socket says 172.21.15.44:39598) 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 -- 172.21.15.44:0/1449261114 learned_addr learned my addr 172.21.15.44:0/1449261114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41d557640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6f730 0x55a304d909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 -- 172.21.15.44:0/1449261114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6f730 msgr2=0x55a304d909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6f730 0x55a304d909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 -- 172.21.15.44:0/1449261114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d79fd0 msgr2=0x55a304d71ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41e559640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d79fd0 0x55a304d71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d79fd0 0x55a304d71ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 -- 172.21.15.44:0/1449261114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a304c73790 con 0x55a304d6ee10 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41d557640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6f730 0x55a304d909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41e559640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d79fd0 0x55a304d71ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc41dd58640 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 0x55a304d6ad00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc414009900 tx=0x7fc41404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9896101ce5c8a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc40cff9640 1 -- 172.21.15.44:0/1449261114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140c2e70 con 0x55a304d6ee10 2024-09-13T17:29:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc40cff9640 1 -- 172.21.15.44:0/1449261114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4140ba9f0 con 0x55a304d6ee10 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.967+0000 7fc40cff9640 1 -- 172.21.15.44:0/1449261114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140c29e0 con 0x55a304d6ee10 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 -- 172.21.15.44:0/1449261114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 msgr2=0x55a304d6ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 0x55a304d6ad00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc414009900 tx=0x7fc41404c000 comp rx=0 tx=0).stop 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 -- 172.21.15.44:0/1449261114 shutdown_connections 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6f730 0x55a304d909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d79fd0 0x55a304d71ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 --2- 172.21.15.44:0/1449261114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d6ee10 0x55a304d6ad00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 -- 172.21.15.44:0/1449261114 >> 172.21.15.44:0/1449261114 conn(0x55a304d6b380 msgr2=0x55a304d6b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 -- 172.21.15.44:0/1449261114 shutdown_connections 2024-09-13T17:29:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 -- 172.21.15.44:0/1449261114 wait complete. 2024-09-13T17:29:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 -- start start 2024-09-13T17:29:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.968+0000 7fc422dbe740 1 Processor -- start 2024-09-13T17:29:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d80e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304d7dd70 con 0x55a304d79fd0 2024-09-13T17:29:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304d7dee0 con 0x55a304d6f730 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304d7e0a0 con 0x55a304d6ee10 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc422dbe740 10 client.0 _ll_register_callbacks cb 0x7fc4100096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41e559640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41e559640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39600/0 (socket says 172.21.15.44:39600) 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41d557640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41e559640 1 -- 172.21.15.44:0/710444836 learned_addr learned my addr 172.21.15.44:0/710444836 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41dd58640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d80e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41d557640 1 -- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 msgr2=0x55a304d80e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41d557640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d80e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41d557640 1 -- 172.21.15.44:0/710444836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 msgr2=0x55a304d7d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41d557640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41d557640 1 -- 172.21.15.44:0/710444836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4181a94f0 con 0x55a304d6f730 2024-09-13T17:29:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.969+0000 7fc41e559640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc41d557640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d7d2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc408004090 tx=0x7fc408003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec97068bac30a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc4017fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc4017fa640 1 -- 172.21.15.44:0/710444836 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408021070 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc408015070 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc4017fa640 1 -- 172.21.15.44:0/710444836 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc408017040 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc4017fa640 1 -- 172.21.15.44:0/710444836 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc40800bb80 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 10 client.25919 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a304c98ea0 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc4017fa640 1 -- 172.21.15.44:0/710444836 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc408015070 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc4017fa640 1 -- 172.21.15.44:0/710444836 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc40802d040 con 0x55a304d6f730 2024-09-13T17:29:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 10 client.25919 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 20 client.25919 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304de10f0 0x55a304de34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 4 client.25919 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.970+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a304a2ae30 con 0x55a304de10f0 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc41dd58640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304de10f0 0x55a304de34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc41dd58640 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304de10f0 0x55a304de34e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5329cb546d5fa9d server_cookie=d333b6b203734e95 in_seq=0 out_seq=0 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc4017fa640 10 client.25919 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc4017fa640 1 -- 172.21.15.44:0/710444836 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a304a2ae30 con 0x55a304de10f0 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc4017fa640 10 client.25919 handle_command_reply: tid=0 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc422dbe740 1 client.25919 shutdown 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 msgr2=0x55a304d7d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.971+0000 7fc422dbe740 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d7d2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc408004090 tx=0x7fc408003910 comp rx=0 tx=0).stop 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 shutdown_connections 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a304de10f0 0x55a304de34e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a304d79fd0 0x55a304d7d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc4017fa640 0 client.25919 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a304d6f730 0x55a304d7d2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 --2- 172.21.15.44:0/710444836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a304d6ee10 0x55a304d80e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 >> 172.21.15.44:0/710444836 conn(0x55a304d6b380 msgr2=0x55a304d909e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 shutdown_connections 2024-09-13T17:29:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 1 -- 172.21.15.44:0/710444836 wait complete. 2024-09-13T17:29:29.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc422dbe740 20 client.25919 trim_cache size 0 max 16384 2024-09-13T17:29:29.982 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:29:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:29:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:29:29.982 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc4040a1280 msgr2=0x7fc4040a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc4040a1280 0x7fc4040a3740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc410005fd0 tx=0x7fc410004ab0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 msgr2=0x7fc4181a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4181a2500 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc414005ab0 tx=0x7fc414032c20 comp rx=0 tx=0).stop 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 shutdown_connections 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc4040a1280 0x7fc4040a3740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ff5f0 0x7fc4181a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4181041c0 0x7fc41819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 --2- 172.21.15.44:0/4000214969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4180fec40 0x7fc4181a2500 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 >> 172.21.15.44:0/4000214969 conn(0x7fc4180fa930 msgr2=0x7fc4180fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:29.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 shutdown_connections 2024-09-13T17:29:29.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:29.972+0000 7fc41ffbc640 1 -- 172.21.15.44:0/4000214969 wait complete. 2024-09-13T17:29:29.985 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:29:29.985 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:29:30.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 -- 172.21.15.44:0/2739566857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9500ff830 msgr2=0x7fd950103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 --2- 172.21.15.44:0/2739566857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9500ff830 0x7fd950103870 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd944009930 tx=0x7fd94402a420 comp rx=0 tx=0).stop 2024-09-13T17:29:30.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 -- 172.21.15.44:0/2739566857 shutdown_connections 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 --2- 172.21.15.44:0/2739566857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 0x7fd95010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 --2- 172.21.15.44:0/2739566857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9500ff830 0x7fd950103870 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 --2- 172.21.15.44:0/2739566857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9500fee80 0x7fd9500ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 -- 172.21.15.44:0/2739566857 >> 172.21.15.44:0/2739566857 conn(0x7fd9500fa930 msgr2=0x7fd9500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 -- 172.21.15.44:0/2739566857 shutdown_connections 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 -- 172.21.15.44:0/2739566857 wait complete. 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.105+0000 7fd955c87640 1 Processor -- start 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 -- start start 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9500fee80 0x7fd95010b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 0x7fd9501078d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 0x7fd9501082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501a8e10 con 0x7fd950107e10 2024-09-13T17:29:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501a8f50 con 0x7fd9500fee80 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd955c87640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501a90e0 con 0x7fd950103fd0 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 0x7fd9501082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 0x7fd9501082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39616/0 (socket says 172.21.15.44:39616) 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 -- 172.21.15.44:0/840747479 learned_addr learned my addr 172.21.15.44:0/840747479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94effd640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 0x7fd9501078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94f7fe640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9500fee80 0x7fd95010b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 -- 172.21.15.44:0/840747479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 msgr2=0x7fd9501078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 0x7fd9501078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 -- 172.21.15.44:0/840747479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9500fee80 msgr2=0x7fd95010b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9500fee80 0x7fd95010b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 -- 172.21.15.44:0/840747479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944009560 con 0x7fd950107e10 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94effd640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 0x7fd9501078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:30.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.106+0000 7fd94ffff640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 0x7fd9501082c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd94800eb40 tx=0x7fd94800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.107+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948004b20 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.107+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9501a93c0 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.107+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd948004c80 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.107+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948010690 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.107+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9501a98a0 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.108+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950100350 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.109+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd948005020 con 0x7fd950107e10 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.109+0000 7fd94cff9640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9380a1200 0x7fd9380a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.110+0000 7fd94f7fe640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9380a1200 0x7fd9380a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.110+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd948014070 con 0x7fd950107e10 2024-09-13T17:29:30.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.110+0000 7fd94f7fe640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9380a1200 0x7fd9380a36c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd940005fd0 tx=0x7fd94000b590 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.113+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd9480967d0 con 0x7fd950107e10 2024-09-13T17:29:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.322+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd950108e00 con 0x7fd950107e10 2024-09-13T17:29:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.323+0000 7fd94cff9640 1 -- 172.21.15.44:0/840747479 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fd948099df0 con 0x7fd950107e10 2024-09-13T17:29:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:29:30.325 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:29:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:29:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9380a1200 msgr2=0x7fd9380a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9380a1200 0x7fd9380a36c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd940005fd0 tx=0x7fd94000b590 comp rx=0 tx=0).stop 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 msgr2=0x7fd9501082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 0x7fd9501082c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd94800eb40 tx=0x7fd94800c2d0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 shutdown_connections 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd9380a1200 0x7fd9380a36c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd950103fd0 0x7fd9501078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd950107e10 0x7fd9501082c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 --2- 172.21.15.44:0/840747479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9500fee80 0x7fd95010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 >> 172.21.15.44:0/840747479 conn(0x7fd9500fa930 msgr2=0x7fd9500fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 shutdown_connections 2024-09-13T17:29:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.325+0000 7fd955c87640 1 -- 172.21.15.44:0/840747479 wait complete. 2024-09-13T17:29:30.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 -- 172.21.15.44:0/149349660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc0fee60 msgr2=0x7f26dc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 --2- 172.21.15.44:0/149349660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc0fee60 0x7f26dc0ff240 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009cd0 tx=0x7f26cc02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 -- 172.21.15.44:0/149349660 shutdown_connections 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 --2- 172.21.15.44:0/149349660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc103fb0 0x7f26dc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 --2- 172.21.15.44:0/149349660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc0ff810 0x7f26dc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 --2- 172.21.15.44:0/149349660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc0fee60 0x7f26dc0ff240 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 -- 172.21.15.44:0/149349660 >> 172.21.15.44:0/149349660 conn(0x7f26dc075c60 msgr2=0x7f26dc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 -- 172.21.15.44:0/149349660 shutdown_connections 2024-09-13T17:29:30.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 -- 172.21.15.44:0/149349660 wait complete. 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.441+0000 7f26e0a19640 1 Processor -- start 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 -- start start 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc0fee60 0x7f26dc1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 0x7f26dc196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc19aaf0 0x7f26dc19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19b4e0 con 0x7f26dc103fb0 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19b650 con 0x7f26dc0fee60 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26e0a19640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19b7c0 con 0x7f26dc19aaf0 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 0x7f26dc196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 0x7f26dc196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39632/0 (socket says 172.21.15.44:39632) 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 -- 172.21.15.44:0/902086525 learned_addr learned my addr 172.21.15.44:0/902086525 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26dad9d640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc19aaf0 0x7f26dc19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26da59c640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc0fee60 0x7f26dc1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 -- 172.21.15.44:0/902086525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc19aaf0 msgr2=0x7f26dc19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc19aaf0 0x7f26dc19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 -- 172.21.15.44:0/902086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc0fee60 msgr2=0x7f26dc1961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc0fee60 0x7f26dc1961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 -- 172.21.15.44:0/902086525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc009900 con 0x7f26dc103fb0 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.442+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 0x7f26dc196700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f26c4005fd0 tx=0x7f26c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.443+0000 7f26d37fe640 1 -- 172.21.15.44:0/902086525 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4009c30 con 0x7f26dc103fb0 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.443+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc1a2da0 con 0x7f26dc103fb0 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.443+0000 7f26d37fe640 1 -- 172.21.15.44:0/902086525 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26c40044b0 con 0x7f26dc103fb0 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.443+0000 7f26d37fe640 1 -- 172.21.15.44:0/902086525 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4003da0 con 0x7f26dc103fb0 2024-09-13T17:29:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.443+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc1a31f0 con 0x7f26dc103fb0 2024-09-13T17:29:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26d37fe640 1 -- 172.21.15.44:0/902086525 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f26c4004000 con 0x7f26dc103fb0 2024-09-13T17:29:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 Processor -- start 2024-09-13T17:29:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 -- start start 2024-09-13T17:29:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd494b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd49f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd3fecc0 con 0x559abd4950b0 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd3fc450 con 0x559abd494790 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.445+0000 7f26e381c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd3f7d40 con 0x559abd49f950 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d37fe640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26c80a1200 0x7f26c80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd49f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd49f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39642/0 (socket says 172.21.15.44:39642) 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 -- 172.21.15.44:0/2146555095 learned_addr learned my addr 172.21.15.44:0/2146555095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26dad9d640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26c80a1200 0x7f26c80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d37fe640 1 -- 172.21.15.44:0/902086525 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f26c40ccbc0 con 0x7f26dc103fb0 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26dad9d640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd494b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 -- 172.21.15.44:0/2146555095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 msgr2=0x559abd4a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 -- 172.21.15.44:0/2146555095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 msgr2=0x559abd494b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd494b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 -- 172.21.15.44:0/2146555095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559abd3bf620 con 0x559abd4950b0 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26dad9d640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd494b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26dad9d640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26c80a1200 0x7f26c80a36c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f26d40115e0 tx=0x7f26d4009880 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26da59c640 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd49f410 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ae00 tx=0x7f26cc005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d866f71b408884 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d0ff9640 1 -- 172.21.15.44:0/2146555095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0307a0 con 0x559abd4950b0 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d0ff9640 1 -- 172.21.15.44:0/2146555095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26cc040070 con 0x559abd4950b0 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.446+0000 7f26d0ff9640 1 -- 172.21.15.44:0/2146555095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc003b90 con 0x559abd4950b0 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 -- 172.21.15.44:0/2146555095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 msgr2=0x559abd49f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd49f410 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ae00 tx=0x7f26cc005fa0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 -- 172.21.15.44:0/2146555095 shutdown_connections 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd49f410 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 --2- 172.21.15.44:0/2146555095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd494b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 -- 172.21.15.44:0/2146555095 >> 172.21.15.44:0/2146555095 conn(0x559abd490d00 msgr2=0x559abd4910e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 -- 172.21.15.44:0/2146555095 shutdown_connections 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.447+0000 7f26e381c740 1 -- 172.21.15.44:0/2146555095 wait complete. 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 -- start start 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 Processor -- start 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd494790 0x559abd49ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd4950b0 0x559abd49f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 0x559abd49c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd3b6180 con 0x559abd49f950 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd0d0430 con 0x559abd4950b0 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd37b5e0 con 0x559abd494790 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26e381c740 10 client.0 _ll_register_callbacks cb 0x559abd3e7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 0x559abd49c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 0x559abd49c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39656/0 (socket says 172.21.15.44:39656) 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 -- 172.21.15.44:0/3459192297 learned_addr learned my addr 172.21.15.44:0/3459192297 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd4950b0 0x559abd49f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26da59c640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd494790 0x559abd49ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 -- 172.21.15.44:0/3459192297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd494790 msgr2=0x559abd49ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd494790 0x559abd49ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 -- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd4950b0 msgr2=0x559abd49f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd4950b0 0x559abd49f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26dad9d640 1 -- 172.21.15.44:0/3459192297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc10c340 con 0x559abd49f950 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.448+0000 7f26da59c640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd494790 0x559abd49ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26dad9d640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 0x559abd49c3b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004650 tx=0x7f26d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffd7370f1579ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26c1ffb640 1 -- 172.21.15.44:0/3459192297 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d401e980 con 0x559abd49f950 2024-09-13T17:29:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26c1ffb640 1 -- 172.21.15.44:0/3459192297 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f26d401eae0 con 0x559abd49f950 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559abd421b80 con 0x559abd49f950 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26c1ffb640 1 -- 172.21.15.44:0/3459192297 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d401eca0 con 0x559abd49f950 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 10 client.16431 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559abd3f63c0 con 0x559abd49f950 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26c1ffb640 1 -- 172.21.15.44:0/3459192297 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559abd421b80 con 0x559abd49f950 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26c1ffb640 1 -- 172.21.15.44:0/3459192297 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26d4004b40 con 0x559abd49f950 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 20 client.16431 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd4af220 0x559abd4b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 4 client.16431 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.449+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559abd3ea1f0 con 0x559abd4af220 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.450+0000 7f26da59c640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd4af220 0x559abd4b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.450+0000 7f26da59c640 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd4af220 0x559abd4b1640 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a86ac54351d1084f server_cookie=fa7f11636ea4c458 in_seq=0 out_seq=0 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.450+0000 7f26c1ffb640 10 client.16431 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26c1ffb640 1 -- 172.21.15.44:0/3459192297 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559abd3ea1f0 con 0x559abd4af220 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26c1ffb640 10 client.16431 handle_command_reply: tid=0 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 client.16431 shutdown 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 msgr2=0x559abd49c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 0x559abd49c3b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004650 tx=0x7f26d4004730 comp rx=0 tx=0).stop 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 shutdown_connections 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd4af220 0x559abd4b1640 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd49f950 0x559abd49c3b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd4950b0 0x559abd49f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 --2- 172.21.15.44:0/3459192297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd494790 0x559abd49ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26c1ffb640 0 client.16431 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 >> 172.21.15.44:0/3459192297 conn(0x559abd490d00 msgr2=0x559abd496a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 shutdown_connections 2024-09-13T17:29:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 1 -- 172.21.15.44:0/3459192297 wait complete. 2024-09-13T17:29:30.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.451+0000 7f26e381c740 20 client.16431 trim_cache size 0 max 16384 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 Processor -- start 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 -- start start 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd4b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd4a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd4919f0 con 0x559abd4950b0 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd491df0 con 0x559abd494790 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26e381c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd4af050 con 0x559abd49f950 2024-09-13T17:29:30.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd4a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd4a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39658/0 (socket says 172.21.15.44:39658) 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.462+0000 7f26d9d9b640 1 -- 172.21.15.44:0/3908473543 learned_addr learned my addr 172.21.15.44:0/3908473543 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26da59c640 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd4b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d9d9b640 1 -- 172.21.15.44:0/3908473543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 msgr2=0x559abd4a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d9d9b640 1 -- 172.21.15.44:0/3908473543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 msgr2=0x559abd4b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd4b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d9d9b640 1 -- 172.21.15.44:0/3908473543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559abd398770 con 0x559abd4950b0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd4a6620 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f26c40ca050 tx=0x7f26c40994f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb839cfb0d8f7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d17fa640 1 -- 172.21.15.44:0/3908473543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c409a0b0 con 0x559abd4950b0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d17fa640 1 -- 172.21.15.44:0/3908473543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26c40999b0 con 0x559abd4950b0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26d17fa640 1 -- 172.21.15.44:0/3908473543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c40b2210 con 0x559abd4950b0 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 -- 172.21.15.44:0/3908473543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 msgr2=0x559abd4a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd4a6620 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f26c40ca050 tx=0x7f26c40994f0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 -- 172.21.15.44:0/3908473543 shutdown_connections 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd49f950 0x559abd4a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd4950b0 0x559abd4a6620 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 --2- 172.21.15.44:0/3908473543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd494790 0x559abd4b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 -- 172.21.15.44:0/3908473543 >> 172.21.15.44:0/3908473543 conn(0x559abd490d00 msgr2=0x559abd4ab0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.463+0000 7f26e381c740 1 -- 172.21.15.44:0/3908473543 shutdown_connections 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 -- 172.21.15.44:0/3908473543 wait complete. 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 -- start start 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 Processor -- start 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 0x559abd4a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd4950b0 0x559abd4a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd49f950 0x559abd4a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd5057f0 con 0x559abd494790 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd505930 con 0x559abd49f950 2024-09-13T17:29:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559abd505af0 con 0x559abd4950b0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26e381c740 10 client.0 _ll_register_callbacks cb 0x7f26cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 0x559abd4a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 0x559abd4a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39672/0 (socket says 172.21.15.44:39672) 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 -- 172.21.15.44:0/2514281466 learned_addr learned my addr 172.21.15.44:0/2514281466 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26dad9d640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd49f950 0x559abd4a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd4950b0 0x559abd4a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 -- 172.21.15.44:0/2514281466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd4950b0 msgr2=0x559abd4a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd4950b0 0x559abd4a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 -- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd49f950 msgr2=0x559abd4a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd49f950 0x559abd4a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.464+0000 7f26da59c640 1 -- 172.21.15.44:0/2514281466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc1a31f0 con 0x559abd494790 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26da59c640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 0x559abd4a5280 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02aed0 tx=0x7f26cc031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f2f57b9fda7899 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26c1ffb640 1 -- 172.21.15.44:0/2514281466 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc04d040 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26c1ffb640 1 -- 172.21.15.44:0/2514281466 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f26cc042070 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26cc040070 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26c1ffb640 1 -- 172.21.15.44:0/2514281466 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc030ba0 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 10 client.16437 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559abd3bde80 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26c1ffb640 1 -- 172.21.15.44:0/2514281466 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26cc040070 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26c1ffb640 1 -- 172.21.15.44:0/2514281466 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26cc003ae0 con 0x559abd494790 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 10 client.16437 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 20 client.16437 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd507040 0x559abd509430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 4 client.16437 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.465+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559abd0eb4e0 con 0x559abd507040 2024-09-13T17:29:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd507040 0x559abd509430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26c1ffb640 10 client.16437 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:29:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26d9d9b640 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd507040 0x559abd509430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f16cbc53edd1360f server_cookie=84be0efb209d1942 in_seq=0 out_seq=0 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26c1ffb640 1 -- 172.21.15.44:0/2514281466 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559abd0eb4e0 con 0x559abd507040 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26c1ffb640 10 client.16437 handle_command_reply: tid=0 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 client.16437 shutdown 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 msgr2=0x559abd4a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 0x559abd4a5280 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02aed0 tx=0x7f26cc031c70 comp rx=0 tx=0).stop 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 shutdown_connections 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559abd507040 0x559abd509430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559abd4950b0 0x559abd4a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559abd49f950 0x559abd4a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26c1ffb640 0 client.16437 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:29:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.466+0000 7f26e381c740 1 --2- 172.21.15.44:0/2514281466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559abd494790 0x559abd4a5280 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 >> 172.21.15.44:0/2514281466 conn(0x559abd490d00 msgr2=0x559abd4ab0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 shutdown_connections 2024-09-13T17:29:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e381c740 1 -- 172.21.15.44:0/2514281466 wait complete. 2024-09-13T17:29:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e381c740 20 client.16437 trim_cache size 0 max 16384 2024-09-13T17:29:30.475 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26c80a1200 msgr2=0x7f26c80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26c80a1200 0x7f26c80a36c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f26d40115e0 tx=0x7f26d4009880 comp rx=0 tx=0).stop 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 msgr2=0x7f26dc196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 0x7f26dc196700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f26c4005fd0 tx=0x7f26c400dde0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 shutdown_connections 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26c80a1200 0x7f26c80a36c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc19aaf0 0x7f26dc19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26dc103fb0 0x7f26dc196700 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 --2- 172.21.15.44:0/902086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc0fee60 0x7f26dc1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 >> 172.21.15.44:0/902086525 conn(0x7f26dc075c60 msgr2=0x7f26dc0677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 shutdown_connections 2024-09-13T17:29:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.467+0000 7f26e0a19640 1 -- 172.21.15.44:0/902086525 wait complete. 2024-09-13T17:29:30.480 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:29:30.502 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:30 smithi044 ceph-mon[31175]: pgmap v86: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-09-13T17:29:30.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:30 smithi100 ceph-mon[35055]: pgmap v86: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-09-13T17:29:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 -- 172.21.15.44:0/2321099340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c1008c0 msgr2=0x7fb02c104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 --2- 172.21.15.44:0/2321099340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c1008c0 0x7fb02c104900 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009930 tx=0x7fb01c02a420 comp rx=0 tx=0).stop 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 -- 172.21.15.44:0/2321099340 shutdown_connections 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 --2- 172.21.15.44:0/2321099340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c105060 0x7fb02c10cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 --2- 172.21.15.44:0/2321099340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c1008c0 0x7fb02c104900 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 --2- 172.21.15.44:0/2321099340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb02c0fff10 0x7fb02c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 -- 172.21.15.44:0/2321099340 >> 172.21.15.44:0/2321099340 conn(0x7fb02c0fb9a0 msgr2=0x7fb02c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 -- 172.21.15.44:0/2321099340 shutdown_connections 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.556+0000 7fb033eb1640 1 -- 172.21.15.44:0/2321099340 wait complete. 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 Processor -- start 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 -- start start 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb02c0fff10 0x7fb02c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 0x7fb02c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c19aac0 0x7fb02c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19b4b0 con 0x7fb02c105060 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19b620 con 0x7fb02c0fff10 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb033eb1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19b790 con 0x7fb02c19aac0 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb03144c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 0x7fb02c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb03144c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 0x7fb02c196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39690/0 (socket says 172.21.15.44:39690) 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb03144c640 1 -- 172.21.15.44:0/3009996030 learned_addr learned my addr 172.21.15.44:0/3009996030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.557+0000 7fb03244e640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c19aac0 0x7fb02c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb031c4d640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb02c0fff10 0x7fb02c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03144c640 1 -- 172.21.15.44:0/3009996030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c19aac0 msgr2=0x7fb02c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03144c640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c19aac0 0x7fb02c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03144c640 1 -- 172.21.15.44:0/3009996030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb02c0fff10 msgr2=0x7fb02c1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03144c640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb02c0fff10 0x7fb02c1961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03144c640 1 -- 172.21.15.44:0/3009996030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c009560 con 0x7fb02c105060 2024-09-13T17:29:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03244e640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c19aac0 0x7fb02c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb03144c640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 0x7fb02c196720 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb01c005ab0 tx=0x7fb01c0329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb022ffd640 1 -- 172.21.15.44:0/3009996030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c032df0 con 0x7fb02c105060 2024-09-13T17:29:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c1a2d70 con 0x7fb02c105060 2024-09-13T17:29:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb022ffd640 1 -- 172.21.15.44:0/3009996030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb01c005490 con 0x7fb02c105060 2024-09-13T17:29:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb022ffd640 1 -- 172.21.15.44:0/3009996030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0035f0 con 0x7fb02c105060 2024-09-13T17:29:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.558+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c1a31a0 con 0x7fb02c105060 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.560+0000 7fb022ffd640 1 -- 172.21.15.44:0/3009996030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb01c0037d0 con 0x7fb02c105060 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 Processor -- start 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 -- start start 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 0x55f35c32e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32ec50 0x55f35c338fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c3394f0 0x55f35c33b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c299c30 con 0x55f35c32ec50 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c2973c0 con 0x55f35c3394f0 2024-09-13T17:29:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb036bd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c292cb0 con 0x55f35c32e330 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb031c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32ec50 0x55f35c338fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb03244e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 0x55f35c32e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb03144c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c3394f0 0x55f35c33b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb031c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32ec50 0x55f35c338fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39700/0 (socket says 172.21.15.44:39700) 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb03244e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 0x55f35c32e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:33984/0 (socket says 172.21.15.44:33984) 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.561+0000 7fb031c4d640 1 -- 172.21.15.44:0/886530977 learned_addr learned my addr 172.21.15.44:0/886530977 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb022ffd640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb0180a1200 0x7fb0180a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 -- 172.21.15.44:0/886530977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c3394f0 msgr2=0x55f35c33b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c3394f0 0x55f35c33b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 -- 172.21.15.44:0/886530977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32ec50 msgr2=0x55f35c338fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32ec50 0x55f35c338fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 -- 172.21.15.44:0/886530977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f35c239910 con 0x55f35c32e330 2024-09-13T17:29:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb022ffd640 1 -- 172.21.15.44:0/3009996030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb01c0edad0 con 0x7fb02c105060 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb0180a1200 0x7fb0180a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 0x55f35c32e710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb024005fd0 tx=0x7fb024011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2f60a55827f723 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb03244e640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb0180a1200 0x7fb0180a36c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb024023000 tx=0x7fb024024000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb017fff640 1 -- 172.21.15.44:0/886530977 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02401bad0 con 0x55f35c32e330 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb017fff640 1 -- 172.21.15.44:0/886530977 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02401bcf0 con 0x55f35c32e330 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.562+0000 7fb017fff640 1 -- 172.21.15.44:0/886530977 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02401a360 con 0x55f35c32e330 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 -- 172.21.15.44:0/886530977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 msgr2=0x55f35c32e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 0x55f35c32e710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb024005fd0 tx=0x7fb024011c90 comp rx=0 tx=0).stop 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 -- 172.21.15.44:0/886530977 shutdown_connections 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c3394f0 0x55f35c33b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32ec50 0x55f35c338fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 --2- 172.21.15.44:0/886530977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32e330 0x55f35c32e710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.563+0000 7fb036bd1740 1 -- 172.21.15.44:0/886530977 >> 172.21.15.44:0/886530977 conn(0x55f35c32a8c0 msgr2=0x55f35c32aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.564+0000 7fb036bd1740 1 -- 172.21.15.44:0/886530977 shutdown_connections 2024-09-13T17:29:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.564+0000 7fb036bd1740 1 -- 172.21.15.44:0/886530977 wait complete. 2024-09-13T17:29:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.564+0000 7fb036bd1740 1 -- start start 2024-09-13T17:29:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.564+0000 7fb036bd1740 1 Processor -- start 2024-09-13T17:29:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 0x55f35c338e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32ec50 0x55f35c336110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c3394f0 0x55f35c336720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c250e80 con 0x55f35c32e330 2024-09-13T17:29:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c250b20 con 0x55f35c32ec50 2024-09-13T17:29:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c216260 con 0x55f35c3394f0 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb036bd1740 10 client.0 _ll_register_callbacks cb 0x55f35c24bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 0x55f35c338e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 0x55f35c338e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39716/0 (socket says 172.21.15.44:39716) 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 -- 172.21.15.44:0/4248432745 learned_addr learned my addr 172.21.15.44:0/4248432745 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb03244e640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c3394f0 0x55f35c336720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb03144c640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32ec50 0x55f35c336110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 -- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c3394f0 msgr2=0x55f35c336720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c3394f0 0x55f35c336720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 -- 172.21.15.44:0/4248432745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32ec50 msgr2=0x55f35c336110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32ec50 0x55f35c336110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 -- 172.21.15.44:0/4248432745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb024011440 con 0x55f35c32e330 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb03144c640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32ec50 0x55f35c336110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb031c4d640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 0x55f35c338e90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb028012720 tx=0x7fb028012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ac0db344c29d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb0157fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.565+0000 7fb0157fa640 1 -- 172.21.15.44:0/4248432745 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028017070 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb0157fa640 1 -- 172.21.15.44:0/4248432745 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb028004050 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f35c2bb680 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb0157fa640 1 -- 172.21.15.44:0/4248432745 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb028004210 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 10 client.16449 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f35c291330 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb0157fa640 1 -- 172.21.15.44:0/4248432745 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f35c2bb680 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb0157fa640 1 -- 172.21.15.44:0/4248432745 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb028003820 con 0x55f35c32e330 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 20 client.16449 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c348ed0 0x55f35c34b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 4 client.16449 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f35c2865d0 con 0x55f35c348ed0 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.566+0000 7fb03144c640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c348ed0 0x55f35c34b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.567+0000 7fb03144c640 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c348ed0 0x55f35c34b2c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444200fe8197d1ae server_cookie=61ffaf492a9cb2de in_seq=0 out_seq=0 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.567+0000 7fb0157fa640 10 client.16449 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.567+0000 7fb0157fa640 1 -- 172.21.15.44:0/4248432745 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f35c2865d0 con 0x55f35c348ed0 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.567+0000 7fb0157fa640 10 client.16449 handle_command_reply: tid=0 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.567+0000 7fb036bd1740 1 client.16449 shutdown 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 msgr2=0x55f35c338e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 0x55f35c338e90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb028012720 tx=0x7fb028012be0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 shutdown_connections 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c348ed0 0x55f35c34b2c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c3394f0 0x55f35c336720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb0157fa640 0 client.16449 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32ec50 0x55f35c336110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 --2- 172.21.15.44:0/4248432745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c32e330 0x55f35c338e90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 >> 172.21.15.44:0/4248432745 conn(0x55f35c32a8c0 msgr2=0x55f35c329b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 shutdown_connections 2024-09-13T17:29:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 1 -- 172.21.15.44:0/4248432745 wait complete. 2024-09-13T17:29:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.568+0000 7fb036bd1740 20 client.16449 trim_cache size 0 max 16384 2024-09-13T17:29:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 Processor -- start 2024-09-13T17:29:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 -- start start 2024-09-13T17:29:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c32a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c3389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c3401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c32f610 con 0x55f35c3394f0 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c3504d0 con 0x55f35c32e330 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb036bd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c34b5f0 con 0x55f35c32ec50 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c3401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c3401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39726/0 (socket says 172.21.15.44:39726) 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 -- 172.21.15.44:0/2286666061 learned_addr learned my addr 172.21.15.44:0/2286666061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb031c4d640 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c32a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03144c640 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c3389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 -- 172.21.15.44:0/2286666061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 msgr2=0x55f35c3389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c3389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 -- 172.21.15.44:0/2286666061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 msgr2=0x55f35c32a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c32a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03244e640 1 -- 172.21.15.44:0/2286666061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f35c233300 con 0x55f35c3394f0 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.581+0000 7fb03144c640 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c3389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.582+0000 7fb03244e640 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c3401b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb024004af0 tx=0x7fb024009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4965c7cbccaa3535 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.582+0000 7fb020ff9640 1 -- 172.21.15.44:0/2286666061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02402c040 con 0x55f35c3394f0 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.582+0000 7fb020ff9640 1 -- 172.21.15.44:0/2286666061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb024031910 con 0x55f35c3394f0 2024-09-13T17:29:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.582+0000 7fb020ff9640 1 -- 172.21.15.44:0/2286666061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02401bad0 con 0x55f35c3394f0 2024-09-13T17:29:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 -- 172.21.15.44:0/2286666061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 msgr2=0x55f35c3401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c3401b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb024004af0 tx=0x7fb024009570 comp rx=0 tx=0).stop 2024-09-13T17:29:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 -- 172.21.15.44:0/2286666061 shutdown_connections 2024-09-13T17:29:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c3389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c3401b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 --2- 172.21.15.44:0/2286666061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c32a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 -- 172.21.15.44:0/2286666061 >> 172.21.15.44:0/2286666061 conn(0x55f35c32a8c0 msgr2=0x55f35c325a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 -- 172.21.15.44:0/2286666061 shutdown_connections 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 -- 172.21.15.44:0/2286666061 wait complete. 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 -- start start 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 Processor -- start 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.583+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c340650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c33ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c33d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c33d710 con 0x55f35c3394f0 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c33d880 con 0x55f35c32e330 2024-09-13T17:29:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35c39f3d0 con 0x55f35c32ec50 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 10 client.0 _ll_register_callbacks cb 0x7fb024009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c33d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03144c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c33ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c33d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39728/0 (socket says 172.21.15.44:39728) 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 -- 172.21.15.44:0/565092841 learned_addr learned my addr 172.21.15.44:0/565092841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb031c4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c340650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 -- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 msgr2=0x55f35c33ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c33ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 -- 172.21.15.44:0/565092841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 msgr2=0x55f35c340650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c340650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 -- 172.21.15.44:0/565092841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f35c239910 con 0x55f35c3394f0 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb031c4d640 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c340650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb03244e640 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c33d010 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb024034040 tx=0x7fb02401ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8c4978ca26e962 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb0157fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb0157fa640 1 -- 172.21.15.44:0/565092841 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024031a80 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb0157fa640 1 -- 172.21.15.44:0/565092841 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb024031be0 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.584+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb02401bcf0 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb0157fa640 1 -- 172.21.15.44:0/565092841 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb024031da0 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f35c2588e0 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb0157fa640 1 -- 172.21.15.44:0/565092841 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02401bcf0 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb0157fa640 1 -- 172.21.15.44:0/565092841 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb024024070 con 0x55f35c3394f0 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 20 client.16455 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c3a0760 0x55f35c3a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 4 client.16455 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:29:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f35bf86300 con 0x55f35c3a0760 2024-09-13T17:29:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.585+0000 7fb031c4d640 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c3a0760 0x55f35c3a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.586+0000 7fb0157fa640 10 client.16455 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.586+0000 7fb031c4d640 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c3a0760 0x55f35c3a2b50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c893e6fa29f6107f server_cookie=19cf625715454e46 in_seq=0 out_seq=0 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 -- 172.21.15.44:0/3585044501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48740fff70 msgr2=0x7f4874100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 --2- 172.21.15.44:0/3585044501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48740fff70 0x7f4874100350 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4864009930 tx=0x7f486402a420 comp rx=0 tx=0).stop 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 -- 172.21.15.44:0/3585044501 shutdown_connections 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 --2- 172.21.15.44:0/3585044501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4874100920 0x7f4874104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 --2- 172.21.15.44:0/3585044501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48741050c0 0x7f487410cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 --2- 172.21.15.44:0/3585044501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48740fff70 0x7f4874100350 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 -- 172.21.15.44:0/3585044501 >> 172.21.15.44:0/3585044501 conn(0x7f48740fba20 msgr2=0x7f48740fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 -- 172.21.15.44:0/3585044501 shutdown_connections 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.588+0000 7f4878b76640 1 -- 172.21.15.44:0/3585044501 wait complete. 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 Processor -- start 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 -- start start 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48740fff70 0x7f4874196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 0x7f4874196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487419aa90 0x7f487419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487419b480 con 0x7f48741050c0 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487419b5f0 con 0x7f48740fff70 2024-09-13T17:29:30.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4878b76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487419b760 con 0x7f487419aa90 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4871d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 0x7f4874196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4871d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 0x7f4874196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39742/0 (socket says 172.21.15.44:39742) 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.589+0000 7f4871d9b640 1 -- 172.21.15.44:0/2613046385 learned_addr learned my addr 172.21.15.44:0/2613046385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f487259c640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48740fff70 0x7f4874196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4872d9d640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487419aa90 0x7f487419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4871d9b640 1 -- 172.21.15.44:0/2613046385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487419aa90 msgr2=0x7f487419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4871d9b640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487419aa90 0x7f487419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4871d9b640 1 -- 172.21.15.44:0/2613046385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48740fff70 msgr2=0x7f4874196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4871d9b640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48740fff70 0x7f4874196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4871d9b640 1 -- 172.21.15.44:0/2613046385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864009560 con 0x7f48741050c0 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4872d9d640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487419aa90 0x7f487419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4871d9b640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 0x7f4874196760 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4868005fd0 tx=0x7f486800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f48637fe640 1 -- 172.21.15.44:0/2613046385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868009c30 con 0x7f48741050c0 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f48637fe640 1 -- 172.21.15.44:0/2613046385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48680044b0 con 0x7f48741050c0 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48741a2d40 con 0x7f48741050c0 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.590+0000 7f48637fe640 1 -- 172.21.15.44:0/2613046385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868003da0 con 0x7f48741050c0 2024-09-13T17:29:30.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.591+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48741a31e0 con 0x7f48741050c0 2024-09-13T17:29:30.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 Processor -- start 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 -- start start 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f48637fe640 1 -- 172.21.15.44:0/2613046385 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4868003fe0 con 0x7f48741050c0 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d20d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3c7dcc0 con 0x55ada3d140b0 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3c7b450 con 0x55ada3d13790 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487b897740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3c76d40 con 0x55ada3d1e950 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39758/0 (socket says 172.21.15.44:39758) 2024-09-13T17:29:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f487259c640 1 -- 172.21.15.44:0/814945264 learned_addr learned my addr 172.21.15.44:0/814945264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.594+0000 7f4871d9b640 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d20d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4872d9d640 1 -- 172.21.15.44:0/814945264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 msgr2=0x55ada3d20d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4872d9d640 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d20d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4872d9d640 1 -- 172.21.15.44:0/814945264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 msgr2=0x55ada3d1e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4872d9d640 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4872d9d640 1 -- 172.21.15.44:0/814945264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada3c3e620 con 0x55ada3d13790 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f487259c640 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f48637fe640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f485c0a1280 0x7f485c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f48637fe640 1 -- 172.21.15.44:0/2613046385 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f48680cc910 con 0x7f48741050c0 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f487259c640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f485c0a1280 0x7f485c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4872d9d640 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d13b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f486c007db0 tx=0x7f486c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38b2c10bdcbcfa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4860ff9640 1 -- 172.21.15.44:0/814945264 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c004d20 con 0x55ada3d13790 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4860ff9640 1 -- 172.21.15.44:0/814945264 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f486c004e80 con 0x55ada3d13790 2024-09-13T17:29:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.595+0000 7f4860ff9640 1 -- 172.21.15.44:0/814945264 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c010790 con 0x55ada3d13790 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487259c640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f485c0a1280 0x7f485c0a3740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4864005ec0 tx=0x7f4864005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 -- 172.21.15.44:0/814945264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 msgr2=0x55ada3d13b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d13b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f486c007db0 tx=0x7f486c00e720 comp rx=0 tx=0).stop 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 -- 172.21.15.44:0/814945264 shutdown_connections 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d20d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 --2- 172.21.15.44:0/814945264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d13b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 -- 172.21.15.44:0/814945264 >> 172.21.15.44:0/814945264 conn(0x55ada3d0fd00 msgr2=0x55ada3d100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 -- 172.21.15.44:0/814945264 shutdown_connections 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 -- 172.21.15.44:0/814945264 wait complete. 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 -- start start 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.596+0000 7f487b897740 1 Processor -- start 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d1df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d1b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3c35180 con 0x55ada3d140b0 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada394f430 con 0x55ada3d13790 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3bfa5e0 con 0x55ada3d1e950 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487b897740 10 client.0 _ll_register_callbacks cb 0x55ada3c66aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39772/0 (socket says 172.21.15.44:39772) 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 -- 172.21.15.44:0/3144727536 learned_addr learned my addr 172.21.15.44:0/3144727536 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f487259c640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d1df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4872d9d640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d1b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 -- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 msgr2=0x55ada3d1b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d1b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 -- 172.21.15.44:0/3144727536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 msgr2=0x55ada3d1df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d1df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 -- 172.21.15.44:0/3144727536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486c0079e0 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4871d9b640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1b230 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4868099690 tx=0x7f4868003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c314887165c49c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.597+0000 7f4859ffb640 1 -- 172.21.15.44:0/3144727536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868099060 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f4859ffb640 1 -- 172.21.15.44:0/3144727536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f48680035f0 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ada3c691f0 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f4859ffb640 1 -- 172.21.15.44:0/3144727536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4868099e00 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 10 client.16467 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ada3c753c0 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f4859ffb640 1 -- 172.21.15.44:0/3144727536 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ada3c691f0 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f4859ffb640 1 -- 172.21.15.44:0/3144727536 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48680d4070 con 0x55ada3d140b0 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 10 client.16467 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 10 client.16467 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 20 client.16467 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d2e140 0x55ada3d30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 4 client.16467 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ada396a4e0 con 0x55ada3d2e140 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.598+0000 7f487259c640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d2e140 0x55ada3d30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.599+0000 7f487259c640 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d2e140 0x55ada3d30560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e282f1c37b94a77e server_cookie=b2144743da57299 in_seq=0 out_seq=0 2024-09-13T17:29:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.599+0000 7f4859ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb0157fa640 1 -- 172.21.15.44:0/565092841 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f35bf86300 con 0x55f35c3a0760 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb0157fa640 10 client.16455 handle_command_reply: tid=0 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 client.16455 shutdown 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 msgr2=0x55f35c33d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c33d010 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb024034040 tx=0x7fb02401ba80 comp rx=0 tx=0).stop 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 shutdown_connections 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f35c3a0760 0x55f35c3a2b50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f35c32ec50 0x55f35c33ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f35c3394f0 0x55f35c33d010 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 --2- 172.21.15.44:0/565092841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f35c32e330 0x55f35c340650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb0157fa640 0 client.16455 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 >> 172.21.15.44:0/565092841 conn(0x55f35c32a8c0 msgr2=0x55f35c334940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 shutdown_connections 2024-09-13T17:29:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 1 -- 172.21.15.44:0/565092841 wait complete. 2024-09-13T17:29:30.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.599+0000 7f4859ffb640 1 -- 172.21.15.44:0/3144727536 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ada396a4e0 con 0x55ada3d2e140 2024-09-13T17:29:30.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.599+0000 7f4859ffb640 10 client.16467 handle_command_reply: tid=0 2024-09-13T17:29:30.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.599+0000 7f487b897740 1 client.16467 shutdown 2024-09-13T17:29:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.599+0000 7fb036bd1740 20 client.16455 trim_cache size 0 max 16384 2024-09-13T17:29:30.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 msgr2=0x55ada3d1b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1b230 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4868099690 tx=0x7f4868003850 comp rx=0 tx=0).stop 2024-09-13T17:29:30.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 shutdown_connections 2024-09-13T17:29:30.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d2e140 0x55ada3d30560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:29:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:29:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:29:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d1e950 0x55ada3d1b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f4859ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d1b230 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 --2- 172.21.15.44:0/3144727536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d1df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 >> 172.21.15.44:0/3144727536 conn(0x55ada3d0fd00 msgr2=0x55ada3d15a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 shutdown_connections 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 1 -- 172.21.15.44:0/3144727536 wait complete. 2024-09-13T17:29:30.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.600+0000 7f487b897740 20 client.16467 trim_cache size 0 max 16384 2024-09-13T17:29:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb0180a1200 msgr2=0x7fb0180a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb0180a1200 0x7fb0180a36c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb024023000 tx=0x7fb024024000 comp rx=0 tx=0).stop 2024-09-13T17:29:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 msgr2=0x7fb02c196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 0x7fb02c196720 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb01c005ab0 tx=0x7fb01c0329f0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 shutdown_connections 2024-09-13T17:29:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb0180a1200 0x7fb0180a36c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb02c19aac0 0x7fb02c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb02c105060 0x7fb02c196720 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 --2- 172.21.15.44:0/3009996030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb02c0fff10 0x7fb02c1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 >> 172.21.15.44:0/3009996030 conn(0x7fb02c0fb9a0 msgr2=0x7fb02c0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 shutdown_connections 2024-09-13T17:29:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:30.600+0000 7fb033eb1640 1 -- 172.21.15.44:0/3009996030 wait complete. 2024-09-13T17:29:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.611+0000 7f487b897740 1 Processor -- start 2024-09-13T17:29:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.611+0000 7f487b897740 1 -- start start 2024-09-13T17:29:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.611+0000 7f487b897740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d0fd00 0x55ada3d34ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.611+0000 7f487b897740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d13790 0x55ada3d1d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f487b897740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d16250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f487b897740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3c691f0 con 0x55ada3d140b0 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f487b897740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3d13510 con 0x55ada3d0fd00 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f487b897740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3d35400 con 0x55ada3d13790 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d16250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4871d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d13790 0x55ada3d1d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d16250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39778/0 (socket says 172.21.15.44:39778) 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 -- 172.21.15.44:0/1071207220 learned_addr learned my addr 172.21.15.44:0/1071207220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f487259c640 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d0fd00 0x55ada3d34ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 -- 172.21.15.44:0/1071207220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d13790 msgr2=0x55ada3d1d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d13790 0x55ada3d1d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 -- 172.21.15.44:0/1071207220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d0fd00 msgr2=0x55ada3d34ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d0fd00 0x55ada3d34ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 -- 172.21.15.44:0/1071207220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada3c17770 con 0x55ada3d140b0 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4871d9b640 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d13790 0x55ada3d1d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f487259c640 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d0fd00 0x55ada3d34ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f4872d9d640 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d16250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f486c005cc0 tx=0x7f486c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a21f6258edd555 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f48617fa640 1 -- 172.21.15.44:0/1071207220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c01f040 con 0x55ada3d140b0 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.612+0000 7f48617fa640 1 -- 172.21.15.44:0/1071207220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f486c005000 con 0x55ada3d140b0 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f48617fa640 1 -- 172.21.15.44:0/1071207220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c010770 con 0x55ada3d140b0 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 -- 172.21.15.44:0/1071207220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 msgr2=0x55ada3d16250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d16250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f486c005cc0 tx=0x7f486c010430 comp rx=0 tx=0).stop 2024-09-13T17:29:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 -- 172.21.15.44:0/1071207220 shutdown_connections 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d13790 0x55ada3d1d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d16250 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 --2- 172.21.15.44:0/1071207220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d0fd00 0x55ada3d34ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 -- 172.21.15.44:0/1071207220 >> 172.21.15.44:0/1071207220 conn(0x55ada3d1e070 msgr2=0x55ada3d1c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 -- 172.21.15.44:0/1071207220 shutdown_connections 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.613+0000 7f487b897740 1 -- 172.21.15.44:0/1071207220 wait complete. 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 -- start start 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 Processor -- start 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d0fd00 0x55ada3d180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d18630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3d84800 con 0x55ada3d140b0 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3d84970 con 0x55ada3d13790 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada3d84b30 con 0x55ada3d0fd00 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487b897740 10 client.0 _ll_register_callbacks cb 0x7f486c007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d18b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39786/0 (socket says 172.21.15.44:39786) 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 -- 172.21.15.44:0/2827180674 learned_addr learned my addr 172.21.15.44:0/2827180674 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4871d9b640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d18630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f487259c640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d0fd00 0x55ada3d180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 -- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d0fd00 msgr2=0x55ada3d180f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d0fd00 0x55ada3d180f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 -- 172.21.15.44:0/2827180674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 msgr2=0x55ada3d18630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d18630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.614+0000 7f4872d9d640 1 -- 172.21.15.44:0/2827180674 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4864004270 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4872d9d640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d18b70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f486c007cd0 tx=0x7f486c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff4f7c8d79e4d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4859ffb640 1 -- 172.21.15.44:0/2827180674 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c003e50 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4859ffb640 1 -- 172.21.15.44:0/2827180674 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f486c005380 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ada3ca0b80 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4859ffb640 1 -- 172.21.15.44:0/2827180674 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f486c005540 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 10 client.16473 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ada3c753c0 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4859ffb640 1 -- 172.21.15.44:0/2827180674 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ada3ca0b80 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f4859ffb640 1 -- 172.21.15.44:0/2827180674 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f486c02d480 con 0x55ada3d140b0 2024-09-13T17:29:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 10 client.16473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 20 client.16473 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d1e950 0x55ada3d87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 4 client.16473 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.615+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ada397b7e0 con 0x55ada3d1e950 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487259c640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d1e950 0x55ada3d87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487259c640 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d1e950 0x55ada3d87e50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b28c2051511f823 server_cookie=30e1b8ae22c121bc in_seq=0 out_seq=0 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f4859ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f4859ffb640 1 -- 172.21.15.44:0/2827180674 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ada397b7e0 con 0x55ada3d1e950 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f4859ffb640 10 client.16473 handle_command_reply: tid=0 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 client.16473 shutdown 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 msgr2=0x55ada3d18b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d18b70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f486c007cd0 tx=0x7f486c003ba0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 shutdown_connections 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ada3d1e950 0x55ada3d87e50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ada3d140b0 0x55ada3d18b70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ada3d13790 0x55ada3d18630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f4859ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 --2- 172.21.15.44:0/2827180674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ada3d0fd00 0x55ada3d180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 >> 172.21.15.44:0/2827180674 conn(0x55ada3d1e070 msgr2=0x55ada3d1d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 shutdown_connections 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.616+0000 7f487b897740 1 -- 172.21.15.44:0/2827180674 wait complete. 2024-09-13T17:29:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f487b897740 20 client.16473 trim_cache size 0 max 16384 2024-09-13T17:29:30.630 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:29:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f485c0a1280 msgr2=0x7f485c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f485c0a1280 0x7f485c0a3740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4864005ec0 tx=0x7f4864005a20 comp rx=0 tx=0).stop 2024-09-13T17:29:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 msgr2=0x7f4874196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 0x7f4874196760 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4868005fd0 tx=0x7f486800dde0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 shutdown_connections 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f485c0a1280 0x7f485c0a3740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487419aa90 0x7f487419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48741050c0 0x7f4874196760 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 --2- 172.21.15.44:0/2613046385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48740fff70 0x7f4874196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 >> 172.21.15.44:0/2613046385 conn(0x7f48740fba20 msgr2=0x7f48740fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 shutdown_connections 2024-09-13T17:29:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.617+0000 7f4878b76640 1 -- 172.21.15.44:0/2613046385 wait complete. 2024-09-13T17:29:30.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:29:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:30 smithi070 ceph-mon[34502]: pgmap v86: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-09-13T17:29:30.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 -- 172.21.15.44:0/3066476329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf010c6a0 msgr2=0x7f0cf010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/3066476329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf010c6a0 0x7f0cf010ea90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009ca0 tx=0x7f0ce802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 -- 172.21.15.44:0/3066476329 shutdown_connections 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/3066476329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 0x7f0cf010c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/3066476329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf010c6a0 0x7f0cf010ea90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/3066476329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf00feaf0 0x7f0cf00feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 -- 172.21.15.44:0/3066476329 >> 172.21.15.44:0/3066476329 conn(0x7f0cf00fa8d0 msgr2=0x7f0cf00fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 -- 172.21.15.44:0/3066476329 shutdown_connections 2024-09-13T17:29:30.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.774+0000 7f0cf77b5640 1 -- 172.21.15.44:0/3066476329 wait complete. 2024-09-13T17:29:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 Processor -- start 2024-09-13T17:29:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 -- start start 2024-09-13T17:29:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 0x7f0cf0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 0x7f0cf01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf010c6a0 0x7f0cf019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf019d3a0 con 0x7f0cf00feaf0 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf019d510 con 0x7f0cf010c6a0 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf77b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf019d6d0 con 0x7f0cf00ff4a0 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf5551640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 0x7f0cf0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf5551640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 0x7f0cf0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39806/0 (socket says 172.21.15.44:39806) 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.775+0000 7f0cf5551640 1 -- 172.21.15.44:0/522714454 learned_addr learned my addr 172.21.15.44:0/522714454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 0x7f0cf01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf010c6a0 0x7f0cf019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5551640 1 -- 172.21.15.44:0/522714454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 msgr2=0x7f0cf01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5551640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 0x7f0cf01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5551640 1 -- 172.21.15.44:0/522714454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf010c6a0 msgr2=0x7f0cf019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5551640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf010c6a0 0x7f0cf019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5551640 1 -- 172.21.15.44:0/522714454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8009900 con 0x7f0cf00feaf0 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 0x7f0cf01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf010c6a0 0x7f0cf019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf5551640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 0x7f0cf0198470 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00d950 tx=0x7f0cec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0ce67fc640 1 -- 172.21.15.44:0/522714454 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec009c30 con 0x7f0cf00feaf0 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf019d9b0 con 0x7f0cf00feaf0 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0ce67fc640 1 -- 172.21.15.44:0/522714454 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cec009d90 con 0x7f0cf00feaf0 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0ce67fc640 1 -- 172.21.15.44:0/522714454 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00f620 con 0x7f0cf00feaf0 2024-09-13T17:29:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.776+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf01a5230 con 0x7f0cf00feaf0 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.778+0000 7f0ce67fc640 1 -- 172.21.15.44:0/522714454 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0cec0109f0 con 0x7f0cf00feaf0 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 Processor -- start 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 -- start start 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c25b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c260b0 0x55b6d8c30410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c30950 0x55b6d8c32d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8b8fcc0 con 0x55b6d8c25790 2024-09-13T17:29:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8b8d450 con 0x55b6d8c30950 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8b88d40 con 0x55b6d8c260b0 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0ce67fc640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0cdc0a1200 0x7f0cdc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cf5d52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c25b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cf5d52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c25b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39814/0 (socket says 172.21.15.44:39814) 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cf5d52640 1 -- 172.21.15.44:0/322980206 learned_addr learned my addr 172.21.15.44:0/322980206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0cdc0a1200 0x7f0cdc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c30950 0x55b6d8c32d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0cf5551640 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c260b0 0x55b6d8c30410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.779+0000 7f0ce67fc640 1 -- 172.21.15.44:0/522714454 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0cec0ccd10 con 0x7f0cf00feaf0 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 -- 172.21.15.44:0/322980206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c260b0 msgr2=0x55b6d8c30410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c260b0 0x55b6d8c30410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 -- 172.21.15.44:0/322980206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c30950 msgr2=0x55b6d8c32d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c30950 0x55b6d8c32d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 -- 172.21.15.44:0/322980206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d8b50620 con 0x55b6d8c25790 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c25b70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009430 tx=0x7f0ce8033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340bedf826396a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0cdc0a1200 0x7f0cdc0a36c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8031600 tx=0x7f0ce8043000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cdb7fe640 1 -- 172.21.15.44:0/322980206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8032d70 con 0x55b6d8c25790 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cdb7fe640 1 -- 172.21.15.44:0/322980206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ce8031870 con 0x55b6d8c25790 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.780+0000 7f0cdb7fe640 1 -- 172.21.15.44:0/322980206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80042a0 con 0x55b6d8c25790 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/322980206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 msgr2=0x55b6d8c25b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c25b70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009430 tx=0x7f0ce8033000 comp rx=0 tx=0).stop 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/322980206 shutdown_connections 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c260b0 0x55b6d8c30410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c30950 0x55b6d8c32d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/322980206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c25b70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/322980206 >> 172.21.15.44:0/322980206 conn(0x55b6d8c21d00 msgr2=0x55b6d8c220e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/322980206 shutdown_connections 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/322980206 wait complete. 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 -- start start 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.781+0000 7f0cfa4d9740 1 Processor -- start 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c25790 0x55b6d8c2ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 0x55b6d8c2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8b47180 con 0x55b6d8c30950 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8861430 con 0x55b6d8c260b0 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8b0c5e0 con 0x55b6d8c25790 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cfa4d9740 10 client.0 _ll_register_callbacks cb 0x55b6d8b78aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 0x55b6d8c2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 0x55b6d8c2d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39828/0 (socket says 172.21.15.44:39828) 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 -- 172.21.15.44:0/2920712325 learned_addr learned my addr 172.21.15.44:0/2920712325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c25790 0x55b6d8c2ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 -- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c25790 msgr2=0x55b6d8c2ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c25790 0x55b6d8c2ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 -- 172.21.15.44:0/2920712325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 msgr2=0x55b6d8c2d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 -- 172.21.15.44:0/2920712325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce8046040 con 0x55b6d8c30950 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 0x55b6d8c2d860 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009ca0 tx=0x7f0ce8031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3846035817f3bc75 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.782+0000 7f0cd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2920712325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8031e30 con 0x55b6d8c30950 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2920712325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0ce8032d70 con 0x55b6d8c30950 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6d8bb2b80 con 0x55b6d8c30950 2024-09-13T17:29:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2920712325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce8030c10 con 0x55b6d8c30950 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 10 client.16485 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b6d8b873c0 con 0x55b6d8c30950 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2920712325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b6d8bb2b80 con 0x55b6d8c30950 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2920712325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ce802f8e0 con 0x55b6d8c30950 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 10 client.16485 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 20 client.16485 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c400f0 0x55b6d8c42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 4 client.16485 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.783+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6d8b7b1f0 con 0x55b6d8c400f0 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.784+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c400f0 0x55b6d8c42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.784+0000 7f0cd8ff9640 10 client.16485 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.784+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c400f0 0x55b6d8c42510 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197d06225847c945 server_cookie=b91c2f8c6e7d6f2e in_seq=0 out_seq=0 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2920712325 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b6d8b7b1f0 con 0x55b6d8c400f0 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cd8ff9640 10 client.16485 handle_command_reply: tid=0 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 client.16485 shutdown 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 msgr2=0x55b6d8c2d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 0x55b6d8c2d860 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009ca0 tx=0x7f0ce8031cc0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 shutdown_connections 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c400f0 0x55b6d8c42510 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c30950 0x55b6d8c2d860 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cd8ff9640 0 client.16485 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2920712325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c25790 0x55b6d8c2ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 >> 172.21.15.44:0/2920712325 conn(0x55b6d8c21d00 msgr2=0x55b6d8c20eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 shutdown_connections 2024-09-13T17:29:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2920712325 wait complete. 2024-09-13T17:29:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.785+0000 7f0cfa4d9740 20 client.16485 trim_cache size 0 max 16384 2024-09-13T17:29:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 Processor -- start 2024-09-13T17:29:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 -- start start 2024-09-13T17:29:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c28a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8c22f10 con 0x55b6d8c25790 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8c1c3b0 con 0x55b6d8c260b0 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8c473b0 con 0x55b6d8c30950 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.796+0000 7f0cf5551640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39836/0 (socket says 172.21.15.44:39836) 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 -- 172.21.15.44:0/2895255305 learned_addr learned my addr 172.21.15.44:0/2895255305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c28a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 -- 172.21.15.44:0/2895255305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 msgr2=0x55b6d8c28a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c28a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 -- 172.21.15.44:0/2895255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 msgr2=0x55b6d8c46e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c46e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 -- 172.21.15.44:0/2895255305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d8b29770 con 0x55b6d8c25790 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c216a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0ca140 tx=0x7f0cec095ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4d50199c57dddf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cdbfff640 1 -- 172.21.15.44:0/2895255305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0a1f20 con 0x55b6d8c25790 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cdbfff640 1 -- 172.21.15.44:0/2895255305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cec097e40 con 0x55b6d8c25790 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.797+0000 7f0cdbfff640 1 -- 172.21.15.44:0/2895255305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0ca360 con 0x55b6d8c25790 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2895255305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 msgr2=0x55b6d8c216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c216a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0ca140 tx=0x7f0cec095ea0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2895255305 shutdown_connections 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c28a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c260b0 0x55b6d8c46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2895255305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c25790 0x55b6d8c216a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2895255305 >> 172.21.15.44:0/2895255305 conn(0x55b6d8c21d00 msgr2=0x55b6d8c28360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2895255305 shutdown_connections 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2895255305 wait complete. 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- start start 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 Processor -- start 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c25790 0x55b6d8c2a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c2ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8c96690 con 0x55b6d8c260b0 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8c967d0 con 0x55b6d8c25790 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d8c96990 con 0x55b6d8c30950 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.798+0000 7f0cfa4d9740 10 client.0 _ll_register_callbacks cb 0x7f0ce8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39842/0 (socket says 172.21.15.44:39842) 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 -- 172.21.15.44:0/2801581892 learned_addr learned my addr 172.21.15.44:0/2801581892 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c25790 0x55b6d8c2a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c2ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 -- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 msgr2=0x55b6d8c2ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c2ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 -- 172.21.15.44:0/2801581892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c25790 msgr2=0x55b6d8c2a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c25790 0x55b6d8c2a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 -- 172.21.15.44:0/2801581892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf01a5230 con 0x55b6d8c260b0 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf5d52640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c2ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cf4d50640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2a6f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0199a10 tx=0x7f0ce000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d380b24ffe87d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2801581892 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000eae0 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce8031870 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2801581892 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0ce000ec40 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2801581892 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce000ee00 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cfa4d9740 10 client.16491 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.799+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b6d8b4ee80 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2801581892 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ce8031870 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2801581892 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ce00195d0 con 0x55b6d8c260b0 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cfa4d9740 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cfa4d9740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cfa4d9740 20 client.16491 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c981c0 0x55b6d8c9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cfa4d9740 4 client.16491 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:29:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6d887c4e0 con 0x55b6d8c981c0 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c981c0 0x55b6d8c9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cd8ff9640 10 client.16491 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.800+0000 7f0cf5551640 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c981c0 0x55b6d8c9a5e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1844f15c4b7074c9 server_cookie=1503ab6cf3c8abb7 in_seq=0 out_seq=0 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cd8ff9640 1 -- 172.21.15.44:0/2801581892 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6d887c4e0 con 0x55b6d8c981c0 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cd8ff9640 10 client.16491 handle_command_reply: tid=0 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 client.16491 shutdown 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 msgr2=0x55b6d8c2a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2a6f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0199a10 tx=0x7f0ce000fb30 comp rx=0 tx=0).stop 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 shutdown_connections 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b6d8c981c0 0x55b6d8c9a5e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6d8c30950 0x55b6d8c2ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6d8c260b0 0x55b6d8c2a6f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cd8ff9640 0 client.16491 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 --2- 172.21.15.44:0/2801581892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b6d8c25790 0x55b6d8c2a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 >> 172.21.15.44:0/2801581892 conn(0x55b6d8c21d00 msgr2=0x55b6d8c271f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 shutdown_connections 2024-09-13T17:29:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 1 -- 172.21.15.44:0/2801581892 wait complete. 2024-09-13T17:29:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cfa4d9740 20 client.16491 trim_cache size 0 max 16384 2024-09-13T17:29:30.811 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0cdc0a1200 msgr2=0x7f0cdc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0cdc0a1200 0x7f0cdc0a36c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8031600 tx=0x7f0ce8043000 comp rx=0 tx=0).stop 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 msgr2=0x7f0cf0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.801+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 0x7f0cf0198470 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00d950 tx=0x7f0cec00de10 comp rx=0 tx=0).stop 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 shutdown_connections 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0cdc0a1200 0x7f0cdc0a36c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf00ff4a0 0x7f0cf01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf010c6a0 0x7f0cf019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 --2- 172.21.15.44:0/522714454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf00feaf0 0x7f0cf0198470 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 >> 172.21.15.44:0/522714454 conn(0x7f0cf00fa8d0 msgr2=0x7f0cf00fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 shutdown_connections 2024-09-13T17:29:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:30.802+0000 7f0cf77b5640 1 -- 172.21.15.44:0/522714454 wait complete. 2024-09-13T17:29:31.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:31 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/840747479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:29:31.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:31 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/840747479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:29:31.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:31 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/840747479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:29:31.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:29:31.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 -- 172.21.15.44:0/2625288487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97440728e0 msgr2=0x7f9744072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 --2- 172.21.15.44:0/2625288487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97440728e0 0x7f9744072cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9740009900 tx=0x7f974002a480 comp rx=0 tx=0).stop 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 -- 172.21.15.44:0/2625288487 shutdown_connections 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 --2- 172.21.15.44:0/2625288487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974410c560 0x7f974410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 --2- 172.21.15.44:0/2625288487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9744073200 0x7f974410c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 --2- 172.21.15.44:0/2625288487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97440728e0 0x7f9744072cc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 -- 172.21.15.44:0/2625288487 >> 172.21.15.44:0/2625288487 conn(0x7f97440fba20 msgr2=0x7f97440fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:31.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 -- 172.21.15.44:0/2625288487 shutdown_connections 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 -- 172.21.15.44:0/2625288487 wait complete. 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.933+0000 7f974bed7640 1 Processor -- start 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 -- start start 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97440728e0 0x7f9744198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744073200 0x7f9744198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 0x7f974419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419d270 con 0x7f974410c560 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419d3e0 con 0x7f97440728e0 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974bed7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419d5a0 con 0x7f9744073200 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 0x7f974419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 0x7f974419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39866/0 (socket says 172.21.15.44:39866) 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 -- 172.21.15.44:0/3785772630 learned_addr learned my addr 172.21.15.44:0/3785772630 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f9749c73640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97440728e0 0x7f9744198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f9749472640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744073200 0x7f9744198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 -- 172.21.15.44:0/3785772630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744073200 msgr2=0x7f9744198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744073200 0x7f9744198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 -- 172.21.15.44:0/3785772630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97440728e0 msgr2=0x7f9744198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97440728e0 0x7f9744198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 -- 172.21.15.44:0/3785772630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9740009560 con 0x7f974410c560 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.934+0000 7f974a474640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 0x7f974419cc70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f973c00cce0 tx=0x7f973c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.935+0000 7f973affd640 1 -- 172.21.15.44:0/3785772630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c00f450 con 0x7f974410c560 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.935+0000 7f973affd640 1 -- 172.21.15.44:0/3785772630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f973c00fa40 con 0x7f974410c560 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.935+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974419d880 con 0x7f974410c560 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.935+0000 7f973affd640 1 -- 172.21.15.44:0/3785772630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c0189a0 con 0x7f974410c560 2024-09-13T17:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.935+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97441a5100 con 0x7f974410c560 2024-09-13T17:29:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 Processor -- start 2024-09-13T17:29:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f973affd640 1 -- 172.21.15.44:0/3785772630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f973c00fbb0 con 0x7f974410c560 2024-09-13T17:29:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 -- start start 2024-09-13T17:29:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877d7c0 0x557b3877dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b38788440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b38788980 0x557b3878ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b386e8c60 con 0x557b3877e0e0 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b386e63f0 con 0x557b38788980 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.938+0000 7f974ebf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b386e1ce0 con 0x557b3877d7c0 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b38788440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b38788440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39882/0 (socket says 172.21.15.44:39882) 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 -- 172.21.15.44:0/4022844048 learned_addr learned my addr 172.21.15.44:0/4022844048 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f973affd640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97300a1200 0x7f97300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749472640 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b38788980 0x557b3878ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749472640 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877d7c0 0x557b3877dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 -- 172.21.15.44:0/4022844048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877d7c0 msgr2=0x557b3877dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877d7c0 0x557b3877dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 -- 172.21.15.44:0/4022844048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b38788980 msgr2=0x557b3878ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b38788980 0x557b3878ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 -- 172.21.15.44:0/4022844048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b3869d970 con 0x557b3877e0e0 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749472640 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b38788980 0x557b3878ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f974a474640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97300a1200 0x7f97300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f973affd640 1 -- 172.21.15.44:0/3785772630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f973c0cd3e0 con 0x7f974410c560 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f9749c73640 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b38788440 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f974002a9a0 tx=0x7f9740031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4d3df7d30a95eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f972ffff640 1 -- 172.21.15.44:0/4022844048 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740004300 con 0x557b3877e0e0 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f972ffff640 1 -- 172.21.15.44:0/4022844048 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9740031df0 con 0x557b3877e0e0 2024-09-13T17:29:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.939+0000 7f972ffff640 1 -- 172.21.15.44:0/4022844048 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740003d40 con 0x557b3877e0e0 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 -- 172.21.15.44:0/4022844048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 msgr2=0x557b38788440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b38788440 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f974002a9a0 tx=0x7f9740031670 comp rx=0 tx=0).stop 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974a474640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97300a1200 0x7f97300a36c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f973c099e70 tx=0x7f973c06c250 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 -- 172.21.15.44:0/4022844048 shutdown_connections 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b38788980 0x557b3878ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b38788440 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 --2- 172.21.15.44:0/4022844048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877d7c0 0x557b3877dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 -- 172.21.15.44:0/4022844048 >> 172.21.15.44:0/4022844048 conn(0x557b38779d50 msgr2=0x557b3877a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 -- 172.21.15.44:0/4022844048 shutdown_connections 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.940+0000 7f974ebf9740 1 -- 172.21.15.44:0/4022844048 wait complete. 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 -- start start 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 Processor -- start 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b38788330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b387855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38788980 0x557b38785bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3869ff80 con 0x557b3877e0e0 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3869fc20 con 0x557b3877d7c0 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3866cab0 con 0x557b38788980 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f974ebf9740 10 client.0 _ll_register_callbacks cb 0x557b386db350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f9749c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b387855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f9749c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b387855b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39896/0 (socket says 172.21.15.44:39896) 2024-09-13T17:29:31.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f9749c73640 1 -- 172.21.15.44:0/2018047418 learned_addr learned my addr 172.21.15.44:0/2018047418 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f9749472640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b38788330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.941+0000 7f9749472640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38788980 0x557b38785bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f9749c73640 1 -- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38788980 msgr2=0x557b38785bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f9749c73640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38788980 0x557b38785bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f9749c73640 1 -- 172.21.15.44:0/2018047418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 msgr2=0x557b38788330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f9749c73640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b38788330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f9749c73640 1 -- 172.21.15.44:0/2018047418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974410f630 con 0x557b3877e0e0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f9749c73640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b387855b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f974002aa70 tx=0x7f9740003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468768d12dc88e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f972d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f972d7fa640 1 -- 172.21.15.44:0/2018047418 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974004a040 con 0x557b3877e0e0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b383d5590 con 0x557b3877e0e0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f972d7fa640 1 -- 172.21.15.44:0/2018047418 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9740004300 con 0x557b3877e0e0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f972d7fa640 1 -- 172.21.15.44:0/2018047418 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97400044c0 con 0x557b3877e0e0 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f974ebf9740 10 client.16503 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557b386e0360 con 0x557b3877e0e0 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.942+0000 7f972d7fa640 1 -- 172.21.15.44:0/2018047418 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557b383d5590 con 0x557b3877e0e0 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f972d7fa640 1 -- 172.21.15.44:0/2018047418 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f974004fb60 con 0x557b3877e0e0 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f974ebf9740 10 client.16503 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f974ebf9740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f974ebf9740 20 client.16503 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f974ebf9740 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38798310 0x557b3879a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f974ebf9740 4 client.16503 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b383e6860 con 0x557b38798310 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f9749472640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38798310 0x557b3879a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f9749472640 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38798310 0x557b3879a700 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecef0a92cf860caa server_cookie=32714a020d9f219e in_seq=0 out_seq=0 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.943+0000 7f972d7fa640 10 client.16503 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f972d7fa640 1 -- 172.21.15.44:0/2018047418 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557b383e6860 con 0x557b38798310 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f972d7fa640 10 client.16503 handle_command_reply: tid=0 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 client.16503 shutdown 2024-09-13T17:29:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 msgr2=0x557b387855b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b387855b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f974002aa70 tx=0x7f9740003890 comp rx=0 tx=0).stop 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 shutdown_connections 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38798310 0x557b3879a700 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38788980 0x557b38785bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f972d7fa640 0 client.16503 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877e0e0 0x557b387855b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 --2- 172.21.15.44:0/2018047418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b38788330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.944+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 >> 172.21.15.44:0/2018047418 conn(0x557b38779d50 msgr2=0x557b3877f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.945+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 shutdown_connections 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.945+0000 7f974ebf9740 1 -- 172.21.15.44:0/2018047418 wait complete. 2024-09-13T17:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.945+0000 7f974ebf9740 20 client.16503 trim_cache size 0 max 16384 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 Processor -- start 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 -- start start 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38779d50 0x557b3878f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 0x557b3878fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877e0e0 0x557b38784e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3878b430 con 0x557b3877d7c0 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b38794590 con 0x557b3877e0e0 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f974ebf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3877aea0 con 0x557b38779d50 2024-09-13T17:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.959+0000 7f9749c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 0x557b3878fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 0x557b3878fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39906/0 (socket says 172.21.15.44:39906) 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 -- 172.21.15.44:0/3842034281 learned_addr learned my addr 172.21.15.44:0/3842034281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749472640 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877e0e0 0x557b38784e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 -- 172.21.15.44:0/3842034281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38779d50 msgr2=0x557b3878f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38779d50 0x557b3878f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 -- 172.21.15.44:0/3842034281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877e0e0 msgr2=0x557b38784e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877e0e0 0x557b38784e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 -- 172.21.15.44:0/3842034281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b386821c0 con 0x557b3877d7c0 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749472640 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877e0e0 0x557b38784e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9749c73640 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 0x557b3878fbc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9740000c00 tx=0x7f9740031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8170f8a7b9e7e73c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9738ff9640 1 -- 172.21.15.44:0/3842034281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974004a070 con 0x557b3877d7c0 2024-09-13T17:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9738ff9640 1 -- 172.21.15.44:0/3842034281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9740046530 con 0x557b3877d7c0 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.960+0000 7f9738ff9640 1 -- 172.21.15.44:0/3842034281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974003c800 con 0x557b3877d7c0 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 -- 172.21.15.44:0/3842034281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 msgr2=0x557b3878fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 0x557b3878fbc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9740000c00 tx=0x7f9740031670 comp rx=0 tx=0).stop 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 -- 172.21.15.44:0/3842034281 shutdown_connections 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877e0e0 0x557b38784e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3877d7c0 0x557b3878fbc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- 172.21.15.44:0/3842034281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b38779d50 0x557b3878f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 -- 172.21.15.44:0/3842034281 >> 172.21.15.44:0/3842034281 conn(0x557b38788490 msgr2=0x557b38783e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 -- 172.21.15.44:0/3842034281 shutdown_connections 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 -- 172.21.15.44:0/3842034281 wait complete. 2024-09-13T17:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 -- start start 2024-09-13T17:29:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 Processor -- start 2024-09-13T17:29:31.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 0x557b3878e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b3878ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.961+0000 7f974ebf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877e0e0 0x557b387ee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f974ebf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3878f360 con 0x557b38779d50 2024-09-13T17:29:31.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f974ebf9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b387eed50 con 0x557b3877d7c0 2024-09-13T17:29:31.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f974ebf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b387eef10 con 0x557b3877e0e0 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f974ebf9740 10 client.0 _ll_register_callbacks cb 0x7f9740031540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 0x557b3878e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 0x557b3878e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39918/0 (socket says 172.21.15.44:39918) 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 -- 172.21.15.44:0/1481082261 learned_addr learned my addr 172.21.15.44:0/1481082261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749c73640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b3878ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877e0e0 0x557b387ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 -- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877e0e0 msgr2=0x557b387ee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877e0e0 0x557b387ee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 -- 172.21.15.44:0/1481082261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 msgr2=0x557b3878ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b3878ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 -- 172.21.15.44:0/1481082261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b386e0360 con 0x557b38779d50 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f9749472640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 0x557b3878e6c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x557b3877ae40 tx=0x7f97340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cef113d5304752 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f972d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f972d7fa640 1 -- 172.21.15.44:0/1481082261 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734023110 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f972d7fa640 1 -- 172.21.15.44:0/1481082261 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9734004bf0 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.962+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9740031df0 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f972d7fa640 1 -- 172.21.15.44:0/1481082261 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9734004db0 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 10 client.16509 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557b386a7990 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f972d7fa640 1 -- 172.21.15.44:0/1481082261 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9740031df0 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f972d7fa640 1 -- 172.21.15.44:0/1481082261 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9734012730 con 0x557b38779d50 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 10 client.16509 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 20 client.16509 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38788980 0x557b387f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 4 client.16509 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:29:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557b38439eb0 con 0x557b38788980 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.963+0000 7f9749c73640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38788980 0x557b387f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.964+0000 7f972d7fa640 10 client.16509 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.964+0000 7f9749c73640 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38788980 0x557b387f2750 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e125bf9567968d server_cookie=eae281804dc1b211 in_seq=0 out_seq=0 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f972d7fa640 1 -- 172.21.15.44:0/1481082261 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557b38439eb0 con 0x557b38788980 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f972d7fa640 10 client.16509 handle_command_reply: tid=0 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 client.16509 shutdown 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 msgr2=0x557b3878e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 0x557b3878e6c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x557b3877ae40 tx=0x7f97340046c0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 shutdown_connections 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557b38788980 0x557b387f2750 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b3877e0e0 0x557b387ee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b3877d7c0 0x557b3878ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f972d7fa640 0 client.16509 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 --2- 172.21.15.44:0/1481082261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b38779d50 0x557b3878e6c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 >> 172.21.15.44:0/1481082261 conn(0x557b38788490 msgr2=0x557b38784de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 shutdown_connections 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 1 -- 172.21.15.44:0/1481082261 wait complete. 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.965+0000 7f974ebf9740 20 client.16509 trim_cache size 0 max 16384 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "e0ad5152-c79b-4719-bfbe-70162ed3b9cd", 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:29:31.973 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:29:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97300a1200 msgr2=0x7f97300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97300a1200 0x7f97300a36c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f973c099e70 tx=0x7f973c06c250 comp rx=0 tx=0).stop 2024-09-13T17:29:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 msgr2=0x7f974419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 0x7f974419cc70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f973c00cce0 tx=0x7f973c00a620 comp rx=0 tx=0).stop 2024-09-13T17:29:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 shutdown_connections 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97300a1200 0x7f97300a36c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744073200 0x7f9744198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f974410c560 0x7f974419cc70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 --2- 172.21.15.44:0/3785772630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97440728e0 0x7f9744198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 >> 172.21.15.44:0/3785772630 conn(0x7f97440fba20 msgr2=0x7f97440fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 shutdown_connections 2024-09-13T17:29:31.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:31.966+0000 7f974bed7640 1 -- 172.21.15.44:0/3785772630 wait complete. 2024-09-13T17:29:31.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 -- 172.21.15.44:0/961010506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa950103fb0 msgr2=0x7fa95010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 --2- 172.21.15.44:0/961010506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa950103fb0 0x7fa95010bb00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa948009cd0 tx=0x7fa94802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 -- 172.21.15.44:0/961010506 shutdown_connections 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 --2- 172.21.15.44:0/961010506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa950103fb0 0x7fa95010bb00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 --2- 172.21.15.44:0/961010506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500ff810 0x7fa950103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 --2- 172.21.15.44:0/961010506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9500fee60 0x7fa9500ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 -- 172.21.15.44:0/961010506 >> 172.21.15.44:0/961010506 conn(0x7fa950075c60 msgr2=0x7fa9500fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 -- 172.21.15.44:0/961010506 shutdown_connections 2024-09-13T17:29:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.081+0000 7fa9551bc640 1 -- 172.21.15.44:0/961010506 wait complete. 2024-09-13T17:29:32.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.082+0000 7fa9551bc640 1 Processor -- start 2024-09-13T17:29:32.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.082+0000 7fa9551bc640 1 -- start start 2024-09-13T17:29:32.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa9551bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500fee60 0x7fa9501950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa9551bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950103fb0 0x7fa9501955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa9551bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 0x7fa950199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa9551bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95019a3d0 con 0x7fa9501999e0 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa9551bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95019a540 con 0x7fa9500fee60 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa9551bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95019a6b0 con 0x7fa950103fb0 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 0x7fa950199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 0x7fa950199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39936/0 (socket says 172.21.15.44:39936) 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500fee60 0x7fa9501950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 -- 172.21.15.44:0/945896544 learned_addr learned my addr 172.21.15.44:0/945896544 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94e59c640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950103fb0 0x7fa9501955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 -- 172.21.15.44:0/945896544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950103fb0 msgr2=0x7fa9501955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950103fb0 0x7fa9501955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 -- 172.21.15.44:0/945896544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500fee60 msgr2=0x7fa9501950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500fee60 0x7fa9501950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 -- 172.21.15.44:0/945896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009900 con 0x7fa9501999e0 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500fee60 0x7fa9501950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.083+0000 7fa94f59e640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 0x7fa950199e90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa948004af0 tx=0x7fa948030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.084+0000 7fa937fff640 1 -- 172.21.15.44:0/945896544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948031b80 con 0x7fa9501999e0 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.084+0000 7fa937fff640 1 -- 172.21.15.44:0/945896544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa948003e00 con 0x7fa9501999e0 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.084+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9501a1c90 con 0x7fa9501999e0 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.084+0000 7fa937fff640 1 -- 172.21.15.44:0/945896544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948045410 con 0x7fa9501999e0 2024-09-13T17:29:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.084+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9501a2110 con 0x7fa9501999e0 2024-09-13T17:29:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.086+0000 7fa937fff640 1 -- 172.21.15.44:0/945896544 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa94800a040 con 0x7fa9501999e0 2024-09-13T17:29:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.086+0000 7fa957b22740 1 Processor -- start 2024-09-13T17:29:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.086+0000 7fa957b22740 1 -- start start 2024-09-13T17:29:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.086+0000 7fa957b22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac00293030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.086+0000 7fa957b22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac00293570 0x55ac0029fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa957b22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac002a01e0 0x55ac002a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa957b22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac001fdf20 con 0x55ac00292c50 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa957b22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac001fb6b0 con 0x55ac002a01e0 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa957b22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac001f6fa0 con 0x55ac00293570 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac00293030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac00293030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39946/0 (socket says 172.21.15.44:39946) 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94e59c640 1 -- 172.21.15.44:0/3390086258 learned_addr learned my addr 172.21.15.44:0/3390086258 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac00293570 0x55ac0029fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94e59c640 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac002a01e0 0x55ac002a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 -- 172.21.15.44:0/3390086258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac002a01e0 msgr2=0x55ac002a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa937fff640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9380a1200 0x7fa9380a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac002a01e0 0x55ac002a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 -- 172.21.15.44:0/3390086258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 msgr2=0x55ac00293030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac00293030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 -- 172.21.15.44:0/3390086258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac001b1840 con 0x55ac00293570 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94e59c640 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac00293030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94f59e640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9380a1200 0x7fa9380a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac00293570 0x55ac0029fca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9400103d0 tx=0x7fa940010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd932ad3db78510d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa937fff640 1 -- 172.21.15.44:0/945896544 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa9480ee360 con 0x7fa9501999e0 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.087+0000 7fa9357fa640 1 -- 172.21.15.44:0/3390086258 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940011700 con 0x55ac00293570 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa9357fa640 1 -- 172.21.15.44:0/3390086258 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa940015070 con 0x55ac00293570 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa9357fa640 1 -- 172.21.15.44:0/3390086258 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94001cb90 con 0x55ac00293570 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 -- 172.21.15.44:0/3390086258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac00293570 msgr2=0x55ac0029fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa94f59e640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9380a1200 0x7fa9380a36c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ebd90 tx=0x7fa94802a8e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac00293570 0x55ac0029fca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9400103d0 tx=0x7fa940010890 comp rx=0 tx=0).stop 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 -- 172.21.15.44:0/3390086258 shutdown_connections 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac00293570 0x55ac0029fca0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac002a01e0 0x55ac002a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 --2- 172.21.15.44:0/3390086258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac00293030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 -- 172.21.15.44:0/3390086258 >> 172.21.15.44:0/3390086258 conn(0x55ac0028f380 msgr2=0x55ac0028f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 -- 172.21.15.44:0/3390086258 shutdown_connections 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.088+0000 7fa957b22740 1 -- 172.21.15.44:0/3390086258 wait complete. 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 -- start start 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 Processor -- start 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac002a01e0 0x55ac0029d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac001982a0 con 0x55ac00293570 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abffecf430 con 0x55ac00292c50 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0017a680 con 0x55ac002a01e0 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa957b22740 10 client.0 _ll_register_callbacks cb 0x55ac001f0610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39962/0 (socket says 172.21.15.44:39962) 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2529264750 learned_addr learned my addr 172.21.15.44:0/2529264750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac002a01e0 0x55ac0029d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac002a01e0 msgr2=0x55ac0029d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac002a01e0 0x55ac0029d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2529264750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 msgr2=0x55ac0029faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2529264750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94000bc60 con 0x55ac00293570 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.089+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac002a01e0 0x55ac0029d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029cd70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa94001c770 tx=0x7fa940004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b105285acf2ca26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa932ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa932ff5640 1 -- 172.21.15.44:0/2529264750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940021070 con 0x55ac00293570 2024-09-13T17:29:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa932ff5640 1 -- 172.21.15.44:0/2529264750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa940017070 con 0x55ac00293570 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abffeea4e0 con 0x55ac00293570 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa932ff5640 1 -- 172.21.15.44:0/2529264750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa940004490 con 0x55ac00293570 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 10 client.16521 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ac001f5620 con 0x55ac00293570 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa932ff5640 1 -- 172.21.15.44:0/2529264750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55abffeea4e0 con 0x55ac00293570 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa932ff5640 1 -- 172.21.15.44:0/2529264750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa940011500 con 0x55ac00293570 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 10 client.16521 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 20 client.16521 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002afe60 0x55ac002b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 4 client.16521 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.090+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abffefb7e0 con 0x55ac002afe60 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.091+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002afe60 0x55ac002b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.091+0000 7fa932ff5640 10 client.16521 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:32.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.091+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002afe60 0x55ac002b2280 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8ec0590db1e624 server_cookie=e0eeea4bc6f8152f in_seq=0 out_seq=0 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa932ff5640 1 -- 172.21.15.44:0/2529264750 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55abffefb7e0 con 0x55ac002afe60 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa932ff5640 10 client.16521 handle_command_reply: tid=0 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 client.16521 shutdown 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 msgr2=0x55ac0029cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029cd70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa94001c770 tx=0x7fa940004270 comp rx=0 tx=0).stop 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 shutdown_connections 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002afe60 0x55ac002b2280 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac002a01e0 0x55ac0029d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029cd70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 --2- 172.21.15.44:0/2529264750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa932ff5640 0 client.16521 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:32.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 >> 172.21.15.44:0/2529264750 conn(0x55ac0028f380 msgr2=0x55ac0028e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:32.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 shutdown_connections 2024-09-13T17:29:32.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 1 -- 172.21.15.44:0/2529264750 wait complete. 2024-09-13T17:29:32.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.092+0000 7fa957b22740 20 client.16521 trim_cache size 0 max 16384 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 Processor -- start 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 -- start start 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac0028ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac002acce0 con 0x55ac00293570 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac002abec0 con 0x55ac00292c50 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa957b22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac00290510 con 0x55ac0028f380 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39976/0 (socket says 172.21.15.44:39976) 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 -- 172.21.15.44:0/13336600 learned_addr learned my addr 172.21.15.44:0/13336600 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac0028ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 -- 172.21.15.44:0/13336600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 msgr2=0x55ac0028ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac0028ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 -- 172.21.15.44:0/13336600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 msgr2=0x55ac0029b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 -- 172.21.15.44:0/13336600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac00197790 con 0x55ac00293570 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.107+0000 7fa94e59c640 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029bb60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa944011580 tx=0x7fa94400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b1629234b989e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa935ffb640 1 -- 172.21.15.44:0/13336600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944017040 con 0x55ac00293570 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa935ffb640 1 -- 172.21.15.44:0/13336600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9440054b0 con 0x55ac00293570 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa935ffb640 1 -- 172.21.15.44:0/13336600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944004890 con 0x55ac00293570 2024-09-13T17:29:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 -- 172.21.15.44:0/13336600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 msgr2=0x55ac0029bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029bb60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa944011580 tx=0x7fa94400b780 comp rx=0 tx=0).stop 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 -- 172.21.15.44:0/13336600 shutdown_connections 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00293570 0x55ac0029bb60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00292c50 0x55ac0029b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 --2- 172.21.15.44:0/13336600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac0028ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 -- 172.21.15.44:0/13336600 >> 172.21.15.44:0/13336600 conn(0x55ac0029fc50 msgr2=0x55ac00298000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 -- 172.21.15.44:0/13336600 shutdown_connections 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 -- 172.21.15.44:0/13336600 wait complete. 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.108+0000 7fa957b22740 1 -- start start 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 Processor -- start 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac002a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac002a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00293570 0x55ac002a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac002a5ca0 con 0x55ac00292c50 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac002a5e10 con 0x55ac00293570 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac002a5fd0 con 0x55ac0028f380 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa957b22740 10 client.0 _ll_register_callbacks cb 0x7fa94001c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac002a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac002a7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39980/0 (socket says 172.21.15.44:39980) 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2600170555 learned_addr learned my addr 172.21.15.44:0/2600170555 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac002a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00293570 0x55ac002a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 msgr2=0x55ac002a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac002a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2600170555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00293570 msgr2=0x55ac002a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00293570 0x55ac002a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94ed9d640 1 -- 172.21.15.44:0/2600170555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94401a040 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.109+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00293570 0x55ac002a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa94ed9d640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac002a7480 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa9400108d0 tx=0x7fa94000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c880afd3d25777 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa932ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa932ff5640 1 -- 172.21.15.44:0/2600170555 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940021070 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa932ff5640 1 -- 172.21.15.44:0/2600170555 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa940000e40 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa940015070 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa932ff5640 1 -- 172.21.15.44:0/2600170555 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa940011710 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 10 client.16527 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ac001bcea0 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa932ff5640 1 -- 172.21.15.44:0/2600170555 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa940015070 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa932ff5640 1 -- 172.21.15.44:0/2600170555 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa94002d040 con 0x55ac00292c50 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 10 client.16527 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 20 client.16527 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002a01e0 0x55ac00309340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 4 client.16527 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55abfff4ee30 con 0x55ac002a01e0 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.110+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002a01e0 0x55ac00309340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa94e59c640 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002a01e0 0x55ac00309340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f8757e3bf1a2a6f server_cookie=f9cfffb0a5fb4e8c in_seq=0 out_seq=0 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa932ff5640 10 client.16527 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa932ff5640 1 -- 172.21.15.44:0/2600170555 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55abfff4ee30 con 0x55ac002a01e0 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa932ff5640 10 client.16527 handle_command_reply: tid=0 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 client.16527 shutdown 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 msgr2=0x55ac002a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac002a7480 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa9400108d0 tx=0x7fa94000bf80 comp rx=0 tx=0).stop 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 shutdown_connections 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ac002a01e0 0x55ac00309340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac00293570 0x55ac002a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac00292c50 0x55ac002a7480 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa932ff5640 0 client.16527 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 --2- 172.21.15.44:0/2600170555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac0028f380 0x55ac002a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 >> 172.21.15.44:0/2600170555 conn(0x55ac0029fc50 msgr2=0x55ac0029b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 shutdown_connections 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 1 -- 172.21.15.44:0/2600170555 wait complete. 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.111+0000 7fa957b22740 20 client.16527 trim_cache size 0 max 16384 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:29:32.120 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout: "e0ad5152-c79b-4719-bfbe-70162ed3b9cd": { 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "e0ad5152-c79b-4719-bfbe-70162ed3b9cd", 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:29:32.121 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9380a1200 msgr2=0x7fa9380a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9380a1200 0x7fa9380a36c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ebd90 tx=0x7fa94802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 msgr2=0x7fa950199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 0x7fa950199e90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa948004af0 tx=0x7fa948030e50 comp rx=0 tx=0).stop 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 shutdown_connections 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9380a1200 0x7fa9380a36c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950103fb0 0x7fa9501955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9501999e0 0x7fa950199e90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 --2- 172.21.15.44:0/945896544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9500fee60 0x7fa9501950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 >> 172.21.15.44:0/945896544 conn(0x7fa950075c60 msgr2=0x7fa950102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 shutdown_connections 2024-09-13T17:29:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:29:32.112+0000 7fa9551bc640 1 -- 172.21.15.44:0/945896544 wait complete. 2024-09-13T17:29:32.124 INFO:tasks.cephfs.filesystem:scrub status for tag:e0ad5152-c79b-4719-bfbe-70162ed3b9cd - {'path': '/', 'tag': 'e0ad5152-c79b-4719-bfbe-70162ed3b9cd', 'options': 'recursive,force'} 2024-09-13T17:29:32.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:32 smithi100 ceph-mon[35055]: pgmap v87: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:32.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:32 smithi044 ceph-mon[31175]: pgmap v87: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:32.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:32 smithi070 ceph-mon[34502]: pgmap v87: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:33 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:29:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:33 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:33 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:29:33.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:33 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:33.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:33 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:29:33.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:33 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:33.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:33 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:29:33.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:33 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:33 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:29:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:33 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:33 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:29:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:33 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:29:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:34 smithi100 ceph-mon[35055]: pgmap v88: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:34.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:34 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:29:34.598 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:34 smithi070 ceph-mon[34502]: pgmap v88: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:34.598 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:34 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:29:34.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:34 smithi044 ceph-mon[31175]: pgmap v88: 129 pgs: 129 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:34.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:34 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:29:36.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:36 smithi100 ceph-mon[35055]: pgmap v89: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-09-13T17:29:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:36 smithi044 ceph-mon[31175]: pgmap v89: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-09-13T17:29:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:36 smithi070 ceph-mon[34502]: pgmap v89: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-09-13T17:29:38.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:38 smithi100 ceph-mon[35055]: pgmap v90: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-13T17:29:38.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:38 smithi044 ceph-mon[31175]: pgmap v90: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-13T17:29:38.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:38 smithi070 ceph-mon[34502]: pgmap v90: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-13T17:29:39.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:39 smithi044 ceph-mon[31175]: pgmap v91: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-09-13T17:29:39.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:39 smithi070 ceph-mon[34502]: pgmap v91: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-09-13T17:29:39.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:39 smithi100 ceph-mon[35055]: pgmap v91: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-09-13T17:29:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:41 smithi100 ceph-mon[35055]: pgmap v92: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-09-13T17:29:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:41 smithi044 ceph-mon[31175]: pgmap v92: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-09-13T17:29:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:41 smithi070 ceph-mon[34502]: pgmap v92: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-09-13T17:29:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 -- 172.21.15.44:0/3015210369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 msgr2=0x7f8d901050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/3015210369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d901050c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009900 tx=0x7f8d8002a410 comp rx=0 tx=0).stop 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 -- 172.21.15.44:0/3015210369 shutdown_connections 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/3015210369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d900681d0 0x7f8d9010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/3015210369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d90067830 0x7f8d90067c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/3015210369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d901050c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 -- 172.21.15.44:0/3015210369 >> 172.21.15.44:0/3015210369 conn(0x7f8d90074940 msgr2=0x7f8d90074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 -- 172.21.15.44:0/3015210369 shutdown_connections 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.700+0000 7f8d95fbc640 1 -- 172.21.15.44:0/3015210369 wait complete. 2024-09-13T17:29:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 Processor -- start 2024-09-13T17:29:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 -- start start 2024-09-13T17:29:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d90067830 0x7f8d90196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d900681d0 0x7f8d90196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d9019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019b180 con 0x7f8d90104ce0 2024-09-13T17:29:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019b2f0 con 0x7f8d90067830 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d95fbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019b4b0 con 0x7f8d900681d0 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d9019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d9019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58206/0 (socket says 172.21.15.44:58206) 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.701+0000 7f8d8ffff640 1 -- 172.21.15.44:0/91035030 learned_addr learned my addr 172.21.15.44:0/91035030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d90067830 0x7f8d90196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8effd640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d900681d0 0x7f8d90196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8ffff640 1 -- 172.21.15.44:0/91035030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d900681d0 msgr2=0x7f8d90196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8ffff640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d900681d0 0x7f8d90196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8ffff640 1 -- 172.21.15.44:0/91035030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d90067830 msgr2=0x7f8d90196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8ffff640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d90067830 0x7f8d90196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8ffff640 1 -- 172.21.15.44:0/91035030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d80009560 con 0x7f8d90104ce0 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8ffff640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d9019ab80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009fd0 tx=0x7f8d8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8cff9640 1 -- 172.21.15.44:0/91035030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800bea0 con 0x7f8d90104ce0 2024-09-13T17:29:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8cff9640 1 -- 172.21.15.44:0/91035030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d88009940 con 0x7f8d90104ce0 2024-09-13T17:29:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9019b790 con 0x7f8d90104ce0 2024-09-13T17:29:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d8cff9640 1 -- 172.21.15.44:0/91035030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88010480 con 0x7f8d90104ce0 2024-09-13T17:29:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.702+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d901a3010 con 0x7f8d90104ce0 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d8cff9640 1 -- 172.21.15.44:0/91035030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d88003970 con 0x7f8d90104ce0 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 Processor -- start 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 -- start start 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 0x55bc91a51710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc919bcc30 con 0x55bc91a51330 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc919ba3c0 con 0x55bc91a51c50 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc919b5cb0 con 0x55bc91a5c4f0 2024-09-13T17:29:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 0x55bc91a51710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d8cff9640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d780a1200 0x7f8d780a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 0x55bc91a51710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58216/0 (socket says 172.21.15.44:58216) 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.705+0000 7f8d8effd640 1 -- 172.21.15.44:0/1087256842 learned_addr learned my addr 172.21.15.44:0/1087256842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8ffff640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d780a1200 0x7f8d780a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8cff9640 1 -- 172.21.15.44:0/91035030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8d880ccc80 con 0x7f8d90104ce0 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1087256842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 msgr2=0x55bc91a5e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1087256842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 msgr2=0x55bc91a51710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 0x55bc91a51710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1087256842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9195c910 con 0x55bc91a51c50 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 0x55bc91a51710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5bfb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005ab0 tx=0x7f8d80005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6afd8512a88b295 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d8ffff640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d780a1200 0x7f8d780a36c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8d88099770 tx=0x7f8d8806cc00 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d75ffb640 1 -- 172.21.15.44:0/1087256842 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80004220 con 0x55bc91a51c50 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d75ffb640 1 -- 172.21.15.44:0/1087256842 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d80031df0 con 0x55bc91a51c50 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.706+0000 7f8d75ffb640 1 -- 172.21.15.44:0/1087256842 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80003bd0 con 0x55bc91a51c50 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1087256842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 msgr2=0x55bc91a5bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5bfb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005ab0 tx=0x7f8d80005c00 comp rx=0 tx=0).stop 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1087256842 shutdown_connections 2024-09-13T17:29:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5bfb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1087256842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51330 0x55bc91a51710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1087256842 >> 172.21.15.44:0/1087256842 conn(0x55bc91a4d8c0 msgr2=0x55bc91a4dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1087256842 shutdown_connections 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1087256842 wait complete. 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.707+0000 7f8d98dbc740 1 -- start start 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 Processor -- start 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51330 0x55bc91a5bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 0x55bc91a591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91973e80 con 0x55bc91a51c50 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91973b20 con 0x55bc91a51330 2024-09-13T17:29:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91939260 con 0x55bc91a5c4f0 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d98dbc740 10 client.0 _ll_register_callbacks cb 0x55bc9196edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 0x55bc91a591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 0x55bc91a591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58228/0 (socket says 172.21.15.44:58228) 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1878044034 learned_addr learned my addr 172.21.15.44:0/1878044034 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.708+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51330 0x55bc91a5bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 msgr2=0x55bc91a597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1878044034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51330 msgr2=0x55bc91a5bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51330 0x55bc91a5bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8f7fe640 1 -- 172.21.15.44:0/1878044034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9010d4c0 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 0x55bc91a591d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d80000c00 tx=0x7f8d8002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4450a453039f2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d6effd640 1 -- 172.21.15.44:0/1878044034 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8004b070 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d6effd640 1 -- 172.21.15.44:0/1878044034 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8d80031b70 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc919de680 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d6effd640 1 -- 172.21.15.44:0/1878044034 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d80004020 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d98dbc740 10 client.16539 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bc919b4330 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.709+0000 7f8d6effd640 1 -- 172.21.15.44:0/1878044034 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc919de680 con 0x55bc91a51c50 2024-09-13T17:29:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d6effd640 1 -- 172.21.15.44:0/1878044034 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d80043030 con 0x55bc91a51c50 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d98dbc740 10 client.16539 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d98dbc740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d98dbc740 20 client.16539 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91a6bc20 0x55bc91a6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d98dbc740 4 client.16539 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc919a95d0 con 0x55bc91a6bc20 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91a6bc20 0x55bc91a6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91a6bc20 0x55bc91a6e040 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19475262cdc6d533 server_cookie=7956582fc217d295 in_seq=0 out_seq=0 2024-09-13T17:29:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.710+0000 7f8d6effd640 10 client.16539 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d6effd640 1 -- 172.21.15.44:0/1878044034 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bc919a95d0 con 0x55bc91a6bc20 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d6effd640 10 client.16539 handle_command_reply: tid=0 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 client.16539 shutdown 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 msgr2=0x55bc91a591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 0x55bc91a591d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d80000c00 tx=0x7f8d8002a410 comp rx=0 tx=0).stop 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 shutdown_connections 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91a6bc20 0x55bc91a6e040 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a5c4f0 0x55bc91a597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a51c50 0x55bc91a591d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1878044034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51330 0x55bc91a5bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.711+0000 7f8d6effd640 0 client.16539 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.712+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 >> 172.21.15.44:0/1878044034 conn(0x55bc91a4d8c0 msgr2=0x55bc91a4cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.712+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 shutdown_connections 2024-09-13T17:29:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.712+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1878044034 wait complete. 2024-09-13T17:29:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.712+0000 7f8d98dbc740 20 client.16539 trim_cache size 0 max 16384 2024-09-13T17:29:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.724+0000 7f8d98dbc740 1 Processor -- start 2024-09-13T17:29:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 -- start start 2024-09-13T17:29:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a4d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a48f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9196ed60 con 0x55bc91a5c4f0 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91a494c0 con 0x55bc91a51c50 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91a57750 con 0x55bc91a51330 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a48f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a48f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58232/0 (socket says 172.21.15.44:58232) 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 -- 172.21.15.44:0/2448590862 learned_addr learned my addr 172.21.15.44:0/2448590862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a4d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 -- 172.21.15.44:0/2448590862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 msgr2=0x55bc91a4d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a4d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 -- 172.21.15.44:0/2448590862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 msgr2=0x55bc91a48a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a48a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 -- 172.21.15.44:0/2448590862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc91956300 con 0x55bc91a5c4f0 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.725+0000 7f8d8effd640 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a48f50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c00 tx=0x7f8d8400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286946a00d92f700 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d767fc640 1 -- 172.21.15.44:0/2448590862 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84012c50 con 0x55bc91a5c4f0 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d767fc640 1 -- 172.21.15.44:0/2448590862 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d8402b030 con 0x55bc91a5c4f0 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d767fc640 1 -- 172.21.15.44:0/2448590862 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84024900 con 0x55bc91a5c4f0 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 -- 172.21.15.44:0/2448590862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 msgr2=0x55bc91a48f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a48f50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c00 tx=0x7f8d8400b4c0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 -- 172.21.15.44:0/2448590862 shutdown_connections 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a48f50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/2448590862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 -- 172.21.15.44:0/2448590862 >> 172.21.15.44:0/2448590862 conn(0x55bc91a4d8c0 msgr2=0x55bc91a54510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 -- 172.21.15.44:0/2448590862 shutdown_connections 2024-09-13T17:29:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 -- 172.21.15.44:0/2448590862 wait complete. 2024-09-13T17:29:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.726+0000 7f8d98dbc740 1 -- start start 2024-09-13T17:29:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 Processor -- start 2024-09-13T17:29:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91a60570 con 0x55bc91a5c4f0 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91a606e0 con 0x55bc91a51c50 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc91ac2120 con 0x55bc91a51330 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d98dbc740 10 client.0 _ll_register_callbacks cb 0x7f8d80005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58246/0 (socket says 172.21.15.44:58246) 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 -- 172.21.15.44:0/1967281623 learned_addr learned my addr 172.21.15.44:0/1967281623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 -- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 msgr2=0x55bc91a634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 -- 172.21.15.44:0/1967281623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 msgr2=0x55bc91a5f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.727+0000 7f8d8effd640 1 -- 172.21.15.44:0/1967281623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d84014050 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d8effd640 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5fe70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400bc20 tx=0x7f8d84024620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85037938ba0b47d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d6effd640 1 -- 172.21.15.44:0/1967281623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84005220 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d80031df0 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d6effd640 1 -- 172.21.15.44:0/1967281623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8d84005380 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d6effd640 1 -- 172.21.15.44:0/1967281623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d84005540 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 10 client.16545 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bc9197b8e0 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d6effd640 1 -- 172.21.15.44:0/1967281623 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d80031df0 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d6effd640 1 -- 172.21.15.44:0/1967281623 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d840306d0 con 0x55bc91a5c4f0 2024-09-13T17:29:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 10 client.16545 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 20 client.16545 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91ac35e0 0x55bc91ac59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 4 client.16545 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.728+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc916a9300 con 0x55bc91ac35e0 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.729+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91ac35e0 0x55bc91ac59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.729+0000 7f8d6effd640 10 client.16545 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.729+0000 7f8d8f7fe640 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91ac35e0 0x55bc91ac59d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56de8b59fda4b5b8 server_cookie=33783e84e8c0d98a in_seq=0 out_seq=0 2024-09-13T17:29:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.740+0000 7f8d6effd640 1 -- 172.21.15.44:0/1967281623 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc916a9300 con 0x55bc91ac35e0 2024-09-13T17:29:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.740+0000 7f8d6effd640 10 client.16545 handle_command_reply: tid=0 2024-09-13T17:29:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 client.16545 shutdown 2024-09-13T17:29:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 msgr2=0x55bc91a5fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5fe70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400bc20 tx=0x7f8d84024620 comp rx=0 tx=0).stop 2024-09-13T17:29:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 shutdown_connections 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc91ac35e0 0x55bc91ac59d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc91a5c4f0 0x55bc91a5fe70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc91a51c50 0x55bc91a5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d6effd640 0 client.16545 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 --2- 172.21.15.44:0/1967281623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc91a51330 0x55bc91a634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 >> 172.21.15.44:0/1967281623 conn(0x55bc91a4d8c0 msgr2=0x55bc91a48a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 shutdown_connections 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 1 -- 172.21.15.44:0/1967281623 wait complete. 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d98dbc740 20 client.16545 trim_cache size 0 max 16384 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:29:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:29:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:29:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:29:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.741+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d780a1200 msgr2=0x7f8d780a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d780a1200 0x7f8d780a36c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8d88099770 tx=0x7f8d8806cc00 comp rx=0 tx=0).stop 2024-09-13T17:29:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 msgr2=0x7f8d9019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d9019ab80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009fd0 tx=0x7f8d8800b260 comp rx=0 tx=0).stop 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 shutdown_connections 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d780a1200 0x7f8d780a36c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d900681d0 0x7f8d90196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d90104ce0 0x7f8d9019ab80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 --2- 172.21.15.44:0/91035030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d90067830 0x7f8d90196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 >> 172.21.15.44:0/91035030 conn(0x7f8d90074940 msgr2=0x7f8d900fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 shutdown_connections 2024-09-13T17:29:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:43.742+0000 7f8d95fbc640 1 -- 172.21.15.44:0/91035030 wait complete. 2024-09-13T17:29:43.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:43 smithi100 ceph-mon[35055]: pgmap v93: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-09-13T17:29:43.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:43 smithi044 ceph-mon[31175]: pgmap v93: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-09-13T17:29:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:43 smithi070 ceph-mon[34502]: pgmap v93: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-09-13T17:29:45.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:45 smithi100 ceph-mon[35055]: pgmap v94: 129 pgs: 129 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-09-13T17:29:45.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:45 smithi044 ceph-mon[31175]: pgmap v94: 129 pgs: 129 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-09-13T17:29:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:45 smithi070 ceph-mon[34502]: pgmap v94: 129 pgs: 129 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-09-13T17:29:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:47 smithi100 ceph-mon[35055]: pgmap v95: 129 pgs: 129 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:47.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:47 smithi044 ceph-mon[31175]: pgmap v95: 129 pgs: 129 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:47 smithi070 ceph-mon[34502]: pgmap v95: 129 pgs: 129 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:29:49.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:49 smithi100 ceph-mon[35055]: pgmap v96: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-09-13T17:29:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:49 smithi044 ceph-mon[31175]: pgmap v96: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-09-13T17:29:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:49 smithi070 ceph-mon[34502]: pgmap v96: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-09-13T17:29:51.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:51 smithi100 ceph-mon[35055]: pgmap v97: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-13T17:29:51.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:51 smithi044 ceph-mon[31175]: pgmap v97: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-13T17:29:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:51 smithi070 ceph-mon[34502]: pgmap v97: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-13T17:29:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:53 smithi100 ceph-mon[35055]: pgmap v98: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-13T17:29:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:53 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:29:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:53 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:29:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:53 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:29:53.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:53 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:29:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:53 smithi044 ceph-mon[31175]: pgmap v98: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-13T17:29:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:53 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:29:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:53 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:29:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:53 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:29:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:53 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:29:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:53 smithi070 ceph-mon[34502]: pgmap v98: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-13T17:29:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:53 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:29:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:53 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:29:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:53 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:29:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:53 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:29:55.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:55 smithi100 ceph-mon[35055]: pgmap v99: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-13T17:29:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:55 smithi044 ceph-mon[31175]: pgmap v99: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-13T17:29:55.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:55 smithi070 ceph-mon[34502]: pgmap v99: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-13T17:29:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 -- 172.21.15.44:0/1563303509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44100ee0 msgr2=0x7fba441012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/1563303509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44100ee0 0x7fba441012c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fba40009900 tx=0x7fba4002a480 comp rx=0 tx=0).stop 2024-09-13T17:29:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 -- 172.21.15.44:0/1563303509 shutdown_connections 2024-09-13T17:29:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/1563303509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba44101800 0x7fba44109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/1563303509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba4410a2b0 0x7fba4410c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/1563303509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44100ee0 0x7fba441012c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 -- 172.21.15.44:0/1563303509 >> 172.21.15.44:0/1563303509 conn(0x7fba440fa8f0 msgr2=0x7fba440fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 -- 172.21.15.44:0/1563303509 shutdown_connections 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.841+0000 7fba4bc8b640 1 -- 172.21.15.44:0/1563303509 wait complete. 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 Processor -- start 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 -- start start 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba44100ee0 0x7fba440feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 0x7fba440ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba4410a2b0 0x7fba440ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba441a4580 con 0x7fba44101800 2024-09-13T17:29:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba441a46f0 con 0x7fba4410a2b0 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4bc8b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba441a48b0 con 0x7fba44100ee0 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba49226640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 0x7fba440ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba49226640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 0x7fba440ff070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52830/0 (socket says 172.21.15.44:52830) 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba49226640 1 -- 172.21.15.44:0/681469098 learned_addr learned my addr 172.21.15.44:0/681469098 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba49a27640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba44100ee0 0x7fba440feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.842+0000 7fba4a228640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba4410a2b0 0x7fba440ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba49226640 1 -- 172.21.15.44:0/681469098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba44100ee0 msgr2=0x7fba440feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba49226640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba44100ee0 0x7fba440feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba49226640 1 -- 172.21.15.44:0/681469098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba4410a2b0 msgr2=0x7fba440ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba49226640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba4410a2b0 0x7fba440ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba49226640 1 -- 172.21.15.44:0/681469098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40009560 con 0x7fba44101800 2024-09-13T17:29:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba49226640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 0x7fba440ff070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fba3400d950 tx=0x7fba3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba3affd640 1 -- 172.21.15.44:0/681469098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34009c30 con 0x7fba44101800 2024-09-13T17:29:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba441a4b90 con 0x7fba44101800 2024-09-13T17:29:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba3affd640 1 -- 172.21.15.44:0/681469098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba34009d90 con 0x7fba44101800 2024-09-13T17:29:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba3affd640 1 -- 172.21.15.44:0/681469098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400f620 con 0x7fba44101800 2024-09-13T17:29:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.843+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba441a50a0 con 0x7fba44101800 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba3affd640 1 -- 172.21.15.44:0/681469098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fba340109f0 con 0x7fba44101800 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 Processor -- start 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 -- start start 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087223c30 con 0x55c0872b8330 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872213c0 con 0x55c0872c34f0 2024-09-13T17:29:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.846+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08721ccb0 con 0x55c0872b8c50 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52842/0 (socket says 172.21.15.44:52842) 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 -- 172.21.15.44:0/2174093361 learned_addr learned my addr 172.21.15.44:0/2174093361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba49226640 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba3affd640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba300a1230 0x7fba300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba49a27640 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 -- 172.21.15.44:0/2174093361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 msgr2=0x55c0872c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 -- 172.21.15.44:0/2174093361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 msgr2=0x55c0872c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 -- 172.21.15.44:0/2174093361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0871c3910 con 0x55c0872b8330 2024-09-13T17:29:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba49226640 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba300a1230 0x7fba300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba3affd640 1 -- 172.21.15.44:0/681469098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fba340ccbb0 con 0x7fba44101800 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba4a228640 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872b8710 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fba3c011590 tx=0x7fba3c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7c08f3fe72e635 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.847+0000 7fba2ffff640 1 -- 172.21.15.44:0/2174093361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c01bb90 con 0x55c0872b8330 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba2ffff640 1 -- 172.21.15.44:0/2174093361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba3c01f070 con 0x55c0872b8330 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba2ffff640 1 -- 172.21.15.44:0/2174093361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c025cf0 con 0x55c0872b8330 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4a228640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba300a1230 0x7fba300a36f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba3c022000 tx=0x7fba3c023000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 -- 172.21.15.44:0/2174093361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 msgr2=0x55c0872b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872b8710 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fba3c011590 tx=0x7fba3c011a50 comp rx=0 tx=0).stop 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 -- 172.21.15.44:0/2174093361 shutdown_connections 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/2174093361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872b8710 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 -- 172.21.15.44:0/2174093361 >> 172.21.15.44:0/2174093361 conn(0x55c0872b48c0 msgr2=0x55c0872b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 -- 172.21.15.44:0/2174093361 shutdown_connections 2024-09-13T17:29:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.848+0000 7fba4ea8a740 1 -- 172.21.15.44:0/2174093361 wait complete. 2024-09-13T17:29:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 -- start start 2024-09-13T17:29:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 Processor -- start 2024-09-13T17:29:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872c34f0 0x55c0872c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0871dae80 con 0x55c0872b8330 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0871dab20 con 0x55c0872b8c50 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0871a0260 con 0x55c0872c34f0 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba49a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.849+0000 7fba4ea8a740 10 client.0 _ll_register_callbacks cb 0x55c0871d5df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52854/0 (socket says 172.21.15.44:52854) 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 -- 172.21.15.44:0/467735290 learned_addr learned my addr 172.21.15.44:0/467735290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49226640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba4a228640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872c34f0 0x55c0872c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 -- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872c34f0 msgr2=0x55c0872c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872c34f0 0x55c0872c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 -- 172.21.15.44:0/467735290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 msgr2=0x55c0872c0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872c0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 -- 172.21.15.44:0/467735290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c009c50 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba49a27640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2e60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fba40009230 tx=0x7fba40003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a786893bc51fd6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba2d7fa640 1 -- 172.21.15.44:0/467735290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40003b70 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba2d7fa640 1 -- 172.21.15.44:0/467735290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fba40003cd0 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.850+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c087245680 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba2d7fa640 1 -- 172.21.15.44:0/467735290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba40003e90 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 10 client.16557 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c08721b330 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba2d7fa640 1 -- 172.21.15.44:0/467735290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c087245680 con 0x55c0872b8330 2024-09-13T17:29:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba2d7fa640 1 -- 172.21.15.44:0/467735290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba4003d5e0 con 0x55c0872b8330 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 10 client.16557 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 20 client.16557 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0872d2de0 0x55c0872d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 4 client.16557 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0872105d0 con 0x55c0872d2de0 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.851+0000 7fba49226640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0872d2de0 0x55c0872d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.852+0000 7fba2d7fa640 10 client.16557 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.852+0000 7fba49226640 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0872d2de0 0x55c0872d51d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3247f942dd5c0d3e server_cookie=839d11e0eb8f5ef0 in_seq=0 out_seq=0 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.852+0000 7fba2d7fa640 1 -- 172.21.15.44:0/467735290 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c0872105d0 con 0x55c0872d2de0 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.852+0000 7fba2d7fa640 10 client.16557 handle_command_reply: tid=0 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.852+0000 7fba4ea8a740 1 client.16557 shutdown 2024-09-13T17:29:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 msgr2=0x55c0872c2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2e60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fba40009230 tx=0x7fba40003800 comp rx=0 tx=0).stop 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 shutdown_connections 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c0872d2de0 0x55c0872d51d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872c34f0 0x55c0872c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/467735290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2e60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 >> 172.21.15.44:0/467735290 conn(0x55c0872b48c0 msgr2=0x55c0872b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba2d7fa640 0 client.16557 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 shutdown_connections 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 1 -- 172.21.15.44:0/467735290 wait complete. 2024-09-13T17:29:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.853+0000 7fba4ea8a740 20 client.16557 trim_cache size 0 max 16384 2024-09-13T17:29:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.865+0000 7fba4ea8a740 1 Processor -- start 2024-09-13T17:29:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 -- start start 2024-09-13T17:29:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872b5970 con 0x55c0872b8330 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872cfb60 con 0x55c0872c34f0 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872b7b40 con 0x55c0872b8c50 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52860/0 (socket says 172.21.15.44:52860) 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 -- 172.21.15.44:0/1984194993 learned_addr learned my addr 172.21.15.44:0/1984194993 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4a228640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49226640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 -- 172.21.15.44:0/1984194993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 msgr2=0x55c0872ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 -- 172.21.15.44:0/1984194993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 msgr2=0x55c0872ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 -- 172.21.15.44:0/1984194993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0871bd300 con 0x55c0872b8330 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49226640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba4a228640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.866+0000 7fba49a27640 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fba40005ec0 tx=0x7fba400043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed71076379b3d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba38ff9640 1 -- 172.21.15.44:0/1984194993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40005850 con 0x55c0872b8330 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba38ff9640 1 -- 172.21.15.44:0/1984194993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba40041070 con 0x55c0872b8330 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba38ff9640 1 -- 172.21.15.44:0/1984194993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4003b630 con 0x55c0872b8330 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 -- 172.21.15.44:0/1984194993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 msgr2=0x55c0872c2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fba40005ec0 tx=0x7fba400043e0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 -- 172.21.15.44:0/1984194993 shutdown_connections 2024-09-13T17:29:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8c50 0x55c0872ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872c34f0 0x55c0872ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/1984194993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872b8330 0x55c0872c2970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 -- 172.21.15.44:0/1984194993 >> 172.21.15.44:0/1984194993 conn(0x55c0872b48c0 msgr2=0x55c0872afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 -- 172.21.15.44:0/1984194993 shutdown_connections 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 -- 172.21.15.44:0/1984194993 wait complete. 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.867+0000 7fba4ea8a740 1 -- start start 2024-09-13T17:29:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 Processor -- start 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8330 0x55c0872c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 0x55c0872c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872c8a50 con 0x55c0872c34f0 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872c8bc0 con 0x55c0872b8c50 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0872c8d80 con 0x55c0872b8330 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4ea8a740 10 client.0 _ll_register_callbacks cb 0x7fba3c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 0x55c0872c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 0x55c0872c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52870/0 (socket says 172.21.15.44:52870) 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 -- 172.21.15.44:0/966163128 learned_addr learned my addr 172.21.15.44:0/966163128 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:29:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba49a27640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8330 0x55c0872c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba49226640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 -- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8330 msgr2=0x55c0872c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8330 0x55c0872c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 -- 172.21.15.44:0/966163128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 msgr2=0x55c0872ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba4a228640 1 -- 172.21.15.44:0/966163128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c08721b330 con 0x55c0872c34f0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.868+0000 7fba49226640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4a228640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 0x55c0872c8370 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fba3c011590 tx=0x7fba3c004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f043d5722d4c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba2d7fa640 1 -- 172.21.15.44:0/966163128 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00f780 con 0x55c0872c34f0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba2d7fa640 1 -- 172.21.15.44:0/966163128 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fba3c01b990 con 0x55c0872c34f0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba3c01f070 con 0x55c0872c34f0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba2d7fa640 1 -- 172.21.15.44:0/966163128 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba3c01bb50 con 0x55c0872c34f0 2024-09-13T17:29:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 10 client.16563 fetch_fsmap learned FSMap version 25 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c0871e28e0 con 0x55c0872c34f0 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba2d7fa640 1 -- 172.21.15.44:0/966163128 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba3c01f070 con 0x55c0872c34f0 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba2d7fa640 1 -- 172.21.15.44:0/966163128 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba3c005040 con 0x55c0872c34f0 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 20 client.16563 populate_metadata read hostname 'smithi044' 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.869+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08732a420 0x55c08732c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.870+0000 7fba4ea8a740 4 client.16563 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.870+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c086f10300 con 0x55c08732a420 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.870+0000 7fba49a27640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08732a420 0x55c08732c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.870+0000 7fba49a27640 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08732a420 0x55c08732c840 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c32a2392fb03d2 server_cookie=59803f967f9d85d in_seq=0 out_seq=0 2024-09-13T17:29:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.870+0000 7fba2d7fa640 10 client.16563 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba2d7fa640 1 -- 172.21.15.44:0/966163128 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c086f10300 con 0x55c08732a420 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba2d7fa640 10 client.16563 handle_command_reply: tid=0 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 client.16563 shutdown 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 msgr2=0x55c0872c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 0x55c0872c8370 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fba3c011590 tx=0x7fba3c004f50 comp rx=0 tx=0).stop 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 shutdown_connections 2024-09-13T17:29:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08732a420 0x55c08732c840 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0872c34f0 0x55c0872c8370 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba2d7fa640 0 client.16563 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c0872b8c50 0x55c0872ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 --2- 172.21.15.44:0/966163128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0872b8330 0x55c0872c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.883+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 >> 172.21.15.44:0/966163128 conn(0x55c0872b48c0 msgr2=0x55c0872be850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 shutdown_connections 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4ea8a740 1 -- 172.21.15.44:0/966163128 wait complete. 2024-09-13T17:29:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4ea8a740 20 client.16563 trim_cache size 0 max 16384 2024-09-13T17:29:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:29:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:29:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:29:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba300a1230 msgr2=0x7fba300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba300a1230 0x7fba300a36f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba3c022000 tx=0x7fba3c023000 comp rx=0 tx=0).stop 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 msgr2=0x7fba440ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 0x7fba440ff070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fba3400d950 tx=0x7fba3400de10 comp rx=0 tx=0).stop 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 shutdown_connections 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba300a1230 0x7fba300a36f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba4410a2b0 0x7fba440ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba44101800 0x7fba440ff070 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 --2- 172.21.15.44:0/681469098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba44100ee0 0x7fba440feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:29:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 >> 172.21.15.44:0/681469098 conn(0x7fba440fa8f0 msgr2=0x7fba440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:29:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 shutdown_connections 2024-09-13T17:29:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:29:56.884+0000 7fba4bc8b640 1 -- 172.21.15.44:0/681469098 wait complete. 2024-09-13T17:29:57.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:57 smithi100 ceph-mon[35055]: pgmap v100: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-13T17:29:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:57 smithi044 ceph-mon[31175]: pgmap v100: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-13T17:29:57.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:57 smithi070 ceph-mon[34502]: pgmap v100: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-13T17:29:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:29:59 smithi100 ceph-mon[35055]: pgmap v101: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-09-13T17:29:59.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:29:59 smithi044 ceph-mon[31175]: pgmap v101: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-09-13T17:29:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:29:59 smithi070 ceph-mon[34502]: pgmap v101: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-09-13T17:30:00.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:00 smithi100 ceph-mon[35055]: overall HEALTH_OK 2024-09-13T17:30:00.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:00 smithi044 ceph-mon[31175]: overall HEALTH_OK 2024-09-13T17:30:00.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:00 smithi070 ceph-mon[34502]: overall HEALTH_OK 2024-09-13T17:30:01.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:01 smithi100 ceph-mon[35055]: pgmap v102: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-13T17:30:01.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:01 smithi044 ceph-mon[31175]: pgmap v102: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-13T17:30:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:01 smithi070 ceph-mon[34502]: pgmap v102: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-13T17:30:02.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 -- 172.21.15.44:0/3381353117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb884103fd0 msgr2=0x7fb88410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3381353117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb884103fd0 0x7fb88410bb20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009cd0 tx=0x7fb87c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 -- 172.21.15.44:0/3381353117 shutdown_connections 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3381353117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8840ff830 0x7fb884103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3381353117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb884103fd0 0x7fb88410bb20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3381353117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 0x7fb8840ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 -- 172.21.15.44:0/3381353117 >> 172.21.15.44:0/3381353117 conn(0x7fb8840fa930 msgr2=0x7fb8840fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 -- 172.21.15.44:0/3381353117 shutdown_connections 2024-09-13T17:30:02.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.245+0000 7fb8896e5640 1 -- 172.21.15.44:0/3381353117 wait complete. 2024-09-13T17:30:02.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 Processor -- start 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 -- start start 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 0x7fb884196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb884103fd0 0x7fb8841965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 0x7fb88419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88419b3b0 con 0x7fb88419a9c0 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88419b520 con 0x7fb8840fee80 2024-09-13T17:30:02.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.246+0000 7fb8896e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88419b690 con 0x7fb884103fd0 2024-09-13T17:30:02.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 0x7fb88419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 0x7fb88419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50980/0 (socket says 172.21.15.44:50980) 2024-09-13T17:30:02.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 -- 172.21.15.44:0/3967455369 learned_addr learned my addr 172.21.15.44:0/3967455369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8827fc640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb884103fd0 0x7fb8841965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb882ffd640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 0x7fb884196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 -- 172.21.15.44:0/3967455369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb884103fd0 msgr2=0x7fb8841965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb884103fd0 0x7fb8841965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 -- 172.21.15.44:0/3967455369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 msgr2=0x7fb884196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 0x7fb884196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 -- 172.21.15.44:0/3967455369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c009900 con 0x7fb88419a9c0 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8827fc640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb884103fd0 0x7fb8841965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb882ffd640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 0x7fb884196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:02.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8837fe640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 0x7fb88419ae70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb87c004af0 tx=0x7fb87c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb86bfff640 1 -- 172.21.15.44:0/3967455369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c003970 con 0x7fb88419a9c0 2024-09-13T17:30:02.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8841a2c70 con 0x7fb88419a9c0 2024-09-13T17:30:02.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb86bfff640 1 -- 172.21.15.44:0/3967455369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb87c004040 con 0x7fb88419a9c0 2024-09-13T17:30:02.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb86bfff640 1 -- 172.21.15.44:0/3967455369 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c045650 con 0x7fb88419a9c0 2024-09-13T17:30:02.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.247+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8841a3120 con 0x7fb88419a9c0 2024-09-13T17:30:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.249+0000 7fb86bfff640 1 -- 172.21.15.44:0/3967455369 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb87c00a040 con 0x7fb88419a9c0 2024-09-13T17:30:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 Processor -- start 2024-09-13T17:30:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 -- start start 2024-09-13T17:30:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5aff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5a69f20 con 0x55f8d5aff610 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5a676b0 con 0x55f8d5b0e3e0 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb88c403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5a62fa0 con 0x55f8d5afecf0 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb882ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb882ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50994/0 (socket says 172.21.15.44:50994) 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb882ffd640 1 -- 172.21.15.44:0/2541552404 learned_addr learned my addr 172.21.15.44:0/2541552404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb86bfff640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb86c0a1200 0x7fb86c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.250+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5aff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb882ffd640 1 -- 172.21.15.44:0/2541552404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 msgr2=0x55f8d5aff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5aff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb882ffd640 1 -- 172.21.15.44:0/2541552404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 msgr2=0x55f8d5b107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb8837fe640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb86c0a1200 0x7fb86c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb882ffd640 1 -- 172.21.15.44:0/2541552404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8d5a1d840 con 0x55f8d5aff610 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb86bfff640 1 -- 172.21.15.44:0/3967455369 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb87c0ee920 con 0x7fb88419a9c0 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0dea0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb87400bc50 tx=0x7fb87400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53d2a31b0499625 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb8697fa640 1 -- 172.21.15.44:0/2541552404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874017620 con 0x55f8d5aff610 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb8837fe640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb86c0a1200 0x7fb86c0a36c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0bb410 tx=0x7fb87c0db040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb8697fa640 1 -- 172.21.15.44:0/2541552404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb87400dec0 con 0x55f8d5aff610 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.251+0000 7fb8697fa640 1 -- 172.21.15.44:0/2541552404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400fb10 con 0x55f8d5aff610 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 -- 172.21.15.44:0/2541552404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 msgr2=0x55f8d5b0dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0dea0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb87400bc50 tx=0x7fb87400d710 comp rx=0 tx=0).stop 2024-09-13T17:30:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 -- 172.21.15.44:0/2541552404 shutdown_connections 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0dea0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 --2- 172.21.15.44:0/2541552404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5aff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 -- 172.21.15.44:0/2541552404 >> 172.21.15.44:0/2541552404 conn(0x55f8d5afb380 msgr2=0x55f8d5afb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 -- 172.21.15.44:0/2541552404 shutdown_connections 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 -- 172.21.15.44:0/2541552404 wait complete. 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 -- start start 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.252+0000 7fb88c403740 1 Processor -- start 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5b0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b0b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5a042a0 con 0x55f8d5aff610 2024-09-13T17:30:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d573b430 con 0x55f8d5b0e3e0 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d59e6680 con 0x55f8d5afecf0 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 10 client.0 _ll_register_callbacks cb 0x55f8d5a5c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50996/0 (socket says 172.21.15.44:50996) 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 -- 172.21.15.44:0/3628528065 learned_addr learned my addr 172.21.15.44:0/3628528065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5b0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb8827fc640 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b0b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 -- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 msgr2=0x55f8d5b0de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5b0de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 -- 172.21.15.44:0/3628528065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 msgr2=0x55f8d5b0b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b0b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 -- 172.21.15.44:0/3628528065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87400b850 con 0x55f8d5aff610 2024-09-13T17:30:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb882ffd640 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0b090 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb87400b390 tx=0x7fb87400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bbc19e3e5bd140 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb866ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb866ff5640 1 -- 172.21.15.44:0/3628528065 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874005050 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb866ff5640 1 -- 172.21.15.44:0/3628528065 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb874013070 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.253+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8d57677e0 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb866ff5640 1 -- 172.21.15.44:0/3628528065 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8740051b0 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 10 client.16575 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8d5a61620 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb866ff5640 1 -- 172.21.15.44:0/3628528065 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f8d57677e0 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb866ff5640 1 -- 172.21.15.44:0/3628528065 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb87401f070 con 0x55f8d5aff610 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 10 client.16575 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 20 client.16575 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b1e230 0x55f8d5b20650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 4 client.16575 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8d57bae30 con 0x55f8d5b1e230 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.254+0000 7fb8827fc640 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b1e230 0x55f8d5b20650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.255+0000 7fb8827fc640 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b1e230 0x55f8d5b20650 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb509510e3205 server_cookie=8c1b8a5797c31f0d in_seq=0 out_seq=0 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.255+0000 7fb866ff5640 10 client.16575 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.255+0000 7fb866ff5640 1 -- 172.21.15.44:0/3628528065 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f8d57bae30 con 0x55f8d5b1e230 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.255+0000 7fb866ff5640 10 client.16575 handle_command_reply: tid=0 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.255+0000 7fb88c403740 1 client.16575 shutdown 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 msgr2=0x55f8d5b0b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0b090 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb87400b390 tx=0x7fb87400fe60 comp rx=0 tx=0).stop 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 shutdown_connections 2024-09-13T17:30:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b1e230 0x55f8d5b20650 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b0b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5aff610 0x55f8d5b0b090 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb866ff5640 0 client.16575 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 --2- 172.21.15.44:0/3628528065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5afecf0 0x55f8d5b0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 >> 172.21.15.44:0/3628528065 conn(0x55f8d5afb380 msgr2=0x55f8d5b010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 shutdown_connections 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 1 -- 172.21.15.44:0/3628528065 wait complete. 2024-09-13T17:30:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.256+0000 7fb88c403740 20 client.16575 trim_cache size 0 max 16384 2024-09-13T17:30:02.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 Processor -- start 2024-09-13T17:30:02.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 -- start start 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5afad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5aff610 0x55f8d5b06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d57677e0 con 0x55f8d5afecf0 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5b10b50 con 0x55f8d5b0e3e0 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.270+0000 7fb88c403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5af6760 con 0x55f8d5aff610 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5afad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5afad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51000/0 (socket says 172.21.15.44:51000) 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 -- 172.21.15.44:0/2918018428 learned_addr learned my addr 172.21.15.44:0/2918018428 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5aff610 0x55f8d5b06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 -- 172.21.15.44:0/2918018428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5aff610 msgr2=0x55f8d5b06360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5aff610 0x55f8d5b06360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 -- 172.21.15.44:0/2918018428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 msgr2=0x55f8d5b068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 -- 172.21.15.44:0/2918018428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8d5a03790 con 0x55f8d5afecf0 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5afad00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb878005fd0 tx=0x7fb87800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5280477b7f1284 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb869ffb640 1 -- 172.21.15.44:0/2918018428 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878026070 con 0x55f8d5afecf0 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb869ffb640 1 -- 172.21.15.44:0/2918018428 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb878012dc0 con 0x55f8d5afecf0 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.271+0000 7fb869ffb640 1 -- 172.21.15.44:0/2918018428 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878018040 con 0x55f8d5afecf0 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- 172.21.15.44:0/2918018428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 msgr2=0x55f8d5afad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5afad00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb878005fd0 tx=0x7fb87800ead0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- 172.21.15.44:0/2918018428 shutdown_connections 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5aff610 0x55f8d5b06360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- 172.21.15.44:0/2918018428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5afad00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- 172.21.15.44:0/2918018428 >> 172.21.15.44:0/2918018428 conn(0x55f8d5afb380 msgr2=0x55f8d5afb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- 172.21.15.44:0/2918018428 shutdown_connections 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- 172.21.15.44:0/2918018428 wait complete. 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- start start 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 Processor -- start 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5b11470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5aff610 0x55f8d5b119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5b73c60 con 0x55f8d5afecf0 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5b73da0 con 0x55f8d5aff610 2024-09-13T17:30:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.272+0000 7fb88c403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d5b73f60 con 0x55f8d5b0e3e0 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb88c403740 10 client.0 _ll_register_callbacks cb 0x7fb87800e970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5b11470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5b11470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51012/0 (socket says 172.21.15.44:51012) 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 -- 172.21.15.44:0/2466010936 learned_addr learned my addr 172.21.15.44:0/2466010936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5aff610 0x55f8d5b119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 -- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5b0e3e0 msgr2=0x55f8d5b73720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b73720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 -- 172.21.15.44:0/2466010936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5aff610 msgr2=0x55f8d5b119b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5aff610 0x55f8d5b119b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 -- 172.21.15.44:0/2466010936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87801b040 con 0x55f8d5afecf0 2024-09-13T17:30:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5aff610 0x55f8d5b119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb8827fc640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5b11470 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb878011dc0 tx=0x7fb878004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9212e98d66fd4cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb866ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb866ff5640 1 -- 172.21.15.44:0/2466010936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878026070 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb866ff5640 1 -- 172.21.15.44:0/2466010936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb878022400 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8d57564e0 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb866ff5640 1 -- 172.21.15.44:0/2466010936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb878018040 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb88c403740 10 client.16581 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.273+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8d5a28ea0 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb866ff5640 1 -- 172.21.15.44:0/2466010936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f8d57564e0 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb866ff5640 1 -- 172.21.15.44:0/2466010936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8780046c0 con 0x55f8d5afecf0 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb88c403740 10 client.16581 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb88c403740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb88c403740 20 client.16581 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb88c403740 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b75690 0x55f8d5b77ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb88c403740 4 client.16581 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8d5b78300 con 0x55f8d5b75690 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b75690 0x55f8d5b77ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb882ffd640 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b75690 0x55f8d5b77ab0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807c20fca8c50de server_cookie=75381f771a54bde1 in_seq=0 out_seq=0 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.274+0000 7fb866ff5640 10 client.16581 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb866ff5640 1 -- 172.21.15.44:0/2466010936 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f8d5b78300 con 0x55f8d5b75690 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb866ff5640 10 client.16581 handle_command_reply: tid=0 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 client.16581 shutdown 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 msgr2=0x55f8d5b11470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5b11470 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb878011dc0 tx=0x7fb878004460 comp rx=0 tx=0).stop 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 shutdown_connections 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8d5b75690 0x55f8d5b77ab0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8d5b0e3e0 0x55f8d5b73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8d5aff610 0x55f8d5b119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 --2- 172.21.15.44:0/2466010936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8d5afecf0 0x55f8d5b11470 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb866ff5640 0 client.16581 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 >> 172.21.15.44:0/2466010936 conn(0x55f8d5afb380 msgr2=0x55f8d5afb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 shutdown_connections 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 1 -- 172.21.15.44:0/2466010936 wait complete. 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb88c403740 20 client.16581 trim_cache size 0 max 16384 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:30:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:30:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:30:02.285 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:30:02.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb86c0a1200 msgr2=0x7fb86c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb86c0a1200 0x7fb86c0a36c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0bb410 tx=0x7fb87c0db040 comp rx=0 tx=0).stop 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 msgr2=0x7fb88419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 0x7fb88419ae70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb87c004af0 tx=0x7fb87c030ec0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 shutdown_connections 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb86c0a1200 0x7fb86c0a36c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb884103fd0 0x7fb8841965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb88419a9c0 0x7fb88419ae70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 --2- 172.21.15.44:0/3967455369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840fee80 0x7fb884196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.275+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 >> 172.21.15.44:0/3967455369 conn(0x7fb8840fa930 msgr2=0x7fb8840fbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.276+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 shutdown_connections 2024-09-13T17:30:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.276+0000 7fb8896e5640 1 -- 172.21.15.44:0/3967455369 wait complete. 2024-09-13T17:30:02.288 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:30:02.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:30:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 -- 172.21.15.44:0/3623362450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c0ff840 msgr2=0x7efd6c103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 --2- 172.21.15.44:0/3623362450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c0ff840 0x7efd6c103880 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efd60009930 tx=0x7efd6002a420 comp rx=0 tx=0).stop 2024-09-13T17:30:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 -- 172.21.15.44:0/3623362450 shutdown_connections 2024-09-13T17:30:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 --2- 172.21.15.44:0/3623362450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 0x7efd6c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 --2- 172.21.15.44:0/3623362450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c0ff840 0x7efd6c103880 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 --2- 172.21.15.44:0/3623362450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0fee90 0x7efd6c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 -- 172.21.15.44:0/3623362450 >> 172.21.15.44:0/3623362450 conn(0x7efd6c0fa900 msgr2=0x7efd6c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 -- 172.21.15.44:0/3623362450 shutdown_connections 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 -- 172.21.15.44:0/3623362450 wait complete. 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 Processor -- start 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.391+0000 7efd72db5640 1 -- start start 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0fee90 0x7efd6c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 0x7efd6c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 0x7efd6c1a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1a2820 con 0x7efd6c197b80 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1a2990 con 0x7efd6c103fe0 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1a2b50 con 0x7efd6c0fee90 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 0x7efd6c1a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 0x7efd6c1a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51032/0 (socket says 172.21.15.44:51032) 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 -- 172.21.15.44:0/418370642 learned_addr learned my addr 172.21.15.44:0/418370642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd70b51640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0fee90 0x7efd6c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd6bfff640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 0x7efd6c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 -- 172.21.15.44:0/418370642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0fee90 msgr2=0x7efd6c197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0fee90 0x7efd6c197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 -- 172.21.15.44:0/418370642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 msgr2=0x7efd6c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 0x7efd6c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 -- 172.21.15.44:0/418370642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd60009560 con 0x7efd6c197b80 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd6bfff640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 0x7efd6c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd71352640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 0x7efd6c1a2180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efd6400eb10 tx=0x7efd6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64004b20 con 0x7efd6c197b80 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd64004c80 con 0x7efd6c197b80 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c1a2e30 con 0x7efd6c197b80 2024-09-13T17:30:02.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.392+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64005a10 con 0x7efd6c197b80 2024-09-13T17:30:02.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.393+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1a32b0 con 0x7efd6c197b80 2024-09-13T17:30:02.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.394+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c100360 con 0x7efd6c197b80 2024-09-13T17:30:02.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.395+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efd64005020 con 0x7efd6c197b80 2024-09-13T17:30:02.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.395+0000 7efd69ffb640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efd540a1280 0x7efd540a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.396+0000 7efd70b51640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efd540a1280 0x7efd540a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.396+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efd64014070 con 0x7efd6c197b80 2024-09-13T17:30:02.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.396+0000 7efd70b51640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efd540a1280 0x7efd540a3740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009bd0 tx=0x7efd5c005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.400+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efd64096650 con 0x7efd6c197b80 2024-09-13T17:30:02.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.608+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd6c0ff270 con 0x7efd6c197b80 2024-09-13T17:30:02.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.609+0000 7efd69ffb640 1 -- 172.21.15.44:0/418370642 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7efd64099c70 con 0x7efd6c197b80 2024-09-13T17:30:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:30:02.610 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:30:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efd540a1280 msgr2=0x7efd540a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efd540a1280 0x7efd540a3740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009bd0 tx=0x7efd5c005f90 comp rx=0 tx=0).stop 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 msgr2=0x7efd6c1a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 0x7efd6c1a2180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efd6400eb10 tx=0x7efd6400c2d0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 shutdown_connections 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efd540a1280 0x7efd540a3740 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd6c197b80 0x7efd6c1a2180 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c103fe0 0x7efd6c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 --2- 172.21.15.44:0/418370642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0fee90 0x7efd6c197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.611+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 >> 172.21.15.44:0/418370642 conn(0x7efd6c0fa900 msgr2=0x7efd6c0fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.612+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 shutdown_connections 2024-09-13T17:30:02.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.612+0000 7efd72db5640 1 -- 172.21.15.44:0/418370642 wait complete. 2024-09-13T17:30:02.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:30:02.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 -- 172.21.15.44:0/2285135119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e40fee60 msgr2=0x7f75e40ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 --2- 172.21.15.44:0/2285135119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e40fee60 0x7f75e40ff240 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009930 tx=0x7f75d402a420 comp rx=0 tx=0).stop 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 -- 172.21.15.44:0/2285135119 shutdown_connections 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 --2- 172.21.15.44:0/2285135119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e4103fb0 0x7f75e410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 --2- 172.21.15.44:0/2285135119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e40ff810 0x7f75e4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 --2- 172.21.15.44:0/2285135119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e40fee60 0x7f75e40ff240 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 -- 172.21.15.44:0/2285135119 >> 172.21.15.44:0/2285135119 conn(0x7f75e4075c60 msgr2=0x7f75e40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 -- 172.21.15.44:0/2285135119 shutdown_connections 2024-09-13T17:30:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.727+0000 7f75e91db640 1 -- 172.21.15.44:0/2285135119 wait complete. 2024-09-13T17:30:02.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 Processor -- start 2024-09-13T17:30:02.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 -- start start 2024-09-13T17:30:02.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e40fee60 0x7f75e410c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e4103fb0 0x7f75e410c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 0x7f75e41a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e41a49f0 con 0x7f75e410ce90 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e41a4b60 con 0x7f75e4103fb0 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.728+0000 7f75e91db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e41a4d20 con 0x7f75e40fee60 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 0x7f75e41a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 0x7f75e41a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51056/0 (socket says 172.21.15.44:51056) 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 -- 172.21.15.44:0/3257949087 learned_addr learned my addr 172.21.15.44:0/3257949087 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e40fee60 0x7f75e410c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e259c640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e4103fb0 0x7f75e410c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 -- 172.21.15.44:0/3257949087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e40fee60 msgr2=0x7f75e410c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e40fee60 0x7f75e410c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 -- 172.21.15.44:0/3257949087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e4103fb0 msgr2=0x7f75e410c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e4103fb0 0x7f75e410c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 -- 172.21.15.44:0/3257949087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009560 con 0x7f75e410ce90 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e40fee60 0x7f75e410c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e259c640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e4103fb0 0x7f75e410c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:02.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e359e640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 0x7f75e41a4420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b9e0 tx=0x7f75dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75dbfff640 1 -- 172.21.15.44:0/3257949087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00caa0 con 0x7f75e410ce90 2024-09-13T17:30:02.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75dbfff640 1 -- 172.21.15.44:0/3257949087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75dc00cc00 con 0x7f75e410ce90 2024-09-13T17:30:02.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e41a4fa0 con 0x7f75e410ce90 2024-09-13T17:30:02.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75dbfff640 1 -- 172.21.15.44:0/3257949087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0075d0 con 0x7f75e410ce90 2024-09-13T17:30:02.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.729+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e41a5480 con 0x7f75e410ce90 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75dbfff640 1 -- 172.21.15.44:0/3257949087 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f75dc014440 con 0x7f75e410ce90 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 Processor -- start 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 -- start start 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4571b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e457ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e44dbcc0 con 0x5640e45720b0 2024-09-13T17:30:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e44d9450 con 0x5640e4571790 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e44d4d40 con 0x5640e457c950 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51068/0 (socket says 172.21.15.44:51068) 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.732+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3295195919 learned_addr learned my addr 172.21.15.44:0/3295195919 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75dbfff640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f75d00a1230 0x7f75d00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e259c640 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4571b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e259c640 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e457ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e359e640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f75d00a1230 0x7f75d00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3295195919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 msgr2=0x5640e457ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e457ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3295195919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 msgr2=0x5640e4571b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4571b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3295195919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e449c620 con 0x5640e45720b0 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75dbfff640 1 -- 172.21.15.44:0/3257949087 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f75dc0cc980 con 0x7f75e410ce90 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e259c640 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4571b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c410 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005940 tx=0x7f75d4031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d668d5dfa33797 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75e359e640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f75d00a1230 0x7f75d00a36f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0c9f70 tx=0x7f75dc08efd0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.733+0000 7f75d97fa640 1 -- 172.21.15.44:0/3295195919 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40042c0 con 0x5640e45720b0 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75d97fa640 1 -- 172.21.15.44:0/3295195919 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d4031df0 con 0x5640e45720b0 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75d97fa640 1 -- 172.21.15.44:0/3295195919 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4003cb0 con 0x5640e45720b0 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3295195919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 msgr2=0x5640e457c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c410 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005940 tx=0x7f75d4031730 comp rx=0 tx=0).stop 2024-09-13T17:30:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3295195919 shutdown_connections 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e457ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c410 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3295195919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4571b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3295195919 >> 172.21.15.44:0/3295195919 conn(0x5640e456dd00 msgr2=0x5640e456e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3295195919 shutdown_connections 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.734+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3295195919 wait complete. 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 -- start start 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 Processor -- start 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4571790 0x5640e457beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e457c950 0x5640e4579420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e4493180 con 0x5640e45720b0 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e41ad430 con 0x5640e457c950 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e44585e0 con 0x5640e4571790 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75ebfd8740 10 client.0 _ll_register_callbacks cb 0x5640e44c4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51076/0 (socket says 172.21.15.44:51076) 2024-09-13T17:30:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3303091160 learned_addr learned my addr 172.21.15.44:0/3303091160 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75e259c640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e457c950 0x5640e4579420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.735+0000 7f75e259c640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4571790 0x5640e457beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3303091160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4571790 msgr2=0x5640e457beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4571790 0x5640e457beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e457c950 msgr2=0x5640e4579420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e457c950 0x5640e4579420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e2d9d640 1 -- 172.21.15.44:0/3303091160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e40675a0 con 0x5640e45720b0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e259c640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4571790 0x5640e457beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c3f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f75d402aa10 tx=0x7f75d40318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be08cbd76ee0586 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ca7fc640 1 -- 172.21.15.44:0/3303091160 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d404b070 con 0x5640e45720b0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ca7fc640 1 -- 172.21.15.44:0/3303091160 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f75d40042d0 con 0x5640e45720b0 2024-09-13T17:30:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640e44feb80 con 0x5640e45720b0 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ca7fc640 1 -- 172.21.15.44:0/3303091160 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d4003830 con 0x5640e45720b0 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 10 client.16599 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5640e44d33c0 con 0x5640e45720b0 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ca7fc640 1 -- 172.21.15.44:0/3303091160 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5640e44feb80 con 0x5640e45720b0 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ca7fc640 1 -- 172.21.15.44:0/3303091160 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75d4045970 con 0x5640e45720b0 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 10 client.16599 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 10 client.16599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 20 client.16599 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e458c050 0x5640e458e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 4 client.16599 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.736+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640e44c71f0 con 0x5640e458c050 2024-09-13T17:30:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.737+0000 7f75e259c640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e458c050 0x5640e458e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.737+0000 7f75e259c640 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e458c050 0x5640e458e440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a7becdc7209a6d5 server_cookie=36839f3d1c8a008 in_seq=0 out_seq=0 2024-09-13T17:30:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.737+0000 7f75ca7fc640 10 client.16599 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ca7fc640 1 -- 172.21.15.44:0/3303091160 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5640e44c71f0 con 0x5640e458c050 2024-09-13T17:30:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ca7fc640 10 client.16599 handle_command_reply: tid=0 2024-09-13T17:30:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 client.16599 shutdown 2024-09-13T17:30:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 msgr2=0x5640e457c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c3f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f75d402aa10 tx=0x7f75d40318a0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 shutdown_connections 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e458c050 0x5640e458e440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e457c950 0x5640e4579420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e45720b0 0x5640e457c3f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ca7fc640 0 client.16599 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/3303091160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4571790 0x5640e457beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 >> 172.21.15.44:0/3303091160 conn(0x5640e456dd00 msgr2=0x5640e4573a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 shutdown_connections 2024-09-13T17:30:02.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 1 -- 172.21.15.44:0/3303091160 wait complete. 2024-09-13T17:30:02.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.738+0000 7f75ebfd8740 20 client.16599 trim_cache size 0 max 16384 2024-09-13T17:30:02.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.749+0000 7f75ebfd8740 1 Processor -- start 2024-09-13T17:30:02.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.749+0000 7f75ebfd8740 1 -- start start 2024-09-13T17:30:02.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4583650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e45720b0 0x5640e4583b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 0x5640e4574360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e457f400 con 0x5640e457c950 2024-09-13T17:30:02.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e456e900 con 0x5640e4571790 2024-09-13T17:30:02.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e4578ff0 con 0x5640e45720b0 2024-09-13T17:30:02.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 0x5640e4574360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 0x5640e4574360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51088/0 (socket says 172.21.15.44:51088) 2024-09-13T17:30:02.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 -- 172.21.15.44:0/4084790494 learned_addr learned my addr 172.21.15.44:0/4084790494 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4583650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e45720b0 0x5640e4583b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 -- 172.21.15.44:0/4084790494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e45720b0 msgr2=0x5640e4583b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e45720b0 0x5640e4583b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 -- 172.21.15.44:0/4084790494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 msgr2=0x5640e4583650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4583650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 -- 172.21.15.44:0/4084790494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e4475770 con 0x5640e457c950 2024-09-13T17:30:02.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75e259c640 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 0x5640e4574360 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00dfd0 tx=0x7f75cc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241a5552e94689ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75d9ffb640 1 -- 172.21.15.44:0/4084790494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00acd0 con 0x5640e457c950 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75d9ffb640 1 -- 172.21.15.44:0/4084790494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75cc019040 con 0x5640e457c950 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.750+0000 7f75d9ffb640 1 -- 172.21.15.44:0/4084790494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc004a90 con 0x5640e457c950 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 -- 172.21.15.44:0/4084790494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 msgr2=0x5640e4574360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 0x5640e4574360 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00dfd0 tx=0x7f75cc005d20 comp rx=0 tx=0).stop 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 -- 172.21.15.44:0/4084790494 shutdown_connections 2024-09-13T17:30:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e45720b0 0x5640e4583b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e457c950 0x5640e4574360 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/4084790494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e4571790 0x5640e4583650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 -- 172.21.15.44:0/4084790494 >> 172.21.15.44:0/4084790494 conn(0x5640e456dd00 msgr2=0x5640e4592da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 -- 172.21.15.44:0/4084790494 shutdown_connections 2024-09-13T17:30:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 -- 172.21.15.44:0/4084790494 wait complete. 2024-09-13T17:30:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 -- start start 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.751+0000 7f75ebfd8740 1 Processor -- start 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 0x5640e45818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e45720b0 0x5640e4581e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e45e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e45e1a60 con 0x5640e4571790 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e45e1ba0 con 0x5640e45720b0 2024-09-13T17:30:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e45e1ce0 con 0x5640e457c950 2024-09-13T17:30:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ebfd8740 10 client.0 _ll_register_callbacks cb 0x7f75d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 0x5640e45818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 0x5640e45818c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51098/0 (socket says 172.21.15.44:51098) 2024-09-13T17:30:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 -- 172.21.15.44:0/2637955574 learned_addr learned my addr 172.21.15.44:0/2637955574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e45720b0 0x5640e4581e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e45e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 -- 172.21.15.44:0/2637955574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 msgr2=0x5640e45e1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e45e1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 -- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e45720b0 msgr2=0x5640e4581e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e45720b0 0x5640e4581e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 -- 172.21.15.44:0/2637955574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc01f2d0 con 0x5640e4571790 2024-09-13T17:30:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75e259c640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 0x5640e45818c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005390 tx=0x7f75cc005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8546393f75d55a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.752+0000 7f75ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ca7fc640 1 -- 172.21.15.44:0/2637955574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0046c0 con 0x5640e4571790 2024-09-13T17:30:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ca7fc640 1 -- 172.21.15.44:0/2637955574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f75cc000e40 con 0x5640e4571790 2024-09-13T17:30:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75d4031df0 con 0x5640e4571790 2024-09-13T17:30:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ca7fc640 1 -- 172.21.15.44:0/2637955574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75cc004020 con 0x5640e4571790 2024-09-13T17:30:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 10 client.16605 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5640e449ae80 con 0x5640e4571790 2024-09-13T17:30:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ca7fc640 1 -- 172.21.15.44:0/2637955574 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75cc035080 con 0x5640e4571790 2024-09-13T17:30:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ca7fc640 1 -- 172.21.15.44:0/2637955574 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75cc032020 con 0x5640e4571790 2024-09-13T17:30:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 10 client.16605 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 10 client.16605 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 20 client.16605 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e45e3330 0x5640e45e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 4 client.16605 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5640e41c84e0 con 0x5640e45e3330 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.753+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e45e3330 0x5640e45e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ca7fc640 10 client.16605 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75e2d9d640 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e45e3330 0x5640e45e5750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99d65d6129c24c07 server_cookie=1a481ee700a98062 in_seq=0 out_seq=0 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ca7fc640 1 -- 172.21.15.44:0/2637955574 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5640e41c84e0 con 0x5640e45e3330 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ca7fc640 10 client.16605 handle_command_reply: tid=0 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 client.16605 shutdown 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 msgr2=0x5640e45818c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 0x5640e45818c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005390 tx=0x7f75cc005fa0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 shutdown_connections 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5640e45e3330 0x5640e45e5750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e457c950 0x5640e45e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640e45720b0 0x5640e4581e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 --2- 172.21.15.44:0/2637955574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640e4571790 0x5640e45818c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ca7fc640 0 client.16605 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 >> 172.21.15.44:0/2637955574 conn(0x5640e456dd00 msgr2=0x5640e4574290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 shutdown_connections 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 1 -- 172.21.15.44:0/2637955574 wait complete. 2024-09-13T17:30:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.754+0000 7f75ebfd8740 20 client.16605 trim_cache size 0 max 16384 2024-09-13T17:30:02.774 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:30:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f75d00a1230 msgr2=0x7f75d00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f75d00a1230 0x7f75d00a36f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0c9f70 tx=0x7f75dc08efd0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 msgr2=0x7f75e41a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 0x7f75e41a4420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b9e0 tx=0x7f75dc00bea0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 shutdown_connections 2024-09-13T17:30:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f75d00a1230 0x7f75d00a36f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e410ce90 0x7f75e41a4420 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75e4103fb0 0x7f75e410c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 --2- 172.21.15.44:0/3257949087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e40fee60 0x7f75e410c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 >> 172.21.15.44:0/3257949087 conn(0x7f75e4075c60 msgr2=0x7f75e4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 shutdown_connections 2024-09-13T17:30:02.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.755+0000 7f75e91db640 1 -- 172.21.15.44:0/3257949087 wait complete. 2024-09-13T17:30:02.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 -- 172.21.15.44:0/246392582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0ff810 msgr2=0x7fa3fc103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 --2- 172.21.15.44:0/246392582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0ff810 0x7fa3fc103850 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009930 tx=0x7fa3e402a420 comp rx=0 tx=0).stop 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 -- 172.21.15.44:0/246392582 shutdown_connections 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 --2- 172.21.15.44:0/246392582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 --2- 172.21.15.44:0/246392582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0ff810 0x7fa3fc103850 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 --2- 172.21.15.44:0/246392582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 -- 172.21.15.44:0/246392582 >> 172.21.15.44:0/246392582 conn(0x7fa3fc075c60 msgr2=0x7fa3fc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 -- 172.21.15.44:0/246392582 shutdown_connections 2024-09-13T17:30:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 -- 172.21.15.44:0/246392582 wait complete. 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 Processor -- start 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.882+0000 7fa40119c640 1 -- start start 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa40119c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa40119c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa40119c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3fc19abe0 0x7fa3fc19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa40119c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc19b5d0 con 0x7fa3fc0fee60 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa40119c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc19b740 con 0x7fa3fc19abe0 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa40119c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc19b8b0 con 0x7fa3fc103fb0 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51124/0 (socket says 172.21.15.44:51124) 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3811663679 learned_addr learned my addr 172.21.15.44:0/3811663679 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3fc19abe0 0x7fa3fc19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3811663679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3fc103fb0 msgr2=0x7fa3fc1967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc1967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3811663679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3fc19abe0 msgr2=0x7fa3fc19b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3fc19abe0 0x7fa3fc19b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3811663679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009560 con 0x7fa3fc0fee60 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc1962b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00d770 tx=0x7fa3ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3f8d99640 1 -- 172.21.15.44:0/3811663679 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec009c30 con 0x7fa3fc0fee60 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.883+0000 7fa3f8d99640 1 -- 172.21.15.44:0/3811663679 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec00de80 con 0x7fa3fc0fee60 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.884+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc1a2e90 con 0x7fa3fc0fee60 2024-09-13T17:30:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.884+0000 7fa3f8d99640 1 -- 172.21.15.44:0/3811663679 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec004d40 con 0x7fa3fc0fee60 2024-09-13T17:30:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.884+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1a3200 con 0x7fa3fc0fee60 2024-09-13T17:30:02.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.886+0000 7fa3f8d99640 1 -- 172.21.15.44:0/3811663679 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa3ec004ea0 con 0x7fa3fc0fee60 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.886+0000 7fa403eb8740 1 Processor -- start 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.886+0000 7fa403eb8740 1 -- start start 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567294d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x5567294e1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa403eb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55672943ecc0 con 0x5567294df950 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa403eb8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55672943c450 con 0x5567294d4790 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa403eb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556729437d40 con 0x5567294d50b0 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x5567294e1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3fad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3fb59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567294d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x5567294e1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51130/0 (socket says 172.21.15.44:51130) 2024-09-13T17:30:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 -- 172.21.15.44:0/1420744144 learned_addr learned my addr 172.21.15.44:0/1420744144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 -- 172.21.15.44:0/1420744144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 msgr2=0x5567294df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 -- 172.21.15.44:0/1420744144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 msgr2=0x5567294d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f8d99640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3e80a1200 0x7fa3e80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567294d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3f3fff640 1 -- 172.21.15.44:0/1420744144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567293ff620 con 0x5567294df950 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567294d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.887+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3e80a1200 0x7fa3e80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x5567294e1d40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009900 tx=0x7fa3e4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e03dba4068aac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa3f8d99640 1 -- 172.21.15.44:0/3811663679 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa3ec0ccad0 con 0x7fa3fc0fee60 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa3f17fa640 1 -- 172.21.15.44:0/1420744144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4004220 con 0x5567294df950 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa3f17fa640 1 -- 172.21.15.44:0/1420744144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3e4031df0 con 0x5567294df950 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa3f17fa640 1 -- 172.21.15.44:0/1420744144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003bd0 con 0x5567294df950 2024-09-13T17:30:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa403eb8740 1 -- 172.21.15.44:0/1420744144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 msgr2=0x5567294e1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.888+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3e80a1200 0x7fa3e80a36c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005fd0 tx=0x7fa3f4004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x5567294e1d40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009900 tx=0x7fa3e4005c00 comp rx=0 tx=0).stop 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- 172.21.15.44:0/1420744144 shutdown_connections 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x5567294e1d40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- 172.21.15.44:0/1420744144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567294d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- 172.21.15.44:0/1420744144 >> 172.21.15.44:0/1420744144 conn(0x5567294d0d00 msgr2=0x5567294d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- 172.21.15.44:0/1420744144 shutdown_connections 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- 172.21.15.44:0/1420744144 wait complete. 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- start start 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 Processor -- start 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d4790 0x5567294deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567293f6180 con 0x5567294d50b0 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556729110430 con 0x5567294df950 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567293bb5e0 con 0x5567294d4790 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa403eb8740 10 client.0 _ll_register_callbacks cb 0x556729427aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa3f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa3f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51134/0 (socket says 172.21.15.44:51134) 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.889+0000 7fa3f3fff640 1 -- 172.21.15.44:0/2898559292 learned_addr learned my addr 172.21.15.44:0/2898559292 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d4790 0x5567294deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3f3fff640 1 -- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d4790 msgr2=0x5567294deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d4790 0x5567294deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3f3fff640 1 -- 172.21.15.44:0/2898559292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 msgr2=0x5567294dc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294dc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3f3fff640 1 -- 172.21.15.44:0/2898559292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc0675a0 con 0x5567294d50b0 2024-09-13T17:30:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 0x5567294df410 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005ec0 tx=0x7fa3e4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1156484ca9f05f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3e27fc640 1 -- 172.21.15.44:0/2898559292 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e404a070 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3e27fc640 1 -- 172.21.15.44:0/2898559292 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa3e4004230 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556729461b80 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa3e27fc640 1 -- 172.21.15.44:0/2898559292 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3e40043f0 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa403eb8740 10 client.16617 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.890+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5567294363c0 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa3e27fc640 1 -- 172.21.15.44:0/2898559292 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556729461b80 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa3e27fc640 1 -- 172.21.15.44:0/2898559292 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa3e4043030 con 0x5567294d50b0 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa403eb8740 10 client.16617 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa403eb8740 10 client.16617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa403eb8740 20 client.16617 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa403eb8740 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5567294eefd0 0x5567294f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa403eb8740 4 client.16617 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55672942a1f0 con 0x5567294eefd0 2024-09-13T17:30:02.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.891+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5567294eefd0 0x5567294f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa3e27fc640 10 client.16617 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:02.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5567294eefd0 0x5567294f13c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f0d0ddd07df7091 server_cookie=21738f91cddf9a2b in_seq=0 out_seq=0 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa3e27fc640 1 -- 172.21.15.44:0/2898559292 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55672942a1f0 con 0x5567294eefd0 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa3e27fc640 10 client.16617 handle_command_reply: tid=0 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 client.16617 shutdown 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 msgr2=0x5567294df410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 0x5567294df410 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005ec0 tx=0x7fa3e4005fd0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 shutdown_connections 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5567294eefd0 0x5567294f13c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294dc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d50b0 0x5567294df410 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa3e27fc640 0 client.16617 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:02.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 --2- 172.21.15.44:0/2898559292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d4790 0x5567294deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 >> 172.21.15.44:0/2898559292 conn(0x5567294d0d00 msgr2=0x5567294d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 shutdown_connections 2024-09-13T17:30:02.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.892+0000 7fa403eb8740 1 -- 172.21.15.44:0/2898559292 wait complete. 2024-09-13T17:30:02.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.893+0000 7fa403eb8740 20 client.16617 trim_cache size 0 max 16384 2024-09-13T17:30:02.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.903+0000 7fa403eb8740 1 Processor -- start 2024-09-13T17:30:02.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.903+0000 7fa403eb8740 1 -- start start 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 0x5567294dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294db1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa403eb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567294e2400 con 0x5567294d4790 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa403eb8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567294cc6b0 con 0x5567294df950 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa403eb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567294d45b0 con 0x5567294d50b0 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 0x5567294dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 0x5567294dc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51136/0 (socket says 172.21.15.44:51136) 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3148750973 learned_addr learned my addr 172.21.15.44:0/3148750973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294db1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3148750973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 msgr2=0x5567294e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3148750973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 msgr2=0x5567294db1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294db1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 -- 172.21.15.44:0/3148750973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567293d8770 con 0x5567294d4790 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.904+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 0x5567294dc440 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec095d60 tx=0x7fa3ec0b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e41656195ed706 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa3f1ffb640 1 -- 172.21.15.44:0/3148750973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0991c0 con 0x5567294d4790 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa3f1ffb640 1 -- 172.21.15.44:0/3148750973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec0046c0 con 0x5567294d4790 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa3f1ffb640 1 -- 172.21.15.44:0/3148750973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0a1d50 con 0x5567294d4790 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 -- 172.21.15.44:0/3148750973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 msgr2=0x5567294dc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 0x5567294dc440 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec095d60 tx=0x7fa3ec0b9040 comp rx=0 tx=0).stop 2024-09-13T17:30:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 -- 172.21.15.44:0/3148750973 shutdown_connections 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567294e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294df950 0x5567294db1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 --2- 172.21.15.44:0/3148750973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294d4790 0x5567294dc440 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 -- 172.21.15.44:0/3148750973 >> 172.21.15.44:0/3148750973 conn(0x5567294d0d00 msgr2=0x5567294daac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 -- 172.21.15.44:0/3148750973 shutdown_connections 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.905+0000 7fa403eb8740 1 -- 172.21.15.44:0/3148750973 wait complete. 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 -- start start 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 Processor -- start 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567295423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567295428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x55672953d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55672953dfb0 con 0x5567294df950 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55672953e120 con 0x5567294d4790 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55672953e2e0 con 0x5567294d50b0 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa403eb8740 10 client.0 _ll_register_callbacks cb 0x7fa3e4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x55672953d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x55672953d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51148/0 (socket says 172.21.15.44:51148) 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 -- 172.21.15.44:0/951696218 learned_addr learned my addr 172.21.15.44:0/951696218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3f3fff640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567295428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567295423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 -- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 msgr2=0x5567295428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567295428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 -- 172.21.15.44:0/951696218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 msgr2=0x5567295423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567295423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.906+0000 7fa3fb59e640 1 -- 172.21.15.44:0/951696218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567294363c0 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3fb59e640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x55672953d9f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40138e0 tx=0x7fa3f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f65f1635dfe652e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3e27fc640 1 -- 172.21.15.44:0/951696218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4025070 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3e27fc640 1 -- 172.21.15.44:0/951696218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa3f4020b70 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3e4031df0 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3e27fc640 1 -- 172.21.15.44:0/951696218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f4020d30 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 10 client.16623 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fa3fc1a3200 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3e27fc640 1 -- 172.21.15.44:0/951696218 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f4033080 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa3e27fc640 1 -- 172.21.15.44:0/951696218 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa3f401c420 con 0x5567294df950 2024-09-13T17:30:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 10 client.16623 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 10 client.16623 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 20 client.16623 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55672954b430 0x55672954d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 4 client.16623 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.907+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55672912b4e0 con 0x55672954b430 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55672954b430 0x55672954d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa3fad9d640 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55672954b430 0x55672954d850 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=788354ae0a4d4828 server_cookie=aa6b4d3aab8bfe68 in_seq=0 out_seq=0 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa3e27fc640 10 client.16623 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa3e27fc640 1 -- 172.21.15.44:0/951696218 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55672912b4e0 con 0x55672954b430 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa3e27fc640 10 client.16623 handle_command_reply: tid=0 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 client.16623 shutdown 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 msgr2=0x55672953d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x55672953d9f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40138e0 tx=0x7fa3f4004ab0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 shutdown_connections 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55672954b430 0x55672954d850 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567294d50b0 0x5567295428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa3e27fc640 0 client.16623 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567294df950 0x55672953d9f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 --2- 172.21.15.44:0/951696218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5567294d4790 0x5567295423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 >> 172.21.15.44:0/951696218 conn(0x5567294d0d00 msgr2=0x5567294db2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 shutdown_connections 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.908+0000 7fa403eb8740 1 -- 172.21.15.44:0/951696218 wait complete. 2024-09-13T17:30:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa403eb8740 20 client.16623 trim_cache size 0 max 16384 2024-09-13T17:30:02.917 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3e80a1200 msgr2=0x7fa3e80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3e80a1200 0x7fa3e80a36c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005fd0 tx=0x7fa3f4004a40 comp rx=0 tx=0).stop 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 msgr2=0x7fa3fc1962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc1962b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00d770 tx=0x7fa3ec00dc30 comp rx=0 tx=0).stop 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 shutdown_connections 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3e80a1200 0x7fa3e80a36c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3fc19abe0 0x7fa3fc19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 --2- 172.21.15.44:0/3811663679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc1962b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 >> 172.21.15.44:0/3811663679 conn(0x7fa3fc075c60 msgr2=0x7fa3fc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:02.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 shutdown_connections 2024-09-13T17:30:02.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:02.909+0000 7fa40119c640 1 -- 172.21.15.44:0/3811663679 wait complete. 2024-09-13T17:30:02.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:64/600: open fd = 3 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:65/600: open fd = 3 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.006 INFO:tasks.workunit.client.0.smithi044.stdout:66/600: open fd = 3 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:67/600: open fd = 3 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:68/600: open fd = 3 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.007 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:69/600: open fd = 3 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:70/600: open fd = 3 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:71/600: open fd = 3 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.008 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:72/600: open fd = 3 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:73/600: open fd = 3 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:74/600: open fd = 3 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:75/600: open fd = 3 2024-09-13T17:30:03.009 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:76/600: open fd = 3 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:77/600: open fd = 3 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.010 INFO:tasks.workunit.client.0.smithi044.stdout:78/600: open fd = 3 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:79/600: open fd = 3 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:80/600: open fd = 3 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:81/600: open fd = 3 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.011 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:82/600: open fd = 3 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:83/600: open fd = 3 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:84/600: open fd = 3 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.012 INFO:tasks.workunit.client.0.smithi044.stdout:85/600: open fd = 3 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:86/600: open fd = 3 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:87/600: open fd = 3 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:88/600: open fd = 3 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.013 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:89/600: open fd = 3 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:90/600: open fd = 3 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:91/600: open fd = 3 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.014 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:92/600: open fd = 3 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:93/600: open fd = 3 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:94/600: open fd = 3 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.015 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:95/600: open fd = 3 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:96/600: open fd = 3 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:97/600: open fd = 3 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.016 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:98/600: open fd = 3 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:99/600: open fd = 3 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:100/600: open fd = 3 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.017 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:101/600: open fd = 3 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:102/600: open fd = 3 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:103/600: open fd = 3 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:104/600: open fd = 3 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.018 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:105/600: open fd = 3 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:106/600: open fd = 3 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:107/600: open fd = 3 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:108/600: open fd = 3 2024-09-13T17:30:03.019 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:109/600: open fd = 3 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:110/600: open fd = 3 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:111/600: open fd = 3 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.020 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:112/600: open fd = 3 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:113/600: open fd = 3 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:114/600: open fd = 3 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.021 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:115/600: open fd = 3 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:116/600: open fd = 3 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:117/600: open fd = 3 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.022 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:118/600: open fd = 3 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:119/600: open fd = 3 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:120/600: open fd = 3 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:121/600: open fd = 3 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.023 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:122/600: open fd = 3 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:123/600: open fd = 3 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:124/600: open fd = 3 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.024 INFO:tasks.workunit.client.0.smithi044.stdout:125/600: open fd = 3 2024-09-13T17:30:03.025 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.025 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.025 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:30:03.025 INFO:tasks.workunit.client.0.smithi044.stdout:126/600: open fd = 3 2024-09-13T17:30:03.025 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 -- 172.21.15.44:0/4018819279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7181050c0 msgr2=0x7fc71810cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 --2- 172.21.15.44:0/4018819279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7181050c0 0x7fc71810cc10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc710009cd0 tx=0x7fc71002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 -- 172.21.15.44:0/4018819279 shutdown_connections 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 --2- 172.21.15.44:0/4018819279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc718100920 0x7fc718104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 --2- 172.21.15.44:0/4018819279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7181050c0 0x7fc71810cc10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 --2- 172.21.15.44:0/4018819279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7180fff70 0x7fc718100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 -- 172.21.15.44:0/4018819279 >> 172.21.15.44:0/4018819279 conn(0x7fc7180fb9e0 msgr2=0x7fc7180fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 -- 172.21.15.44:0/4018819279 shutdown_connections 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 -- 172.21.15.44:0/4018819279 wait complete. 2024-09-13T17:30:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 Processor -- start 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.025+0000 7fc71eb33640 1 -- start start 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71eb33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180fff70 0x7fc7181961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71eb33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7181050c0 0x7fc7181966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71eb33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 0x7fc71819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71eb33640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71819b4c0 con 0x7fc71819aad0 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71eb33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71819b630 con 0x7fc7181050c0 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71eb33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71819b7a0 con 0x7fc7180fff70 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 0x7fc71819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 0x7fc71819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51160/0 (socket says 172.21.15.44:51160) 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 -- 172.21.15.44:0/1901790412 learned_addr learned my addr 172.21.15.44:0/1901790412 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180fff70 0x7fc7181961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc717fff640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7181050c0 0x7fc7181966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 -- 172.21.15.44:0/1901790412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180fff70 msgr2=0x7fc7181961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180fff70 0x7fc7181961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 -- 172.21.15.44:0/1901790412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7181050c0 msgr2=0x7fc7181966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7181050c0 0x7fc7181966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 -- 172.21.15.44:0/1901790412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710009900 con 0x7fc71819aad0 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.026+0000 7fc71d0d0640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 0x7fc71819af80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc710004af0 tx=0x7fc710030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.027+0000 7fc715ffb640 1 -- 172.21.15.44:0/1901790412 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710031b80 con 0x7fc71819aad0 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.027+0000 7fc715ffb640 1 -- 172.21.15.44:0/1901790412 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc710003e00 con 0x7fc71819aad0 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.027+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7181a2d80 con 0x7fc71819aad0 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.027+0000 7fc715ffb640 1 -- 172.21.15.44:0/1901790412 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710045410 con 0x7fc71819aad0 2024-09-13T17:30:03.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.027+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7181a3200 con 0x7fc71819aad0 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.029+0000 7fc715ffb640 1 -- 172.21.15.44:0/1901790412 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc71000a040 con 0x7fc71819aad0 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.029+0000 7fc721859740 1 Processor -- start 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.029+0000 7fc721859740 1 -- start start 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.029+0000 7fc721859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.029+0000 7fc721859740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.029+0000 7fc721859740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe1101d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc721859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe105ecc0 con 0x564fe10f4790 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc721859740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe105c450 con 0x564fe10f50b0 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc721859740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe1057d40 con 0x564fe10ff950 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51168/0 (socket says 172.21.15.44:51168) 2024-09-13T17:30:03.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 -- 172.21.15.44:0/4218005659 learned_addr learned my addr 172.21.15.44:0/4218005659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe1101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 -- 172.21.15.44:0/4218005659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 msgr2=0x564fe1101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe1101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 -- 172.21.15.44:0/4218005659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 msgr2=0x564fe10ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc715ffb640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7040a1200 0x7fc7040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 -- 172.21.15.44:0/4218005659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fe101f620 con 0x564fe10f4790 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc71d0d0640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7040a1200 0x7fc7040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc717fff640 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10f4b70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc700009f70 tx=0x7fc700005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c098405b14f48159 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc715ffb640 1 -- 172.21.15.44:0/1901790412 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc7100ee3c0 con 0x7fc71819aad0 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc70effd640 1 -- 172.21.15.44:0/4218005659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700004050 con 0x564fe10f4790 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc70effd640 1 -- 172.21.15.44:0/4218005659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc700005020 con 0x564fe10f4790 2024-09-13T17:30:03.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.030+0000 7fc70effd640 1 -- 172.21.15.44:0/4218005659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700004a00 con 0x564fe10f4790 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc71d0d0640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7040a1200 0x7fc7040a36c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b7c70 tx=0x7fc7100b6f00 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 -- 172.21.15.44:0/4218005659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 msgr2=0x564fe10f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10f4b70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc700009f70 tx=0x7fc700005850 comp rx=0 tx=0).stop 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 -- 172.21.15.44:0/4218005659 shutdown_connections 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe1101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 --2- 172.21.15.44:0/4218005659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10f4b70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 -- 172.21.15.44:0/4218005659 >> 172.21.15.44:0/4218005659 conn(0x564fe10f0d00 msgr2=0x564fe10f10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 -- 172.21.15.44:0/4218005659 shutdown_connections 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 -- 172.21.15.44:0/4218005659 wait complete. 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 -- start start 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.031+0000 7fc721859740 1 Processor -- start 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe10fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe1016180 con 0x564fe10f4790 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe0d30430 con 0x564fe10f50b0 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe0fdb5e0 con 0x564fe10ff950 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc721859740 10 client.0 _ll_register_callbacks cb 0x564fe1047aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51172/0 (socket says 172.21.15.44:51172) 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 -- 172.21.15.44:0/2027924388 learned_addr learned my addr 172.21.15.44:0/2027924388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe10fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 -- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 msgr2=0x564fe10fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe10fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 -- 172.21.15.44:0/2027924388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 msgr2=0x564fe10ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc717fff640 1 -- 172.21.15.44:0/2027924388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc700009cc0 con 0x564fe10f4790 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.032+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc717fff640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10feec0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc70000a010 tx=0x7fc700005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42d4f5044b2eb5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc6f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc6f7fff640 1 -- 172.21.15.44:0/2027924388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700004da0 con 0x564fe10f4790 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc6f7fff640 1 -- 172.21.15.44:0/2027924388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc70000f500 con 0x564fe10f4790 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fe1081b80 con 0x564fe10f4790 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc6f7fff640 1 -- 172.21.15.44:0/2027924388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70000f6c0 con 0x564fe10f4790 2024-09-13T17:30:03.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 10 client.16635 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564fe10563c0 con 0x564fe10f4790 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc6f7fff640 1 -- 172.21.15.44:0/2027924388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564fe1081b80 con 0x564fe10f4790 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc6f7fff640 1 -- 172.21.15.44:0/2027924388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc70001f7a0 con 0x564fe10f4790 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 10 client.16635 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 20 client.16635 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe110f220 0x564fe1111640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 4 client.16635 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.033+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fe104a1f0 con 0x564fe110f220 2024-09-13T17:30:03.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.034+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe110f220 0x564fe1111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.034+0000 7fc6f7fff640 10 client.16635 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.034+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe110f220 0x564fe1111640 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bb67cbc24ac58f server_cookie=3a251c1ecc1ebce7 in_seq=0 out_seq=0 2024-09-13T17:30:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc6f7fff640 1 -- 172.21.15.44:0/2027924388 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564fe104a1f0 con 0x564fe110f220 2024-09-13T17:30:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc6f7fff640 10 client.16635 handle_command_reply: tid=0 2024-09-13T17:30:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 client.16635 shutdown 2024-09-13T17:30:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 msgr2=0x564fe10feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10feec0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc70000a010 tx=0x7fc700005f90 comp rx=0 tx=0).stop 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 shutdown_connections 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe110f220 0x564fe1111640 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10ff950 0x564fe10fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10f50b0 0x564fe10ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc6f7fff640 0 client.16635 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:03.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 --2- 172.21.15.44:0/2027924388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10feec0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 >> 172.21.15.44:0/2027924388 conn(0x564fe10f0d00 msgr2=0x564fe10f6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:03.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 shutdown_connections 2024-09-13T17:30:03.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 1 -- 172.21.15.44:0/2027924388 wait complete. 2024-09-13T17:30:03.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.035+0000 7fc721859740 20 client.16635 trim_cache size 0 max 16384 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 Processor -- start 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 -- start start 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe1115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1106650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe1102400 con 0x564fe10f4790 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe110bff0 con 0x564fe10ff950 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc721859740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe10f3fa0 con 0x564fe10f50b0 2024-09-13T17:30:03.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51180/0 (socket says 172.21.15.44:51180) 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc717fff640 1 -- 172.21.15.44:0/2727834732 learned_addr learned my addr 172.21.15.44:0/2727834732 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe1115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.046+0000 7fc717fff640 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1106650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc717fff640 1 -- 172.21.15.44:0/2727834732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 msgr2=0x564fe1115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc717fff640 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe1115fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc717fff640 1 -- 172.21.15.44:0/2727834732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 msgr2=0x564fe1106650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc717fff640 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1106650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc717fff640 1 -- 172.21.15.44:0/2727834732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fe0ff8770 con 0x564fe10f4790 2024-09-13T17:30:03.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe1115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc717fff640 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10fba80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc70001fec0 tx=0x7fc700005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e701ac1aa1962f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc70f7fe640 1 -- 172.21.15.44:0/2727834732 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700013ea0 con 0x564fe10f4790 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc70f7fe640 1 -- 172.21.15.44:0/2727834732 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc700004e90 con 0x564fe10f4790 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc70f7fe640 1 -- 172.21.15.44:0/2727834732 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70001f440 con 0x564fe10f4790 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc721859740 1 -- 172.21.15.44:0/2727834732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 msgr2=0x564fe10fba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.047+0000 7fc721859740 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10fba80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc70001fec0 tx=0x7fc700005970 comp rx=0 tx=0).stop 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- 172.21.15.44:0/2727834732 shutdown_connections 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe1115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 --2- 172.21.15.44:0/2727834732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe10fba80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- 172.21.15.44:0/2727834732 >> 172.21.15.44:0/2727834732 conn(0x564fe10f0d00 msgr2=0x564fe110b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- 172.21.15.44:0/2727834732 shutdown_connections 2024-09-13T17:30:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- 172.21.15.44:0/2727834732 wait complete. 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- start start 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 Processor -- start 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe11043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe11048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1104e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe1164840 con 0x564fe10f4790 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe11649b0 con 0x564fe10ff950 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe1164b70 con 0x564fe10f50b0 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc721859740 10 client.0 _ll_register_callbacks cb 0x7fc700009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.048+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe11043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe11043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51194/0 (socket says 172.21.15.44:51194) 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 -- 172.21.15.44:0/3270230056 learned_addr learned my addr 172.21.15.44:0/3270230056 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe11048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 -- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 msgr2=0x564fe11048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe11048e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 -- 172.21.15.44:0/3270230056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 msgr2=0x564fe1104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 -- 172.21.15.44:0/3270230056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fe10563c0 con 0x564fe10f4790 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe11048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc717fff640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe11043a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc700009660 tx=0x7fc7000046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb97b00f38f45521 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc6f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc6f7fff640 1 -- 172.21.15.44:0/3270230056 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700013ea0 con 0x564fe10f4790 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc6f7fff640 1 -- 172.21.15.44:0/3270230056 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc700000e40 con 0x564fe10f4790 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc700005020 con 0x564fe10f4790 2024-09-13T17:30:03.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc6f7fff640 1 -- 172.21.15.44:0/3270230056 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc700004a00 con 0x564fe10f4790 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc721859740 10 client.16641 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.049+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564fe101de80 con 0x564fe10f4790 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc6f7fff640 1 -- 172.21.15.44:0/3270230056 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc700005020 con 0x564fe10f4790 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc6f7fff640 1 -- 172.21.15.44:0/3270230056 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc70001e470 con 0x564fe10f4790 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc721859740 10 client.16641 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc721859740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc721859740 20 client.16641 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc721859740 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe1165db0 0x564fe11681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc721859740 4 client.16641 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564fe0d4b4e0 con 0x564fe1165db0 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.050+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe1165db0 0x564fe11681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc71c8cf640 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe1165db0 0x564fe11681a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98a9b402f85ffb8 server_cookie=ff299bc30cbfe8e2 in_seq=0 out_seq=0 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc6f7fff640 10 client.16641 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc6f7fff640 1 -- 172.21.15.44:0/3270230056 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564fe0d4b4e0 con 0x564fe1165db0 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc6f7fff640 10 client.16641 handle_command_reply: tid=0 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 client.16641 shutdown 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 msgr2=0x564fe11043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe11043a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc700009660 tx=0x7fc7000046f0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 shutdown_connections 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564fe1165db0 0x564fe11681a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fe10f50b0 0x564fe11048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564fe10ff950 0x564fe1104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc6f7fff640 0 client.16641 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 --2- 172.21.15.44:0/3270230056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fe10f4790 0x564fe11043a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 >> 172.21.15.44:0/3270230056 conn(0x564fe10f0d00 msgr2=0x564fe10f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:03.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 shutdown_connections 2024-09-13T17:30:03.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 1 -- 172.21.15.44:0/3270230056 wait complete. 2024-09-13T17:30:03.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.051+0000 7fc721859740 20 client.16641 trim_cache size 0 max 16384 2024-09-13T17:30:03.061 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7040a1200 msgr2=0x7fc7040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7040a1200 0x7fc7040a36c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b7c70 tx=0x7fc7100b6f00 comp rx=0 tx=0).stop 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 msgr2=0x7fc71819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 0x7fc71819af80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc710004af0 tx=0x7fc710030e50 comp rx=0 tx=0).stop 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 shutdown_connections 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7040a1200 0x7fc7040a36c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71819aad0 0x7fc71819af80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7181050c0 0x7fc7181966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 --2- 172.21.15.44:0/1901790412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180fff70 0x7fc7181961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 >> 172.21.15.44:0/1901790412 conn(0x7fc7180fb9e0 msgr2=0x7fc7180fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:03.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 shutdown_connections 2024-09-13T17:30:03.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:03.052+0000 7fc71eb33640 1 -- 172.21.15.44:0/1901790412 wait complete. 2024-09-13T17:30:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:03 smithi100 ceph-mon[35055]: pgmap v103: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-13T17:30:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:03 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/418370642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:30:03.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:03 smithi044 ceph-mon[31175]: pgmap v103: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-13T17:30:03.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:03 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/418370642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:30:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:03 smithi070 ceph-mon[34502]: pgmap v103: 129 pgs: 129 active+clean; 4.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-13T17:30:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:03 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/418370642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:30:04.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:30:04.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:30:04.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.184+0000 7fe78f4fe640 1 -- 172.21.15.44:0/4237490200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 msgr2=0x7fe7880ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.184+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/4237490200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7880ff260 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe784009930 tx=0x7fe78402a420 comp rx=0 tx=0).stop 2024-09-13T17:30:04.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 -- 172.21.15.44:0/4237490200 shutdown_connections 2024-09-13T17:30:04.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/4237490200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788103fd0 0x7fe78810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/4237490200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7880ff830 0x7fe788103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/4237490200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7880ff260 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 -- 172.21.15.44:0/4237490200 >> 172.21.15.44:0/4237490200 conn(0x7fe7880fa930 msgr2=0x7fe7880fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 -- 172.21.15.44:0/4237490200 shutdown_connections 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 -- 172.21.15.44:0/4237490200 wait complete. 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 Processor -- start 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.185+0000 7fe78f4fe640 1 -- start start 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7881970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788103fd0 0x7fe7881975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788197b20 0x7fe7881a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881a27c0 con 0x7fe7880fee80 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881a2930 con 0x7fe788103fd0 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881a2af0 con 0x7fe788197b20 2024-09-13T17:30:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7881970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7881970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51220/0 (socket says 172.21.15.44:51220) 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 -- 172.21.15.44:0/1197516874 learned_addr learned my addr 172.21.15.44:0/1197516874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78da9b640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788197b20 0x7fe7881a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78ca99640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788103fd0 0x7fe7881975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 -- 172.21.15.44:0/1197516874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788197b20 msgr2=0x7fe7881a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788197b20 0x7fe7881a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 -- 172.21.15.44:0/1197516874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788103fd0 msgr2=0x7fe7881975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788103fd0 0x7fe7881975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 -- 172.21.15.44:0/1197516874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784009560 con 0x7fe7880fee80 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78ca99640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788103fd0 0x7fe7881975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78d29a640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7881970a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe78402a940 tx=0x7fe7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe77e7fc640 1 -- 172.21.15.44:0/1197516874 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784004cf0 con 0x7fe7880fee80 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe77e7fc640 1 -- 172.21.15.44:0/1197516874 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe784003a80 con 0x7fe7880fee80 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7881a2d70 con 0x7fe7880fee80 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe77e7fc640 1 -- 172.21.15.44:0/1197516874 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78403ca70 con 0x7fe7880fee80 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.186+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7881a3210 con 0x7fe7880fee80 2024-09-13T17:30:04.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.189+0000 7fe77e7fc640 1 -- 172.21.15.44:0/1197516874 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe784003bf0 con 0x7fe7880fee80 2024-09-13T17:30:04.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.189+0000 7fe79221d740 1 Processor -- start 2024-09-13T17:30:04.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.189+0000 7fe79221d740 1 -- start start 2024-09-13T17:30:04.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe79221d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe79221d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ac0e0 0x555fab9b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe79221d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe79221d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab916c60 con 0x555fab9b6980 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe79221d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab9143f0 con 0x555fab9ab7c0 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe79221d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab90fce0 con 0x555fab9ac0e0 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe77e7fc640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe7740a1280 0x7fe7740a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51226/0 (socket says 172.21.15.44:51226) 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 -- 172.21.15.44:0/2094589240 learned_addr learned my addr 172.21.15.44:0/2094589240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78d29a640 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ac0e0 0x555fab9b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78da9b640 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78da9b640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe7740a1280 0x7fe7740a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe77e7fc640 1 -- 172.21.15.44:0/1197516874 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe7840ee370 con 0x7fe7880fee80 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 -- 172.21.15.44:0/2094589240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ac0e0 msgr2=0x555fab9b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ac0e0 0x555fab9b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 -- 172.21.15.44:0/2094589240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 msgr2=0x555fab9abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.190+0000 7fe78ca99640 1 -- 172.21.15.44:0/2094589240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fab8cb970 con 0x555fab9b6980 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe78ca99640 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b8d70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe778005fd0 tx=0x7fe77800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2655c29efb08c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe78da9b640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe7740a1280 0x7fe7740a3740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe780009370 tx=0x7fe780009240 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe7737fe640 1 -- 172.21.15.44:0/2094589240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800eec0 con 0x555fab9b6980 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe7737fe640 1 -- 172.21.15.44:0/2094589240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7780054b0 con 0x555fab9b6980 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe7737fe640 1 -- 172.21.15.44:0/2094589240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778011470 con 0x555fab9b6980 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe79221d740 1 -- 172.21.15.44:0/2094589240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 msgr2=0x555fab9b8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.191+0000 7fe79221d740 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b8d70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe778005fd0 tx=0x7fe77800e6c0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- 172.21.15.44:0/2094589240 shutdown_connections 2024-09-13T17:30:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ac0e0 0x555fab9b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b8d70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 --2- 172.21.15.44:0/2094589240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- 172.21.15.44:0/2094589240 >> 172.21.15.44:0/2094589240 conn(0x555fab9a7d50 msgr2=0x555fab9a8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- 172.21.15.44:0/2094589240 shutdown_connections 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- 172.21.15.44:0/2094589240 wait complete. 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- start start 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 Processor -- start 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555fab9b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab8cdf80 con 0x555fab9ac0e0 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab8cdc20 con 0x555fab9ab7c0 2024-09-13T17:30:04.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab89aab0 con 0x555fab9b6980 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.192+0000 7fe79221d740 10 client.0 _ll_register_callbacks cb 0x555fab909350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555fab9b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555fab9b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51242/0 (socket says 172.21.15.44:51242) 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 -- 172.21.15.44:0/2100185902 learned_addr learned my addr 172.21.15.44:0/2100185902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78d29a640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78da9b640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 -- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 msgr2=0x555fab9b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 -- 172.21.15.44:0/2100185902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 msgr2=0x555fab9b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 -- 172.21.15.44:0/2100185902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7780098a0 con 0x555fab9ac0e0 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78d29a640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:04.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe78ca99640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555fab9b3600 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe778004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4928181aaa17a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe770ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe770ff9640 1 -- 172.21.15.44:0/2100185902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778021070 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe770ff9640 1 -- 172.21.15.44:0/2100185902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe7780046c0 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fab603590 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.193+0000 7fe770ff9640 1 -- 172.21.15.44:0/2100185902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe77800eec0 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 10 client.16653 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fab90e360 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe770ff9640 1 -- 172.21.15.44:0/2100185902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555fab603590 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe770ff9640 1 -- 172.21.15.44:0/2100185902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe77802c080 con 0x555fab9ac0e0 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 10 client.16653 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 20 client.16653 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:04.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555fab9c6570 0x555fab9c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 4 client.16653 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fab614860 con 0x555fab9c6570 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.194+0000 7fe78d29a640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555fab9c6570 0x555fab9c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe78d29a640 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555fab9c6570 0x555fab9c8990 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9bbb3161cecdc1 server_cookie=7c6c7442f6d9ac3d in_seq=0 out_seq=0 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe770ff9640 10 client.16653 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe770ff9640 1 -- 172.21.15.44:0/2100185902 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555fab614860 con 0x555fab9c6570 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe770ff9640 10 client.16653 handle_command_reply: tid=0 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 client.16653 shutdown 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 msgr2=0x555fab9b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555fab9b3600 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe778004ed0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 shutdown_connections 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555fab9c6570 0x555fab9c8990 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555fab9b3600 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe770ff9640 0 client.16653 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 --2- 172.21.15.44:0/2100185902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555fab9b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.195+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 >> 172.21.15.44:0/2100185902 conn(0x555fab9a7d50 msgr2=0x555fab9ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.196+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 shutdown_connections 2024-09-13T17:30:04.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.196+0000 7fe79221d740 1 -- 172.21.15.44:0/2100185902 wait complete. 2024-09-13T17:30:04.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.196+0000 7fe79221d740 20 client.16653 trim_cache size 0 max 16384 2024-09-13T17:30:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 Processor -- start 2024-09-13T17:30:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 -- start start 2024-09-13T17:30:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ab7c0 0x555fab9bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ac0e0 0x555fab9bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab9c2590 con 0x555fab9b6980 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab999ef0 con 0x555fab9ac0e0 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe79221d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab9b1b30 con 0x555fab9ab7c0 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.210+0000 7fe78da9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51248/0 (socket says 172.21.15.44:51248) 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 -- 172.21.15.44:0/3471502991 learned_addr learned my addr 172.21.15.44:0/3471502991 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78ca99640 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ac0e0 0x555fab9bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78d29a640 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ab7c0 0x555fab9bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 -- 172.21.15.44:0/3471502991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ab7c0 msgr2=0x555fab9bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ab7c0 0x555fab9bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 -- 172.21.15.44:0/3471502991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ac0e0 msgr2=0x555fab9bdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ac0e0 0x555fab9bdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 -- 172.21.15.44:0/3471502991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fab8b01c0 con 0x555fab9b6980 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78ca99640 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ac0e0 0x555fab9bdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe78da9b640 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b1eb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe780011dd0 tx=0x7fe7800096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d126233979100225 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe773fff640 1 -- 172.21.15.44:0/3471502991 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800045d0 con 0x555fab9b6980 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe773fff640 1 -- 172.21.15.44:0/3471502991 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe78001a040 con 0x555fab9b6980 2024-09-13T17:30:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.211+0000 7fe773fff640 1 -- 172.21.15.44:0/3471502991 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780005730 con 0x555fab9b6980 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 -- 172.21.15.44:0/3471502991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 msgr2=0x555fab9b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b1eb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe780011dd0 tx=0x7fe7800096d0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 -- 172.21.15.44:0/3471502991 shutdown_connections 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9b6980 0x555fab9b1eb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ac0e0 0x555fab9bdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 --2- 172.21.15.44:0/3471502991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9ab7c0 0x555fab9bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 -- 172.21.15.44:0/3471502991 >> 172.21.15.44:0/3471502991 conn(0x555fab9a7d50 msgr2=0x555fab9ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 -- 172.21.15.44:0/3471502991 shutdown_connections 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 -- 172.21.15.44:0/3471502991 wait complete. 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 -- start start 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.212+0000 7fe79221d740 1 Processor -- start 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555faba15ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555faba163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab9bd130 con 0x555fab9ac0e0 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab9bd2a0 con 0x555fab9ab7c0 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fab9bd460 con 0x555fab9b6980 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 10 client.0 _ll_register_callbacks cb 0x7fe780009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555faba163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555faba163e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51258/0 (socket says 172.21.15.44:51258) 2024-09-13T17:30:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 -- 172.21.15.44:0/4135820253 learned_addr learned my addr 172.21.15.44:0/4135820253 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78d29a640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555faba15ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78da9b640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 -- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 msgr2=0x555fab9bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 -- 172.21.15.44:0/4135820253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 msgr2=0x555faba15ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555faba15ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 -- 172.21.15.44:0/4135820253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7800148d0 con 0x555fab9ac0e0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe78ca99640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555faba163e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x555fab9b2e70 tx=0x7fe7780097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6b97202370b95f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe770ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe770ff9640 1 -- 172.21.15.44:0/4135820253 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778021070 con 0x555fab9ac0e0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe770ff9640 1 -- 172.21.15.44:0/4135820253 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe778000e40 con 0x555fab9ac0e0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.213+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7780054b0 con 0x555fab9ac0e0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe770ff9640 1 -- 172.21.15.44:0/4135820253 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe778011ea0 con 0x555fab9ac0e0 2024-09-13T17:30:04.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 10 client.16659 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fab8d5990 con 0x555fab9ac0e0 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe770ff9640 1 -- 172.21.15.44:0/4135820253 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7780054b0 con 0x555fab9ac0e0 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe770ff9640 1 -- 172.21.15.44:0/4135820253 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe77802c040 con 0x555fab9ac0e0 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 10 client.16659 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 20 client.16659 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555faba1ea30 0x555faba20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 4 client.16659 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555fab667eb0 con 0x555faba1ea30 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.214+0000 7fe78d29a640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555faba1ea30 0x555faba20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.215+0000 7fe770ff9640 10 client.16659 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.215+0000 7fe78d29a640 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555faba1ea30 0x555faba20e50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e2bc0af1c7c66e server_cookie=61bfa32e51e761ce in_seq=0 out_seq=0 2024-09-13T17:30:04.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.215+0000 7fe770ff9640 1 -- 172.21.15.44:0/4135820253 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555fab667eb0 con 0x555faba1ea30 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe770ff9640 10 client.16659 handle_command_reply: tid=0 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 client.16659 shutdown 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 msgr2=0x555faba163e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555faba163e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x555fab9b2e70 tx=0x7fe7780097f0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 shutdown_connections 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555faba1ea30 0x555faba20e50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe770ff9640 0 client.16659 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fab9b6980 0x555fab9bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fab9ac0e0 0x555faba163e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 --2- 172.21.15.44:0/4135820253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555fab9ab7c0 0x555faba15ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 >> 172.21.15.44:0/4135820253 conn(0x555fab9a7d50 msgr2=0x555fab9b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 shutdown_connections 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 1 -- 172.21.15.44:0/4135820253 wait complete. 2024-09-13T17:30:04.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.216+0000 7fe79221d740 20 client.16659 trim_cache size 0 max 16384 2024-09-13T17:30:04.223 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:30:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:30:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "82f7baa0-b621-4388-9ec8-7e34cf66f6d1", 2024-09-13T17:30:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:30:04.223 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe7740a1280 msgr2=0x7fe7740a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe7740a1280 0x7fe7740a3740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe780009370 tx=0x7fe780009240 comp rx=0 tx=0).stop 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 msgr2=0x7fe7881970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7881970a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe78402a940 tx=0x7fe7840046c0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 shutdown_connections 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe7740a1280 0x7fe7740a3740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788197b20 0x7fe7881a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788103fd0 0x7fe7881975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 --2- 172.21.15.44:0/1197516874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7880fee80 0x7fe7881970a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 >> 172.21.15.44:0/1197516874 conn(0x7fe7880fa930 msgr2=0x7fe7880fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 shutdown_connections 2024-09-13T17:30:04.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.217+0000 7fe78f4fe640 1 -- 172.21.15.44:0/1197516874 wait complete. 2024-09-13T17:30:04.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:30:04.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 -- 172.21.15.44:0/3943443676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c1050a0 msgr2=0x7fb50c10cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 --2- 172.21.15.44:0/3943443676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c1050a0 0x7fb50c10cbf0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb504009cd0 tx=0x7fb50402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 -- 172.21.15.44:0/3943443676 shutdown_connections 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 --2- 172.21.15.44:0/3943443676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb50c100900 0x7fb50c104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 --2- 172.21.15.44:0/3943443676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c1050a0 0x7fb50c10cbf0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 --2- 172.21.15.44:0/3943443676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c0fff50 0x7fb50c100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 -- 172.21.15.44:0/3943443676 >> 172.21.15.44:0/3943443676 conn(0x7fb50c0fb9e0 msgr2=0x7fb50c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 -- 172.21.15.44:0/3943443676 shutdown_connections 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 -- 172.21.15.44:0/3943443676 wait complete. 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.333+0000 7fb512365640 1 Processor -- start 2024-09-13T17:30:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 -- start start 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 0x7fb50c196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb50c1050a0 0x7fb50c1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c19aab0 0x7fb50c19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c19b4a0 con 0x7fb50c0fff50 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c19b610 con 0x7fb50c19aab0 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb512365640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c19b780 con 0x7fb50c1050a0 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 0x7fb50c196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 0x7fb50c196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51276/0 (socket says 172.21.15.44:51276) 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 -- 172.21.15.44:0/3779147032 learned_addr learned my addr 172.21.15.44:0/3779147032 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb510902640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c19aab0 0x7fb50c19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb50c1050a0 0x7fb50c1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 -- 172.21.15.44:0/3779147032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb50c1050a0 msgr2=0x7fb50c1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb50c1050a0 0x7fb50c1966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 -- 172.21.15.44:0/3779147032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c19aab0 msgr2=0x7fb50c19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c19aab0 0x7fb50c19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.334+0000 7fb50bfff640 1 -- 172.21.15.44:0/3779147032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504009900 con 0x7fb50c0fff50 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.335+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 0x7fb50c196180 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d950 tx=0x7fb4fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.335+0000 7fb5097fa640 1 -- 172.21.15.44:0/3779147032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc009a30 con 0x7fb50c0fff50 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.335+0000 7fb5097fa640 1 -- 172.21.15.44:0/3779147032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc0044b0 con 0x7fb50c0fff50 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.335+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c1a2d60 con 0x7fb50c0fff50 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.335+0000 7fb5097fa640 1 -- 172.21.15.44:0/3779147032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0047a0 con 0x7fb50c0fff50 2024-09-13T17:30:04.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.335+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c1a3200 con 0x7fb50c0fff50 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb5097fa640 1 -- 172.21.15.44:0/3779147032 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4fc004020 con 0x7fb50c0fff50 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 Processor -- start 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 -- start start 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f3ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4e9ef20 con 0x55c3c4f3efd0 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4e9c6b0 con 0x55c3c4f34730 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.337+0000 7fb515088740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4e97fa0 con 0x55c3c4f33e10 2024-09-13T17:30:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51290/0 (socket says 172.21.15.44:51290) 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3455514784 learned_addr learned my addr 172.21.15.44:0/3455514784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb5097fa640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4f40a1280 0x7fb4f40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f3ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3455514784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 msgr2=0x55c3c4f341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb510902640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3455514784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 msgr2=0x55c3c4f3ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f3ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3455514784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c4e52840 con 0x55c3c4f3efd0 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f3ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb510902640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb510902640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4f40a1280 0x7fb4f40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb5097fa640 1 -- 172.21.15.44:0/3779147032 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb4fc0cc9c0 con 0x7fb50c0fff50 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f413c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb500009b70 tx=0x7fb500010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd924ec5ca9d7a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb4f27fc640 1 -- 172.21.15.44:0/3455514784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500009640 con 0x55c3c4f3efd0 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb4f27fc640 1 -- 172.21.15.44:0/3455514784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb500009e60 con 0x55c3c4f3efd0 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.338+0000 7fb4f27fc640 1 -- 172.21.15.44:0/3455514784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500012880 con 0x55c3c4f3efd0 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 -- 172.21.15.44:0/3455514784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 msgr2=0x55c3c4f413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f413c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb500009b70 tx=0x7fb500010740 comp rx=0 tx=0).stop 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb510902640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4f40a1280 0x7fb4f40a3740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb504009ca0 tx=0x7fb5040095c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 -- 172.21.15.44:0/3455514784 shutdown_connections 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f413c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f3ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 --2- 172.21.15.44:0/3455514784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 -- 172.21.15.44:0/3455514784 >> 172.21.15.44:0/3455514784 conn(0x55c3c4f30380 msgr2=0x55c3c4f30760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 -- 172.21.15.44:0/3455514784 shutdown_connections 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 -- 172.21.15.44:0/3455514784 wait complete. 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.339+0000 7fb515088740 1 -- start start 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 Processor -- start 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f3e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 0x55c3c4f3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f3c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4e392a0 con 0x55c3c4f34730 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4b70430 con 0x55c3c4f33e10 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4e1b680 con 0x55c3c4f3efd0 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb515088740 10 client.0 _ll_register_callbacks cb 0x55c3c4e91610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 0x55c3c4f3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 0x55c3c4f3bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51302/0 (socket says 172.21.15.44:51302) 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3107848395 learned_addr learned my addr 172.21.15.44:0/3107848395 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb510902640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f3c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f3e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f3efd0 msgr2=0x55c3c4f3c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f3c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3107848395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 msgr2=0x55c3c4f3e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f3e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.340+0000 7fb50b7fe640 1 -- 172.21.15.44:0/3107848395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5000097a0 con 0x55c3c4f34730 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 0x55c3c4f3bb10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb500005dd0 tx=0x7fb500009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2b7007b4e5fc07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb4eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/3107848395 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500004ad0 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/3107848395 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb500016070 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3c4b8b4e0 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/3107848395 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb500004c30 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 10 client.16671 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c3c4e96620 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/3107848395 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3c4b8b4e0 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/3107848395 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb500024030 con 0x55c3c4f34730 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 10 client.16671 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 20 client.16671 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:04.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4f4ec20 0x55c3c4f51010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 4 client.16671 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.341+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3c4b9c7e0 con 0x55c3c4f4ec20 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.342+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4f4ec20 0x55c3c4f51010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.342+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4f4ec20 0x55c3c4f51010 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8dc898c4a61b65 server_cookie=fe9d687425f7a67c in_seq=0 out_seq=0 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.342+0000 7fb4eb7fe640 10 client.16671 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/3107848395 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c3c4b9c7e0 con 0x55c3c4f4ec20 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb4eb7fe640 10 client.16671 handle_command_reply: tid=0 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 client.16671 shutdown 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 msgr2=0x55c3c4f3bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 0x55c3c4f3bb10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb500005dd0 tx=0x7fb500009640 comp rx=0 tx=0).stop 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 shutdown_connections 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4f4ec20 0x55c3c4f51010 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f3c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f34730 0x55c3c4f3bb10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 --2- 172.21.15.44:0/3107848395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 >> 172.21.15.44:0/3107848395 conn(0x55c3c4f30380 msgr2=0x55c3c4f2f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb4eb7fe640 0 client.16671 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 shutdown_connections 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 1 -- 172.21.15.44:0/3107848395 wait complete. 2024-09-13T17:30:04.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.343+0000 7fb515088740 20 client.16671 trim_cache size 0 max 16384 2024-09-13T17:30:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.357+0000 7fb515088740 1 Processor -- start 2024-09-13T17:30:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.357+0000 7fb515088740 1 -- start start 2024-09-13T17:30:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb515088740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb515088740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb515088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f36e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb515088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4f34dd0 con 0x55c3c4f3efd0 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb515088740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4f31010 con 0x55c3c4f34730 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb515088740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4f312e0 con 0x55c3c4f33e10 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f36e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f36e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51314/0 (socket says 172.21.15.44:51314) 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 -- 172.21.15.44:0/3849255603 learned_addr learned my addr 172.21.15.44:0/3849255603 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 -- 172.21.15.44:0/3849255603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 msgr2=0x55c3c4f2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 -- 172.21.15.44:0/3849255603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 msgr2=0x55c3c4f46180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f46180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 -- 172.21.15.44:0/3849255603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c4e38790 con 0x55c3c4f3efd0 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f46180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb50bfff640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb510902640 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f36e10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb504035000 tx=0x7fb50403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278f22ff2e494696 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb4f2ffd640 1 -- 172.21.15.44:0/3849255603 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504031a30 con 0x55c3c4f3efd0 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb4f2ffd640 1 -- 172.21.15.44:0/3849255603 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb504031b90 con 0x55c3c4f3efd0 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.358+0000 7fb4f2ffd640 1 -- 172.21.15.44:0/3849255603 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504052480 con 0x55c3c4f3efd0 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 -- 172.21.15.44:0/3849255603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 msgr2=0x55c3c4f36e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f36e10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb504035000 tx=0x7fb50403f000 comp rx=0 tx=0).stop 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 -- 172.21.15.44:0/3849255603 shutdown_connections 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4f36e10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f34730 0x55c3c4f46180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 --2- 172.21.15.44:0/3849255603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f33e10 0x55c3c4f2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 -- 172.21.15.44:0/3849255603 >> 172.21.15.44:0/3849255603 conn(0x55c3c4f30380 msgr2=0x55c3c4f55940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 -- 172.21.15.44:0/3849255603 shutdown_connections 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 -- 172.21.15.44:0/3849255603 wait complete. 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.359+0000 7fb515088740 1 -- start start 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 Processor -- start 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f9cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f34730 0x55c3c4f9d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4fa1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4fa19d0 con 0x55c3c4f3efd0 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4fa1b40 con 0x55c3c4f33e10 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c4fa1d00 con 0x55c3c4f34730 2024-09-13T17:30:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb515088740 10 client.0 _ll_register_callbacks cb 0x7fb5000099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4fa1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4fa1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51320/0 (socket says 172.21.15.44:51320) 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 -- 172.21.15.44:0/2251825679 learned_addr learned my addr 172.21.15.44:0/2251825679 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb50bfff640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f9cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb50b7fe640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f34730 0x55c3c4f9d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 -- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f34730 msgr2=0x55c3c4f9d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f34730 0x55c3c4f9d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 -- 172.21.15.44:0/2251825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 msgr2=0x55c3c4f9cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f9cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 -- 172.21.15.44:0/2251825679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb504042070 con 0x55c3c4f3efd0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb510902640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4fa1460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb504031d20 tx=0x7fb504004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64ac5f454ecd2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.360+0000 7fb4eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/2251825679 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504003b70 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/2251825679 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fb504004110 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb500009e60 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/2251825679 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb504003cd0 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 10 client.16677 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c3c4e5dea0 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/2251825679 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb500009e60 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/2251825679 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb504052510 con 0x55c3c4f3efd0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 10 client.16677 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 20 client.16677 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4faa3d0 0x55c3c4fac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 4 client.16677 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.361+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c3c4befe30 con 0x55c3c4faa3d0 2024-09-13T17:30:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb50bfff640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4faa3d0 0x55c3c4fac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb50bfff640 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4faa3d0 0x55c3c4fac7f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8467314e9411bf8c server_cookie=809ec8bb6ea00687 in_seq=0 out_seq=0 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb4eb7fe640 10 client.16677 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb4eb7fe640 1 -- 172.21.15.44:0/2251825679 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c3c4befe30 con 0x55c3c4faa3d0 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb4eb7fe640 10 client.16677 handle_command_reply: tid=0 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 client.16677 shutdown 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 msgr2=0x55c3c4fa1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4fa1460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb504031d20 tx=0x7fb504004780 comp rx=0 tx=0).stop 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 shutdown_connections 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c3c4faa3d0 0x55c3c4fac7f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3c4f34730 0x55c3c4f9d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c4f3efd0 0x55c3c4fa1460 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb4eb7fe640 0 client.16677 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 --2- 172.21.15.44:0/2251825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c3c4f33e10 0x55c3c4f9cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 >> 172.21.15.44:0/2251825679 conn(0x55c3c4f30380 msgr2=0x55c3c4f36e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 shutdown_connections 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.362+0000 7fb515088740 1 -- 172.21.15.44:0/2251825679 wait complete. 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb515088740 20 client.16677 trim_cache size 0 max 16384 2024-09-13T17:30:04.371 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: "82f7baa0-b621-4388-9ec8-7e34cf66f6d1": { 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "82f7baa0-b621-4388-9ec8-7e34cf66f6d1", 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:30:04.372 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:30:04.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4f40a1280 msgr2=0x7fb4f40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4f40a1280 0x7fb4f40a3740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb504009ca0 tx=0x7fb5040095c0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 msgr2=0x7fb50c196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:04.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 0x7fb50c196180 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d950 tx=0x7fb4fc00de10 comp rx=0 tx=0).stop 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 shutdown_connections 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4f40a1280 0x7fb4f40a3740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb50c1050a0 0x7fb50c1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c19aab0 0x7fb50c19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 --2- 172.21.15.44:0/3779147032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50c0fff50 0x7fb50c196180 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 >> 172.21.15.44:0/3779147032 conn(0x7fb50c0fb9e0 msgr2=0x7fb50c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 shutdown_connections 2024-09-13T17:30:04.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:04.363+0000 7fb512365640 1 -- 172.21.15.44:0/3779147032 wait complete. 2024-09-13T17:30:04.375 INFO:tasks.cephfs.filesystem:scrub status for tag:82f7baa0-b621-4388-9ec8-7e34cf66f6d1 - {'path': '/', 'tag': '82f7baa0-b621-4388-9ec8-7e34cf66f6d1', 'options': 'recursive,force'} 2024-09-13T17:30:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:05 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:30:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:05 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:05 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:30:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:05 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:05 smithi100 ceph-mon[35055]: pgmap v104: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-09-13T17:30:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:05 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:30:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:05 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:05 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:30:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:05 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:05 smithi044 ceph-mon[31175]: pgmap v104: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-09-13T17:30:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:05 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:30:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:05 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:05 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:30:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:05 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:05 smithi070 ceph-mon[34502]: pgmap v104: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-09-13T17:30:07.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:07 smithi100 ceph-mon[35055]: pgmap v105: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-09-13T17:30:07.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:07 smithi044 ceph-mon[31175]: pgmap v105: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-09-13T17:30:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:07 smithi070 ceph-mon[34502]: pgmap v105: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-09-13T17:30:09.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:09 smithi100 ceph-mon[35055]: pgmap v106: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 KiB/s wr, 34 op/s 2024-09-13T17:30:09.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:09 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:30:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:09 smithi044 ceph-mon[31175]: pgmap v106: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 KiB/s wr, 34 op/s 2024-09-13T17:30:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:09 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:30:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:09 smithi070 ceph-mon[34502]: pgmap v106: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 KiB/s wr, 34 op/s 2024-09-13T17:30:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:09 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 -- 172.21.15.44:0/1481993571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c80103fb0 msgr2=0x7f6c8010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 --2- 172.21.15.44:0/1481993571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c80103fb0 0x7f6c8010bb00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009cd0 tx=0x7f6c7802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 -- 172.21.15.44:0/1481993571 shutdown_connections 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 --2- 172.21.15.44:0/1481993571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c800ff810 0x7f6c80103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 --2- 172.21.15.44:0/1481993571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c80103fb0 0x7f6c8010bb00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 --2- 172.21.15.44:0/1481993571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 0x7f6c800ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 -- 172.21.15.44:0/1481993571 >> 172.21.15.44:0/1481993571 conn(0x7f6c800fa8d0 msgr2=0x7f6c800fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 -- 172.21.15.44:0/1481993571 shutdown_connections 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.983+0000 7f6c886d7640 1 -- 172.21.15.44:0/1481993571 wait complete. 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 Processor -- start 2024-09-13T17:30:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 -- start start 2024-09-13T17:30:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 0x7f6c80194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c80103fb0 0x7f6c80195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 0x7f6c801a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c801a4a10 con 0x7f6c801958d0 2024-09-13T17:30:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c801a4b50 con 0x7f6c800fee60 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c886d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c801a4d10 con 0x7f6c80103fb0 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c86c74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 0x7f6c801a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c86c74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 0x7f6c801a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57292/0 (socket says 172.21.15.44:57292) 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c86c74640 1 -- 172.21.15.44:0/2518549331 learned_addr learned my addr 172.21.15.44:0/2518549331 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c86473640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 0x7f6c80194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.984+0000 7f6c85c72640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c80103fb0 0x7f6c80195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86c74640 1 -- 172.21.15.44:0/2518549331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c80103fb0 msgr2=0x7f6c80195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86c74640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c80103fb0 0x7f6c80195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86c74640 1 -- 172.21.15.44:0/2518549331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 msgr2=0x7f6c80194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86c74640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 0x7f6c80194e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86c74640 1 -- 172.21.15.44:0/2518549331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009900 con 0x7f6c801958d0 2024-09-13T17:30:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c85c72640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c80103fb0 0x7f6c80195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86473640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 0x7f6c80194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c86c74640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 0x7f6c801a44d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6c78004af0 tx=0x7f6c78030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c737fe640 1 -- 172.21.15.44:0/2518549331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78031b80 con 0x7f6c801958d0 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c737fe640 1 -- 172.21.15.44:0/2518549331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c78003e00 con 0x7f6c801958d0 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c801a4f90 con 0x7f6c801958d0 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c737fe640 1 -- 172.21.15.44:0/2518549331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78004830 con 0x7f6c801958d0 2024-09-13T17:30:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.985+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c801a54c0 con 0x7f6c801958d0 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.987+0000 7f6c737fe640 1 -- 172.21.15.44:0/2518549331 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6c7800a040 con 0x7f6c801958d0 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 Processor -- start 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 -- start start 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afce7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afc52c30 con 0x5618afcf24f0 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afc503c0 con 0x5618afce7c50 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afc4bcb0 con 0x5618afce7330 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c737fe640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c6c0a1280 0x7f6c6c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57300/0 (socket says 172.21.15.44:57300) 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c85c72640 1 -- 172.21.15.44:0/1753626410 learned_addr learned my addr 172.21.15.44:0/1753626410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.988+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afce7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c86473640 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c86c74640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c6c0a1280 0x7f6c6c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c737fe640 1 -- 172.21.15.44:0/2518549331 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6c780ee6d0 con 0x7f6c801958d0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c85c72640 1 -- 172.21.15.44:0/1753626410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 msgr2=0x5618afce7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afce7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c85c72640 1 -- 172.21.15.44:0/1753626410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 msgr2=0x5618afcf1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c85c72640 1 -- 172.21.15.44:0/1753626410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618afbf2910 con 0x5618afcf24f0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf48e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6c74014940 tx=0x7f6c7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b943b523336766b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c86c74640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c6c0a1280 0x7f6c6c0a3740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c780bb160 tx=0x7f6c78030fc0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c70ff9640 1 -- 172.21.15.44:0/1753626410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740142d0 con 0x5618afcf24f0 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c70ff9640 1 -- 172.21.15.44:0/1753626410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7401a020 con 0x5618afcf24f0 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.989+0000 7f6c70ff9640 1 -- 172.21.15.44:0/1753626410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74012740 con 0x5618afcf24f0 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1753626410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 msgr2=0x5618afcf48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf48e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6c74014940 tx=0x7f6c7400c490 comp rx=0 tx=0).stop 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1753626410 shutdown_connections 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf48e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1753626410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afce7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1753626410 >> 172.21.15.44:0/1753626410 conn(0x5618afce38c0 msgr2=0x5618afce3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1753626410 shutdown_connections 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1753626410 wait complete. 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.990+0000 7f6c8b3fc740 1 -- start start 2024-09-13T17:30:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 Processor -- start 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afcf1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afcef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afc09e80 con 0x5618afcf24f0 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afc09b20 con 0x5618afce7330 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afbcf260 con 0x5618afce7c50 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c8b3fc740 10 client.0 _ll_register_callbacks cb 0x5618afc04df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57304/0 (socket says 172.21.15.44:57304) 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 -- 172.21.15.44:0/2814101855 learned_addr learned my addr 172.21.15.44:0/2814101855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afcf1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c86473640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afcef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 -- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 msgr2=0x5618afcef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afcef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 -- 172.21.15.44:0/2814101855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 msgr2=0x5618afcf1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afcf1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.991+0000 7f6c85c72640 1 -- 172.21.15.44:0/2814101855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c74014620 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c85c72640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef720 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c430 tx=0x7f6c7400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de65ba43d7bc1845 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c69ffb640 1 -- 172.21.15.44:0/2814101855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400fb70 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c69ffb640 1 -- 172.21.15.44:0/2814101855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6c74000e40 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618afc74680 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c69ffb640 1 -- 172.21.15.44:0/2814101855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c7400fcd0 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 10 client.16689 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5618afc4a330 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c69ffb640 1 -- 172.21.15.44:0/2814101855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7402f020 con 0x5618afcf24f0 2024-09-13T17:30:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c69ffb640 1 -- 172.21.15.44:0/2814101855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c74004b10 con 0x5618afcf24f0 2024-09-13T17:30:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 10 client.16689 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 20 client.16689 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd01dd0 0x5618afd041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 4 client.16689 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.992+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618afc3f5d0 con 0x5618afd01dd0 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.993+0000 7f6c86473640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd01dd0 0x5618afd041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.993+0000 7f6c86473640 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd01dd0 0x5618afd041c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37be9536a373037 server_cookie=46d54eba20f33f05 in_seq=0 out_seq=0 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.993+0000 7f6c69ffb640 10 client.16689 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c69ffb640 1 -- 172.21.15.44:0/2814101855 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5618afc3f5d0 con 0x5618afd01dd0 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c69ffb640 10 client.16689 handle_command_reply: tid=0 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 client.16689 shutdown 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 msgr2=0x5618afcef720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef720 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c430 tx=0x7f6c7400ffd0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 shutdown_connections 2024-09-13T17:30:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd01dd0 0x5618afd041c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afcef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c69ffb640 0 client.16689 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef720 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/2814101855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afcf1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 >> 172.21.15.44:0/2814101855 conn(0x5618afce38c0 msgr2=0x5618afce2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 shutdown_connections 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/2814101855 wait complete. 2024-09-13T17:30:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:09.994+0000 7f6c8b3fc740 20 client.16689 trim_cache size 0 max 16384 2024-09-13T17:30:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 Processor -- start 2024-09-13T17:30:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 -- start start 2024-09-13T17:30:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afce3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afce41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afce7b10 con 0x5618afcf24f0 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afcfe4f0 con 0x5618afce7330 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afceee80 con 0x5618afce7c50 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57310/0 (socket says 172.21.15.44:57310) 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c85c72640 1 -- 172.21.15.44:0/1625942410 learned_addr learned my addr 172.21.15.44:0/1625942410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c86473640 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afce41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.007+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afce3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c85c72640 1 -- 172.21.15.44:0/1625942410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 msgr2=0x5618afce41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afce41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c85c72640 1 -- 172.21.15.44:0/1625942410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 msgr2=0x5618afce3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afce3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c85c72640 1 -- 172.21.15.44:0/1625942410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618afbec300 con 0x5618afcf24f0 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c85c72640 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef140 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400ca00 tx=0x7f6c7400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea83dca6c40abe51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c717fa640 1 -- 172.21.15.44:0/1625942410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74021610 con 0x5618afcf24f0 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c717fa640 1 -- 172.21.15.44:0/1625942410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7402f030 con 0x5618afcf24f0 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c717fa640 1 -- 172.21.15.44:0/1625942410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74021bf0 con 0x5618afcf24f0 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1625942410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 msgr2=0x5618afcef140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef140 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400ca00 tx=0x7f6c7400cb90 comp rx=0 tx=0).stop 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1625942410 shutdown_connections 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7c50 0x5618afce41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcef140 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/1625942410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7330 0x5618afce3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.008+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1625942410 >> 172.21.15.44:0/1625942410 conn(0x5618afce38c0 msgr2=0x5618afd08b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1625942410 shutdown_connections 2024-09-13T17:30:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/1625942410 wait complete. 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 -- start start 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 Processor -- start 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afcf5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afd582a0 con 0x5618afcf24f0 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afd583e0 con 0x5618afce7c50 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618afd58550 con 0x5618afce7330 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c8b3fc740 10 client.0 _ll_register_callbacks cb 0x7f6c7400ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c86473640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c85c72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57312/0 (socket says 172.21.15.44:57312) 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c85c72640 1 -- 172.21.15.44:0/525143480 learned_addr learned my addr 172.21.15.44:0/525143480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.009+0000 7f6c85c72640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afcf5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c86473640 1 -- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 msgr2=0x5618afcf5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c86473640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afcf5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c86473640 1 -- 172.21.15.44:0/525143480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 msgr2=0x5618afcf67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c86473640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c86473640 1 -- 172.21.15.44:0/525143480 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618afc4a330 con 0x5618afce7c50 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c85c72640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afcf5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c86473640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5618afd04ad0 tx=0x7f6c7c0044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e4c4a146c13948 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c69ffb640 1 -- 172.21.15.44:0/525143480 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c01f270 con 0x5618afce7c50 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c69ffb640 1 -- 172.21.15.44:0/525143480 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6c7c003e30 con 0x5618afce7c50 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c7401a020 con 0x5618afce7c50 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c69ffb640 1 -- 172.21.15.44:0/525143480 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c7c01f3d0 con 0x5618afce7c50 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c8b3fc740 10 client.26159 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.010+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5618afc118e0 con 0x5618afce7c50 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c69ffb640 1 -- 172.21.15.44:0/525143480 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7401a020 con 0x5618afce7c50 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c69ffb640 1 -- 172.21.15.44:0/525143480 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c7c012c80 con 0x5618afce7c50 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c8b3fc740 10 client.26159 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c8b3fc740 10 client.26159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c8b3fc740 20 client.26159 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd59e10 0x5618afd5c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c8b3fc740 4 client.26159 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5618af93f300 con 0x5618afd59e10 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c85c72640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd59e10 0x5618afd5c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c85c72640 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd59e10 0x5618afd5c230 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea7391f6e940fe3 server_cookie=999fad019db271b4 in_seq=0 out_seq=0 2024-09-13T17:30:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.011+0000 7f6c69ffb640 10 client.26159 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c69ffb640 1 -- 172.21.15.44:0/525143480 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5618af93f300 con 0x5618afd59e10 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c69ffb640 10 client.26159 handle_command_reply: tid=0 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 client.26159 shutdown 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 msgr2=0x5618afcf6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5618afd04ad0 tx=0x7f6c7c0044d0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 shutdown_connections 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5618afd59e10 0x5618afd5c230 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618afcf24f0 0x5618afcf67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5618afce7c50 0x5618afcf6260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 --2- 172.21.15.44:0/525143480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618afce7330 0x5618afcf5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c69ffb640 0 client.26159 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 >> 172.21.15.44:0/525143480 conn(0x5618afce38c0 msgr2=0x5618afce3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 shutdown_connections 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.024+0000 7f6c8b3fc740 1 -- 172.21.15.44:0/525143480 wait complete. 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c8b3fc740 20 client.26159 trim_cache size 0 max 16384 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:30:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:30:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c6c0a1280 msgr2=0x7f6c6c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c6c0a1280 0x7f6c6c0a3740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c780bb160 tx=0x7f6c78030fc0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 msgr2=0x7f6c801a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 0x7f6c801a44d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6c78004af0 tx=0x7f6c78030dd0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 shutdown_connections 2024-09-13T17:30:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c6c0a1280 0x7f6c6c0a3740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c80103fb0 0x7f6c80195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c801958d0 0x7f6c801a44d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 --2- 172.21.15.44:0/2518549331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c800fee60 0x7f6c80194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 >> 172.21.15.44:0/2518549331 conn(0x7f6c800fa8d0 msgr2=0x7f6c800fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 shutdown_connections 2024-09-13T17:30:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:10.025+0000 7f6c886d7640 1 -- 172.21.15.44:0/2518549331 wait complete. 2024-09-13T17:30:11.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:11 smithi100 ceph-mon[35055]: pgmap v107: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-13T17:30:11.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:11 smithi044 ceph-mon[31175]: pgmap v107: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-13T17:30:11.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:11 smithi070 ceph-mon[34502]: pgmap v107: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-13T17:30:13.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:13 smithi044 ceph-mon[31175]: pgmap v108: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-13T17:30:13.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:13 smithi070 ceph-mon[34502]: pgmap v108: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-13T17:30:14.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:13 smithi100 ceph-mon[35055]: pgmap v108: 129 pgs: 129 active+clean; 4.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-13T17:30:15.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:15 smithi044 ceph-mon[31175]: pgmap v109: 129 pgs: 129 active+clean; 4.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-09-13T17:30:15.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:15 smithi070 ceph-mon[34502]: pgmap v109: 129 pgs: 129 active+clean; 4.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-09-13T17:30:16.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:15 smithi100 ceph-mon[35055]: pgmap v109: 129 pgs: 129 active+clean; 4.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-09-13T17:30:17.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:17 smithi044 ceph-mon[31175]: pgmap v110: 129 pgs: 129 active+clean; 4.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-13T17:30:17.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:17 smithi070 ceph-mon[34502]: pgmap v110: 129 pgs: 129 active+clean; 4.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-13T17:30:18.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:17 smithi100 ceph-mon[35055]: pgmap v110: 129 pgs: 129 active+clean; 4.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-13T17:30:19.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:19 smithi044 ceph-mon[31175]: pgmap v111: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 19 op/s 2024-09-13T17:30:19.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:19 smithi070 ceph-mon[34502]: pgmap v111: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 19 op/s 2024-09-13T17:30:20.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:19 smithi100 ceph-mon[35055]: pgmap v111: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 19 op/s 2024-09-13T17:30:21.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:21 smithi044 ceph-mon[31175]: pgmap v112: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-13T17:30:21.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:21 smithi070 ceph-mon[34502]: pgmap v112: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-13T17:30:22.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:21 smithi100 ceph-mon[35055]: pgmap v112: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-13T17:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.123+0000 7f30646f7640 1 -- 172.21.15.44:0/3244854150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0ff7d0 msgr2=0x7f305c103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.123+0000 7f30646f7640 1 --2- 172.21.15.44:0/3244854150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0ff7d0 0x7f305c103810 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3050009930 tx=0x7f305002a420 comp rx=0 tx=0).stop 2024-09-13T17:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 -- 172.21.15.44:0/3244854150 shutdown_connections 2024-09-13T17:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 --2- 172.21.15.44:0/3244854150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c103f70 0x7f305c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 --2- 172.21.15.44:0/3244854150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0ff7d0 0x7f305c103810 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 --2- 172.21.15.44:0/3244854150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c0fee20 0x7f305c0ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 -- 172.21.15.44:0/3244854150 >> 172.21.15.44:0/3244854150 conn(0x7f305c0fa8f0 msgr2=0x7f305c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 -- 172.21.15.44:0/3244854150 shutdown_connections 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 -- 172.21.15.44:0/3244854150 wait complete. 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 Processor -- start 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.124+0000 7f30646f7640 1 -- start start 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 0x7f305c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c103f70 0x7f305c196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c19ab20 0x7f305c19afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c19b510 con 0x7f305c0fee20 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c19b680 con 0x7f305c19ab20 2024-09-13T17:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c19b7f0 con 0x7f305c103f70 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 0x7f305c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 0x7f305c1961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51916/0 (socket says 172.21.15.44:51916) 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 -- 172.21.15.44:0/3048088504 learned_addr learned my addr 172.21.15.44:0/3048088504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062c94640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c19ab20 0x7f305c19afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3061c92640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c103f70 0x7f305c196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 -- 172.21.15.44:0/3048088504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c103f70 msgr2=0x7f305c196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c103f70 0x7f305c196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 -- 172.21.15.44:0/3048088504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c19ab20 msgr2=0x7f305c19afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c19ab20 0x7f305c19afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 -- 172.21.15.44:0/3048088504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050009560 con 0x7f305c0fee20 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3061c92640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c103f70 0x7f305c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062c94640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c19ab20 0x7f305c19afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f3062493640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 0x7f305c1961f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f305800d950 tx=0x7f305800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f304f7fe640 1 -- 172.21.15.44:0/3048088504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058009a30 con 0x7f305c0fee20 2024-09-13T17:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f304f7fe640 1 -- 172.21.15.44:0/3048088504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30580044b0 con 0x7f305c0fee20 2024-09-13T17:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.125+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c1a2dd0 con 0x7f305c0fee20 2024-09-13T17:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.126+0000 7f304f7fe640 1 -- 172.21.15.44:0/3048088504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580047a0 con 0x7f305c0fee20 2024-09-13T17:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.126+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c1a3270 con 0x7f305c0fee20 2024-09-13T17:30:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f304f7fe640 1 -- 172.21.15.44:0/3048088504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3058004020 con 0x7f305c0fee20 2024-09-13T17:30:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 Processor -- start 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 -- start start 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d734c30 con 0x55b62d7ce060 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d7323c0 con 0x55b62d7cd740 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f306741a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d72dcb0 con 0x55b62d7d8900 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f3062493640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f3062493640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51922/0 (socket says 172.21.15.44:51922) 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.129+0000 7f3062493640 1 -- 172.21.15.44:0/2653943207 learned_addr learned my addr 172.21.15.44:0/2653943207 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062c94640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f304f7fe640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30480a1280 0x7f30480a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3061c92640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062c94640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30480a1280 0x7f30480a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f304f7fe640 1 -- 172.21.15.44:0/3048088504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f30580cca20 con 0x7f305c0fee20 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062493640 1 -- 172.21.15.44:0/2653943207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 msgr2=0x55b62d7dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062493640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062493640 1 -- 172.21.15.44:0/2653943207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 msgr2=0x55b62d7cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062493640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062493640 1 -- 172.21.15.44:0/2653943207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b62d6d4910 con 0x55b62d7ce060 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3061c92640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062c94640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062493640 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d83c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f30580c9df0 tx=0x7f3058095c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8d94eb74224ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f304cff9640 1 -- 172.21.15.44:0/2653943207 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058009de0 con 0x55b62d7ce060 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f304cff9640 1 -- 172.21.15.44:0/2653943207 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30580cce90 con 0x55b62d7ce060 2024-09-13T17:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f3062c94640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30480a1280 0x7f30480a3740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3054005fd0 tx=0x7f3054004ab0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.130+0000 7f304cff9640 1 -- 172.21.15.44:0/2653943207 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058099810 con 0x55b62d7ce060 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 -- 172.21.15.44:0/2653943207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 msgr2=0x55b62d7d83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d83c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f30580c9df0 tx=0x7f3058095c80 comp rx=0 tx=0).stop 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 -- 172.21.15.44:0/2653943207 shutdown_connections 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d83c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 --2- 172.21.15.44:0/2653943207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 -- 172.21.15.44:0/2653943207 >> 172.21.15.44:0/2653943207 conn(0x55b62d7c9cd0 msgr2=0x55b62d7ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 -- 172.21.15.44:0/2653943207 shutdown_connections 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.131+0000 7f306741a740 1 -- 172.21.15.44:0/2653943207 wait complete. 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 -- start start 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 Processor -- start 2024-09-13T17:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7ce060 0x55b62d7d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 0x55b62d7d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d6ebe80 con 0x55b62d7d8900 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d6ebb20 con 0x55b62d7cd740 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d6b1260 con 0x55b62d7ce060 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f306741a740 10 client.0 _ll_register_callbacks cb 0x55b62d6e6df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 0x55b62d7d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 0x55b62d7d5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51936/0 (socket says 172.21.15.44:51936) 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062493640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 -- 172.21.15.44:0/393937502 learned_addr learned my addr 172.21.15.44:0/393937502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3061c92640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7ce060 0x55b62d7d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 -- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7ce060 msgr2=0x55b62d7d5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7ce060 0x55b62d7d5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 -- 172.21.15.44:0/393937502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 msgr2=0x55b62d7d8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062c94640 1 -- 172.21.15.44:0/393937502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c1a3270 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.132+0000 7f3062493640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3062c94640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 0x55b62d7d5b50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f305400ebc0 tx=0x7f3054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ad25bd685ea341 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3045ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3045ffb640 1 -- 172.21.15.44:0/393937502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054010d30 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b62d756680 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3045ffb640 1 -- 172.21.15.44:0/393937502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3054004b20 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3045ffb640 1 -- 172.21.15.44:0/393937502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3054010e90 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f306741a740 10 client.16707 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b62d72c330 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3045ffb640 1 -- 172.21.15.44:0/393937502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b62d756680 con 0x55b62d7d8900 2024-09-13T17:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.133+0000 7f3045ffb640 1 -- 172.21.15.44:0/393937502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3054020470 con 0x55b62d7d8900 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f306741a740 10 client.16707 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f306741a740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f306741a740 20 client.16707 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f306741a740 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d7e8320 0x55b62d7ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f306741a740 4 client.16707 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b62d7215d0 con 0x55b62d7e8320 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f3062493640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d7e8320 0x55b62d7ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f3062493640 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d7e8320 0x55b62d7ea740 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83af4740c256368 server_cookie=b39c0f2546df56c4 in_seq=0 out_seq=0 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.134+0000 7f3045ffb640 10 client.16707 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f3045ffb640 1 -- 172.21.15.44:0/393937502 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b62d7215d0 con 0x55b62d7e8320 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f3045ffb640 10 client.16707 handle_command_reply: tid=0 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 client.16707 shutdown 2024-09-13T17:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 msgr2=0x55b62d7d5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 0x55b62d7d5b50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f305400ebc0 tx=0x7f3054004a40 comp rx=0 tx=0).stop 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 shutdown_connections 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d7e8320 0x55b62d7ea740 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7ce060 0x55b62d7d5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7d8900 0x55b62d7d5b50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f3045ffb640 0 client.16707 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 --2- 172.21.15.44:0/393937502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 >> 172.21.15.44:0/393937502 conn(0x55b62d7c9cd0 msgr2=0x55b62d7c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 shutdown_connections 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 1 -- 172.21.15.44:0/393937502 wait complete. 2024-09-13T17:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.135+0000 7f306741a740 20 client.16707 trim_cache size 0 max 16384 2024-09-13T17:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 Processor -- start 2024-09-13T17:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 -- start start 2024-09-13T17:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d6e6d10 con 0x55b62d7ce060 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d7d4890 con 0x55b62d7cd740 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f306741a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d7d3ac0 con 0x55b62d7d8900 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f3061c92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f3061c92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51952/0 (socket says 172.21.15.44:51952) 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f3061c92640 1 -- 172.21.15.44:0/3010771966 learned_addr learned my addr 172.21.15.44:0/3010771966 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f3062493640 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.148+0000 7f3062c94640 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f3061c92640 1 -- 172.21.15.44:0/3010771966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 msgr2=0x55b62d7c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f3061c92640 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f3061c92640 1 -- 172.21.15.44:0/3010771966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 msgr2=0x55b62d7d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f3061c92640 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f3061c92640 1 -- 172.21.15.44:0/3010771966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b62d6ce300 con 0x55b62d7ce060 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f3061c92640 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d4350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f305c069680 tx=0x7f3050004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa452a811cc10822 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f304d7fa640 1 -- 172.21.15.44:0/3010771966 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30500044b0 con 0x55b62d7ce060 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f304d7fa640 1 -- 172.21.15.44:0/3010771966 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3050040070 con 0x55b62d7ce060 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f304d7fa640 1 -- 172.21.15.44:0/3010771966 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305003c900 con 0x55b62d7ce060 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f306741a740 1 -- 172.21.15.44:0/3010771966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 msgr2=0x55b62d7d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f306741a740 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d4350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f305c069680 tx=0x7f3050004140 comp rx=0 tx=0).stop 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f306741a740 1 -- 172.21.15.44:0/3010771966 shutdown_connections 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f306741a740 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f306741a740 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7d4350 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.149+0000 7f306741a740 1 --2- 172.21.15.44:0/3010771966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- 172.21.15.44:0/3010771966 >> 172.21.15.44:0/3010771966 conn(0x55b62d7c9cd0 msgr2=0x55b62d7d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- 172.21.15.44:0/3010771966 shutdown_connections 2024-09-13T17:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- 172.21.15.44:0/3010771966 wait complete. 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- start start 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 Processor -- start 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d7dc960 con 0x55b62d7ce060 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d7dcad0 con 0x55b62d7cd740 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62d83e7e0 con 0x55b62d7d8900 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f306741a740 10 client.0 _ll_register_callbacks cb 0x7f30580d0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f3061c92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f3061c92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7dbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51960/0 (socket says 172.21.15.44:51960) 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f3061c92640 1 -- 172.21.15.44:0/768669788 learned_addr learned my addr 172.21.15.44:0/768669788 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.150+0000 7f3062c94640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3062493640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3061c92640 1 -- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 msgr2=0x55b62d7dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3061c92640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3061c92640 1 -- 172.21.15.44:0/768669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 msgr2=0x55b62d7df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3061c92640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7df8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3061c92640 1 -- 172.21.15.44:0/768669788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c10c300 con 0x55b62d7ce060 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3062493640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3061c92640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7dbd20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f305002a940 tx=0x7f3050004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af02905ae6fadb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3045ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3045ffb640 1 -- 172.21.15.44:0/768669788 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050052070 con 0x55b62d7ce060 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3045ffb640 1 -- 172.21.15.44:0/768669788 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f305003c740 con 0x55b62d7ce060 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30580cce90 con 0x55b62d7ce060 2024-09-13T17:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3045ffb640 1 -- 172.21.15.44:0/768669788 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f305003c900 con 0x55b62d7ce060 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f306741a740 10 client.16713 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b62d6f38e0 con 0x55b62d7ce060 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3045ffb640 1 -- 172.21.15.44:0/768669788 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30580cce90 con 0x55b62d7ce060 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.151+0000 7f3045ffb640 1 -- 172.21.15.44:0/768669788 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3050055020 con 0x55b62d7ce060 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f306741a740 10 client.16713 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f306741a740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f306741a740 20 client.16713 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f306741a740 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d83fb50 0x55b62d841f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f306741a740 4 client.16713 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b62d421300 con 0x55b62d83fb50 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f3062493640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d83fb50 0x55b62d841f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f3062493640 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d83fb50 0x55b62d841f40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4eba6fd0295169d server_cookie=47da033354a9560a in_seq=0 out_seq=0 2024-09-13T17:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.152+0000 7f3045ffb640 10 client.16713 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f3045ffb640 1 -- 172.21.15.44:0/768669788 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b62d421300 con 0x55b62d83fb50 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f3045ffb640 10 client.16713 handle_command_reply: tid=0 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 client.16713 shutdown 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 msgr2=0x55b62d7dbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7dbd20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f305002a940 tx=0x7f3050004420 comp rx=0 tx=0).stop 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 shutdown_connections 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b62d83fb50 0x55b62d841f40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b62d7d8900 0x55b62d7dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b62d7ce060 0x55b62d7dbd20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f3045ffb640 0 client.16713 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 --2- 172.21.15.44:0/768669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b62d7cd740 0x55b62d7df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.166+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 >> 172.21.15.44:0/768669788 conn(0x55b62d7c9cd0 msgr2=0x55b62d7d3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 shutdown_connections 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f306741a740 1 -- 172.21.15.44:0/768669788 wait complete. 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f306741a740 20 client.16713 trim_cache size 0 max 16384 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:30:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30480a1280 msgr2=0x7f30480a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30480a1280 0x7f30480a3740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3054005fd0 tx=0x7f3054004ab0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 msgr2=0x7f305c1961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 0x7f305c1961f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f305800d950 tx=0x7f305800de10 comp rx=0 tx=0).stop 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 shutdown_connections 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30480a1280 0x7f30480a3740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305c103f70 0x7f305c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f305c19ab20 0x7f305c19afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 --2- 172.21.15.44:0/3048088504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c0fee20 0x7f305c1961f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 >> 172.21.15.44:0/3048088504 conn(0x7f305c0fa8f0 msgr2=0x7f305c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 shutdown_connections 2024-09-13T17:30:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:23.167+0000 7f30646f7640 1 -- 172.21.15.44:0/3048088504 wait complete. 2024-09-13T17:30:23.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:23 smithi044 ceph-mon[31175]: pgmap v113: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-13T17:30:23.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:23 smithi070 ceph-mon[34502]: pgmap v113: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-13T17:30:24.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:23 smithi100 ceph-mon[35055]: pgmap v113: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-13T17:30:25.819 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:25 smithi100 ceph-mon[35055]: pgmap v114: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 20 op/s 2024-09-13T17:30:25.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:25 smithi044 ceph-mon[31175]: pgmap v114: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 20 op/s 2024-09-13T17:30:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:25 smithi070 ceph-mon[34502]: pgmap v114: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 20 op/s 2024-09-13T17:30:27.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:27 smithi044 ceph-mon[31175]: pgmap v115: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:27.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:27 smithi070 ceph-mon[34502]: pgmap v115: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:27 smithi100 ceph-mon[35055]: pgmap v115: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:29.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:29 smithi044 ceph-mon[31175]: pgmap v116: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-13T17:30:29.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:29 smithi070 ceph-mon[34502]: pgmap v116: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-13T17:30:30.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:29 smithi100 ceph-mon[35055]: pgmap v116: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-13T17:30:31.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:31 smithi044 ceph-mon[31175]: pgmap v117: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:31.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:31 smithi070 ceph-mon[34502]: pgmap v117: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:31 smithi100 ceph-mon[35055]: pgmap v117: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:33.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:33 smithi044 ceph-mon[31175]: pgmap v118: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:33.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:33 smithi070 ceph-mon[34502]: pgmap v118: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:33 smithi100 ceph-mon[35055]: pgmap v118: 129 pgs: 129 active+clean; 4.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-13T17:30:34.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:30:34.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 -- 172.21.15.44:0/1253775582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 msgr2=0x7f2de010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 --2- 172.21.15.44:0/1253775582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de010bb00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009cd0 tx=0x7f2dd002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 -- 172.21.15.44:0/1253775582 shutdown_connections 2024-09-13T17:30:34.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 --2- 172.21.15.44:0/1253775582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00ff810 0x7f2de0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 --2- 172.21.15.44:0/1253775582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de010bb00 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 --2- 172.21.15.44:0/1253775582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2de00fee60 0x7f2de00ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 -- 172.21.15.44:0/1253775582 >> 172.21.15.44:0/1253775582 conn(0x7f2de0075c60 msgr2=0x7f2de00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 -- 172.21.15.44:0/1253775582 shutdown_connections 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 -- 172.21.15.44:0/1253775582 wait complete. 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.493+0000 7f2de7896640 1 Processor -- start 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 -- start start 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00fee60 0x7f2de0196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de01966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2de019aa90 0x7f2de019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de019b480 con 0x7f2de0103fb0 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de019b5f0 con 0x7f2de019aa90 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de7896640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de019b760 con 0x7f2de00fee60 2024-09-13T17:30:34.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de01966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de01966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52716/0 (socket says 172.21.15.44:52716) 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 -- 172.21.15.44:0/2188072801 learned_addr learned my addr 172.21.15.44:0/2188072801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de5e33640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2de019aa90 0x7f2de019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de5632640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00fee60 0x7f2de0196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 -- 172.21.15.44:0/2188072801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00fee60 msgr2=0x7f2de0196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00fee60 0x7f2de0196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 -- 172.21.15.44:0/2188072801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2de019aa90 msgr2=0x7f2de019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2de019aa90 0x7f2de019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 -- 172.21.15.44:0/2188072801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0009900 con 0x7f2de0103fb0 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de5632640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00fee60 0x7f2de0196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.494+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de01966a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c970 tx=0x7f2dd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.495+0000 7f2dde7fc640 1 -- 172.21.15.44:0/2188072801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4004440 con 0x7f2de0103fb0 2024-09-13T17:30:34.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.495+0000 7f2dde7fc640 1 -- 172.21.15.44:0/2188072801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dd4007cf0 con 0x7f2de0103fb0 2024-09-13T17:30:34.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.495+0000 7f2dde7fc640 1 -- 172.21.15.44:0/2188072801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd400f450 con 0x7f2de0103fb0 2024-09-13T17:30:34.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.495+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de01a2d40 con 0x7f2de0103fb0 2024-09-13T17:30:34.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.495+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de01a31a0 con 0x7f2de0103fb0 2024-09-13T17:30:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.497+0000 7f2dde7fc640 1 -- 172.21.15.44:0/2188072801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2dd4007860 con 0x7f2de0103fb0 2024-09-13T17:30:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 Processor -- start 2024-09-13T17:30:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 -- start start 2024-09-13T17:30:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b43110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b547d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918aadf20 con 0x55d918b43650 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918aab6b0 con 0x55d918b523e0 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918aa6fa0 con 0x55d918b42d30 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dde7fc640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2dc80a1280 0x7f2dc80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2de5632640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2de5632640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b51ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52728/0 (socket says 172.21.15.44:52728) 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2de5632640 1 -- 172.21.15.44:0/2671775414 learned_addr learned my addr 172.21.15.44:0/2671775414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2de5e33640 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b43110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2dde7fc640 1 -- 172.21.15.44:0/2188072801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2dd40cd0d0 con 0x7f2de0103fb0 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2de5e33640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2dc80a1280 0x7f2dc80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.498+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5632640 1 -- 172.21.15.44:0/2671775414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 msgr2=0x55d918b43110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5632640 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b43110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5632640 1 -- 172.21.15.44:0/2671775414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 msgr2=0x55d918b547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5632640 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5632640 1 -- 172.21.15.44:0/2671775414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d918a61840 con 0x55d918b43650 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5632640 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b51ea0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800bd40 tx=0x7f2dd800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d108dcadf97945 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2dc77fe640 1 -- 172.21.15.44:0/2671775414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd801a660 con 0x55d918b43650 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2de5e33640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2dc80a1280 0x7f2dc80a3740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009ca0 tx=0x7f2dd0005c50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2dc77fe640 1 -- 172.21.15.44:0/2671775414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dd800be30 con 0x55d918b43650 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.499+0000 7f2dc77fe640 1 -- 172.21.15.44:0/2671775414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8011980 con 0x55d918b43650 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2671775414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 msgr2=0x55d918b51ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b51ea0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800bd40 tx=0x7f2dd800f730 comp rx=0 tx=0).stop 2024-09-13T17:30:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2671775414 shutdown_connections 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b51ea0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2671775414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b43110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2671775414 >> 172.21.15.44:0/2671775414 conn(0x55d918b3f380 msgr2=0x55d918b3f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2671775414 shutdown_connections 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2671775414 wait complete. 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 -- start start 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.500+0000 7f2dea1fd740 1 Processor -- start 2024-09-13T17:30:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b51d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b4f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918a482a0 con 0x55d918b43650 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d91877f430 con 0x55d918b523e0 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918a2a680 con 0x55d918b42d30 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 10 client.0 _ll_register_callbacks cb 0x55d918aa0610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b4f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b4f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52742/0 (socket says 172.21.15.44:52742) 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 -- 172.21.15.44:0/2564435773 learned_addr learned my addr 172.21.15.44:0/2564435773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de5632640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b51d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de5e33640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 -- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 msgr2=0x55d918b51d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b51d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 -- 172.21.15.44:0/2564435773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 msgr2=0x55d918b4f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b4f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 -- 172.21.15.44:0/2564435773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd800b9a0 con 0x55d918b43650 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de5e33640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b4f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2de4e31640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b4f000 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c940 tx=0x7f2dd4007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a77a533e028304 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/2564435773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40ca920 con 0x55d918b43650 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/2564435773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2dd409dc90 con 0x55d918b43650 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.501+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d91879a4e0 con 0x55d918b43650 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/2564435773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dd40a22e0 con 0x55d918b43650 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 10 client.16725 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d918aa5620 con 0x55d918b43650 2024-09-13T17:30:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/2564435773 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d91879a4e0 con 0x55d918b43650 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/2564435773 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dd4098dc0 con 0x55d918b43650 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 10 client.16725 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 20 client.16725 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918b62170 0x55d918b64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 4 client.16725 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9187ab7e0 con 0x55d918b62170 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.502+0000 7f2de5632640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918b62170 0x55d918b64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.503+0000 7f2dc4ff9640 10 client.16725 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.503+0000 7f2de5632640 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918b62170 0x55d918b64590 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29eabcadbc6800f2 server_cookie=4321f10a46291659 in_seq=0 out_seq=0 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.503+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/2564435773 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d9187ab7e0 con 0x55d918b62170 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.503+0000 7f2dc4ff9640 10 client.16725 handle_command_reply: tid=0 2024-09-13T17:30:34.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.503+0000 7f2dea1fd740 1 client.16725 shutdown 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 msgr2=0x55d918b4f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b4f000 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c940 tx=0x7f2dd4007e60 comp rx=0 tx=0).stop 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 shutdown_connections 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918b62170 0x55d918b64590 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b523e0 0x55d918b4f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b43650 0x55d918b4f000 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dc4ff9640 0 client.16725 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/2564435773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b51d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 >> 172.21.15.44:0/2564435773 conn(0x55d918b3f380 msgr2=0x55d918b45040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 shutdown_connections 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 1 -- 172.21.15.44:0/2564435773 wait complete. 2024-09-13T17:30:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.504+0000 7f2dea1fd740 20 client.16725 trim_cache size 0 max 16384 2024-09-13T17:30:34.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.518+0000 7f2dea1fd740 1 Processor -- start 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.518+0000 7f2dea1fd740 1 -- start start 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.518+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b59140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.518+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 0x55d918b4d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918b49e40 con 0x55d918b523e0 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918b60a50 con 0x55d918b43650 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918b3d9c0 con 0x55d918b42d30 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 0x55d918b4d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 0x55d918b4d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52744/0 (socket says 172.21.15.44:52744) 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 -- 172.21.15.44:0/894317284 learned_addr learned my addr 172.21.15.44:0/894317284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5632640 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b59140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de4e31640 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 -- 172.21.15.44:0/894317284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 msgr2=0x55d918b59140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b59140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 -- 172.21.15.44:0/894317284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 msgr2=0x55d918b59680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b59680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 -- 172.21.15.44:0/894317284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d918a47790 con 0x55d918b523e0 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de4e31640 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2de5e33640 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 0x55d918b4d8b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2dd002ae00 tx=0x7f2dd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18e4fe826099461 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dc7fff640 1 -- 172.21.15.44:0/894317284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0003d40 con 0x55d918b523e0 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dc7fff640 1 -- 172.21.15.44:0/894317284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dd0050070 con 0x55d918b523e0 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.519+0000 7f2dc7fff640 1 -- 172.21.15.44:0/894317284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00576e0 con 0x55d918b523e0 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 -- 172.21.15.44:0/894317284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 msgr2=0x55d918b4d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 0x55d918b4d8b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2dd002ae00 tx=0x7f2dd0004a40 comp rx=0 tx=0).stop 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 -- 172.21.15.44:0/894317284 shutdown_connections 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b523e0 0x55d918b4d8b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/894317284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b42d30 0x55d918b59140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 -- 172.21.15.44:0/894317284 >> 172.21.15.44:0/894317284 conn(0x55d918b3f380 msgr2=0x55d918b4a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 -- 172.21.15.44:0/894317284 shutdown_connections 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 -- 172.21.15.44:0/894317284 wait complete. 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 -- start start 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.520+0000 7f2dea1fd740 1 Processor -- start 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 0x55d918b59530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b55980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b523e0 0x55d918b55ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918b565e0 con 0x55d918b42d30 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918b56750 con 0x55d918b43650 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918bb8670 con 0x55d918b523e0 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2dea1fd740 10 client.0 _ll_register_callbacks cb 0x7f2dd4099a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 0x55d918b59530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 0x55d918b59530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52746/0 (socket says 172.21.15.44:52746) 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 -- 172.21.15.44:0/1596397308 learned_addr learned my addr 172.21.15.44:0/1596397308 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5e33640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b523e0 0x55d918b55ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de4e31640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b55980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 -- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b523e0 msgr2=0x55d918b55ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b523e0 0x55d918b55ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 -- 172.21.15.44:0/1596397308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 msgr2=0x55d918b55980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b55980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de5632640 1 -- 172.21.15.44:0/1596397308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd0046040 con 0x55d918b42d30 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.521+0000 7f2de4e31640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b55980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2de5632640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 0x55d918b59530 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800fc30 tx=0x7f2dd8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63045083d49c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/1596397308 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8005610 con 0x55d918b42d30 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/1596397308 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2dd8004030 con 0x55d918b42d30 2024-09-13T17:30:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dd800be30 con 0x55d918b42d30 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/1596397308 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dd8004210 con 0x55d918b42d30 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 10 client.16731 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d918a6cea0 con 0x55d918b42d30 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/1596397308 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dd800be30 con 0x55d918b42d30 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/1596397308 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dd8026040 con 0x55d918b42d30 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 10 client.16731 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 20 client.16731 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918bb9cd0 0x55d918bbc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 4 client.16731 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.522+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d9187fee30 con 0x55d918bb9cd0 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2de4e31640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918bb9cd0 0x55d918bbc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dc4ff9640 10 client.16731 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2de4e31640 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918bb9cd0 0x55d918bbc0f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ac829c74b7957d server_cookie=f8072d1bde2c2ea5 in_seq=0 out_seq=0 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dc4ff9640 1 -- 172.21.15.44:0/1596397308 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d9187fee30 con 0x55d918bb9cd0 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dc4ff9640 10 client.16731 handle_command_reply: tid=0 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 client.16731 shutdown 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 msgr2=0x55d918b59530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 0x55d918b59530 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800fc30 tx=0x7f2dd8011440 comp rx=0 tx=0).stop 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 shutdown_connections 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d918bb9cd0 0x55d918bbc0f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d918b523e0 0x55d918b55ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d918b43650 0x55d918b55980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dc4ff9640 0 client.16731 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 --2- 172.21.15.44:0/1596397308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d918b42d30 0x55d918b59530 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 >> 172.21.15.44:0/1596397308 conn(0x55d918b3f380 msgr2=0x55d918b4d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 shutdown_connections 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.523+0000 7f2dea1fd740 1 -- 172.21.15.44:0/1596397308 wait complete. 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2dea1fd740 20 client.16731 trim_cache size 0 max 16384 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:30:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:30:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2dc80a1280 msgr2=0x7f2dc80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2dc80a1280 0x7f2dc80a3740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009ca0 tx=0x7f2dd0005c50 comp rx=0 tx=0).stop 2024-09-13T17:30:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 msgr2=0x7f2de01966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de01966a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c970 tx=0x7f2dd400ce30 comp rx=0 tx=0).stop 2024-09-13T17:30:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 shutdown_connections 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2dc80a1280 0x7f2dc80a3740 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2de019aa90 0x7f2de019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de0103fb0 0x7f2de01966a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 --2- 172.21.15.44:0/2188072801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2de00fee60 0x7f2de0196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 >> 172.21.15.44:0/2188072801 conn(0x7f2de0075c60 msgr2=0x7f2de0067710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 shutdown_connections 2024-09-13T17:30:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.524+0000 7f2de7896640 1 -- 172.21.15.44:0/2188072801 wait complete. 2024-09-13T17:30:34.537 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:30:34.537 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:30:34.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 -- 172.21.15.44:0/3798012892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad4100f40 msgr2=0x7fcad4101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 --2- 172.21.15.44:0/3798012892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad4100f40 0x7fcad4101320 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009900 tx=0x7fcac402a480 comp rx=0 tx=0).stop 2024-09-13T17:30:34.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 -- 172.21.15.44:0/3798012892 shutdown_connections 2024-09-13T17:30:34.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 --2- 172.21.15.44:0/3798012892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad4101860 0x7fcad4109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 --2- 172.21.15.44:0/3798012892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcad410a310 0x7fcad410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 --2- 172.21.15.44:0/3798012892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad4100f40 0x7fcad4101320 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 -- 172.21.15.44:0/3798012892 >> 172.21.15.44:0/3798012892 conn(0x7fcad4075c60 msgr2=0x7fcad40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 -- 172.21.15.44:0/3798012892 shutdown_connections 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 -- 172.21.15.44:0/3798012892 wait complete. 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 Processor -- start 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.640+0000 7fcad9318640 1 -- start start 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad4100f40 0x7fcad40feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcad4101860 0x7fcad40ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 0x7fcad40ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad41a4530 con 0x7fcad410a310 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad41a46a0 con 0x7fcad4101860 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad41a4860 con 0x7fcad4100f40 2024-09-13T17:30:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 0x7fcad40ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 0x7fcad40ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52768/0 (socket says 172.21.15.44:52768) 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 -- 172.21.15.44:0/3232937021 learned_addr learned my addr 172.21.15.44:0/3232937021 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad27fc640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcad4101860 0x7fcad40ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad2ffd640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad4100f40 0x7fcad40feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 -- 172.21.15.44:0/3232937021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad4100f40 msgr2=0x7fcad40feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad4100f40 0x7fcad40feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 -- 172.21.15.44:0/3232937021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcad4101860 msgr2=0x7fcad40ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcad4101860 0x7fcad40ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 -- 172.21.15.44:0/3232937021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac4009560 con 0x7fcad410a310 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad37fe640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 0x7fcad40ff610 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00cce0 tx=0x7fcacc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00f450 con 0x7fcad410a310 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcacc00fa40 con 0x7fcad410a310 2024-09-13T17:30:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad41a4b40 con 0x7fcad410a310 2024-09-13T17:30:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0189a0 con 0x7fcad410a310 2024-09-13T17:30:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.641+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad41a5000 con 0x7fcad410a310 2024-09-13T17:30:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.642+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad4067e20 con 0x7fcad410a310 2024-09-13T17:30:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.644+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcacc00fbb0 con 0x7fcad410a310 2024-09-13T17:30:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.644+0000 7fcabbfff640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcabc0a1200 0x7fcabc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.644+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fcacc0cd300 con 0x7fcad410a310 2024-09-13T17:30:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.644+0000 7fcad2ffd640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcabc0a1200 0x7fcabc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.645+0000 7fcad2ffd640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcabc0a1200 0x7fcabc0a36c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcac402aa70 tx=0x7fcac4005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.647+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcacc0964c0 con 0x7fcad410a310 2024-09-13T17:30:34.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.856+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcad4101320 con 0x7fcad410a310 2024-09-13T17:30:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.856+0000 7fcabbfff640 1 -- 172.21.15.44:0/3232937021 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fcacc099ae0 con 0x7fcad410a310 2024-09-13T17:30:34.858 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:30:34.859 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:30:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:30:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcabc0a1200 msgr2=0x7fcabc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcabc0a1200 0x7fcabc0a36c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcac402aa70 tx=0x7fcac4005a20 comp rx=0 tx=0).stop 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 msgr2=0x7fcad40ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 0x7fcad40ff610 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00cce0 tx=0x7fcacc00a620 comp rx=0 tx=0).stop 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 shutdown_connections 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcabc0a1200 0x7fcabc0a36c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad410a310 0x7fcad40ff610 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcad4101860 0x7fcad40ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 --2- 172.21.15.44:0/3232937021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad4100f40 0x7fcad40feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 >> 172.21.15.44:0/3232937021 conn(0x7fcad4075c60 msgr2=0x7fcad40fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 shutdown_connections 2024-09-13T17:30:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.858+0000 7fcad9318640 1 -- 172.21.15.44:0/3232937021 wait complete. 2024-09-13T17:30:34.871 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 -- 172.21.15.44:0/2613176766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8474100e50 msgr2=0x7f8474101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 --2- 172.21.15.44:0/2613176766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8474100e50 0x7f8474101230 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002a410 comp rx=0 tx=0).stop 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 -- 172.21.15.44:0/2613176766 shutdown_connections 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 --2- 172.21.15.44:0/2613176766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8474101800 0x7f8474109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 --2- 172.21.15.44:0/2613176766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f847410a350 0x7f847410c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 --2- 172.21.15.44:0/2613176766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8474100e50 0x7f8474101230 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 -- 172.21.15.44:0/2613176766 >> 172.21.15.44:0/2613176766 conn(0x7f84740fa930 msgr2=0x7f84740fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 -- 172.21.15.44:0/2613176766 shutdown_connections 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.973+0000 7f847bbb5640 1 -- 172.21.15.44:0/2613176766 wait complete. 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 Processor -- start 2024-09-13T17:30:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 -- start start 2024-09-13T17:30:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8474100e50 0x7f84740feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8474101800 0x7f84740ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 0x7f84740ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741a4520 con 0x7f847410a350 2024-09-13T17:30:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741a4690 con 0x7f8474101800 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847bbb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741a4850 con 0x7f8474100e50 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847a152640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 0x7f84740ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847a152640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 0x7f84740ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52780/0 (socket says 172.21.15.44:52780) 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f847a152640 1 -- 172.21.15.44:0/1915394614 learned_addr learned my addr 172.21.15.44:0/1915394614 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f8479951640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8474100e50 0x7f84740feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.974+0000 7f8479150640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8474101800 0x7f84740ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847a152640 1 -- 172.21.15.44:0/1915394614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8474100e50 msgr2=0x7f84740feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847a152640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8474100e50 0x7f84740feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847a152640 1 -- 172.21.15.44:0/1915394614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8474101800 msgr2=0x7f84740ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847a152640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8474101800 0x7f84740ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847a152640 1 -- 172.21.15.44:0/1915394614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009560 con 0x7f847410a350 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847a152640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 0x7f84740ff5b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f846c009fd0 tx=0x7f846c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f846affd640 1 -- 172.21.15.44:0/1915394614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00bea0 con 0x7f847410a350 2024-09-13T17:30:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f846affd640 1 -- 172.21.15.44:0/1915394614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f846c009940 con 0x7f847410a350 2024-09-13T17:30:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f846affd640 1 -- 172.21.15.44:0/1915394614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c010480 con 0x7f847410a350 2024-09-13T17:30:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84741a4b30 con 0x7f847410a350 2024-09-13T17:30:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.975+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84741a4ff0 con 0x7f847410a350 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.977+0000 7f846affd640 1 -- 172.21.15.44:0/1915394614 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f846c003970 con 0x7f847410a350 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.977+0000 7f847e8d5740 1 Processor -- start 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 -- start start 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f0888b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f0893410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0893950 0x55d4f0895d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f07f2cc0 con 0x55d4f0888790 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f07f0450 con 0x55d4f0893950 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847e8d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f07ebd40 con 0x55d4f08890b0 2024-09-13T17:30:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f0888b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f0888b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52790/0 (socket says 172.21.15.44:52790) 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 -- 172.21.15.44:0/764320029 learned_addr learned my addr 172.21.15.44:0/764320029 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479951640 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f0893410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0893950 0x55d4f0895d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f846affd640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84600a1280 0x7f84600a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 -- 172.21.15.44:0/764320029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 msgr2=0x55d4f0893410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f0893410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 -- 172.21.15.44:0/764320029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0893950 msgr2=0x55d4f0895d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0893950 0x55d4f0895d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f8479150640 1 -- 172.21.15.44:0/764320029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4f07b3620 con 0x55d4f0888790 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.978+0000 7f847a152640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84600a1280 0x7f84600a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.979+0000 7f846affd640 1 -- 172.21.15.44:0/1915394614 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f846c0ccbb0 con 0x7f847410a350 2024-09-13T17:30:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.979+0000 7f8479150640 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f0888b70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f846400bfa0 tx=0x7f846400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53c1b8a257a5afb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.979+0000 7f845ffff640 1 -- 172.21.15.44:0/764320029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464004030 con 0x55d4f0888790 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.979+0000 7f845ffff640 1 -- 172.21.15.44:0/764320029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84640035f0 con 0x55d4f0888790 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.979+0000 7f845ffff640 1 -- 172.21.15.44:0/764320029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640106a0 con 0x55d4f0888790 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 -- 172.21.15.44:0/764320029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 msgr2=0x55d4f0888b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847a152640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84600a1280 0x7f84600a3740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f846c095ef0 tx=0x7f846c06bad0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f0888b70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f846400bfa0 tx=0x7f846400b840 comp rx=0 tx=0).stop 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 -- 172.21.15.44:0/764320029 shutdown_connections 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f0893410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0893950 0x55d4f0895d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 --2- 172.21.15.44:0/764320029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f0888b70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 -- 172.21.15.44:0/764320029 >> 172.21.15.44:0/764320029 conn(0x55d4f0884d00 msgr2=0x55d4f08850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 -- 172.21.15.44:0/764320029 shutdown_connections 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 -- 172.21.15.44:0/764320029 wait complete. 2024-09-13T17:30:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 -- start start 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.980+0000 7f847e8d5740 1 Processor -- start 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0888790 0x55d4f0892ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 0x55d4f0890350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0893950 0x55d4f0890890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f07aa180 con 0x55d4f08890b0 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f04c4430 con 0x55d4f0888790 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f076f5e0 con 0x55d4f0893950 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 10 client.0 _ll_register_callbacks cb 0x55d4f07dbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 0x55d4f0890350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 0x55d4f0890350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52794/0 (socket says 172.21.15.44:52794) 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 -- 172.21.15.44:0/3185630688 learned_addr learned my addr 172.21.15.44:0/3185630688 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479150640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0893950 0x55d4f0890890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479150640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0888790 0x55d4f0892ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 -- 172.21.15.44:0/3185630688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0893950 msgr2=0x55d4f0890890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0893950 0x55d4f0890890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 -- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0888790 msgr2=0x55d4f0892ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0888790 0x55d4f0892ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 -- 172.21.15.44:0/3185630688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846400bd80 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479150640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0888790 0x55d4f0892ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f8479951640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 0x55d4f0890350 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f847002a930 tx=0x7f8470004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe301fb59a6094c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f845d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f845d7fa640 1 -- 172.21.15.44:0/3185630688 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470004520 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f845d7fa640 1 -- 172.21.15.44:0/3185630688 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8470031d70 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4f0815b80 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f845d7fa640 1 -- 172.21.15.44:0/3185630688 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84700035f0 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 10 client.16749 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.981+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4f07ea3c0 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f845d7fa640 1 -- 172.21.15.44:0/3185630688 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d4f0815b80 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f845d7fa640 1 -- 172.21.15.44:0/3185630688 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f847003d7b0 con 0x55d4f08890b0 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f847e8d5740 10 client.16749 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f847e8d5740 10 client.16749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f847e8d5740 20 client.16749 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f08a3280 0x55d4f08a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f847e8d5740 4 client.16749 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4f07de1f0 con 0x55d4f08a3280 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.982+0000 7f8479150640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f08a3280 0x55d4f08a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.983+0000 7f845d7fa640 10 client.16749 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.983+0000 7f8479150640 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f08a3280 0x55d4f08a56a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ef4d395bb0b4cc4 server_cookie=b73e09fd370e64ee in_seq=0 out_seq=0 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.983+0000 7f845d7fa640 1 -- 172.21.15.44:0/3185630688 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d4f07de1f0 con 0x55d4f08a3280 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.983+0000 7f845d7fa640 10 client.16749 handle_command_reply: tid=0 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.983+0000 7f847e8d5740 1 client.16749 shutdown 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 msgr2=0x55d4f0890350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 0x55d4f0890350 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f847002a930 tx=0x7f8470004270 comp rx=0 tx=0).stop 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 shutdown_connections 2024-09-13T17:30:34.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f08a3280 0x55d4f08a56a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0893950 0x55d4f0890890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f08890b0 0x55d4f0890350 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f845d7fa640 0 client.16749 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3185630688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0888790 0x55d4f0892ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 >> 172.21.15.44:0/3185630688 conn(0x55d4f0884d00 msgr2=0x55d4f088aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 shutdown_connections 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 1 -- 172.21.15.44:0/3185630688 wait complete. 2024-09-13T17:30:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.984+0000 7f847e8d5740 20 client.16749 trim_cache size 0 max 16384 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 Processor -- start 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 -- start start 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0884d00 0x55d4f089a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f089ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f088ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f089ffb0 con 0x55d4f0888790 2024-09-13T17:30:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f08960c0 con 0x55d4f0884d00 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f847e8d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f0890c00 con 0x55d4f08890b0 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f8479951640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f089ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f8479951640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f089ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52808/0 (socket says 172.21.15.44:52808) 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f8479951640 1 -- 172.21.15.44:0/239204156 learned_addr learned my addr 172.21.15.44:0/239204156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.995+0000 7f8479150640 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0884d00 0x55d4f089a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479150640 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f088ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479951640 1 -- 172.21.15.44:0/239204156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 msgr2=0x55d4f088ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479951640 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f088ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479951640 1 -- 172.21.15.44:0/239204156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0884d00 msgr2=0x55d4f089a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479951640 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0884d00 0x55d4f089a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479951640 1 -- 172.21.15.44:0/239204156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4f078c770 con 0x55d4f0888790 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479150640 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0884d00 0x55d4f089a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8479951640 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f089ab50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f84700318e0 tx=0x7f847002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4755a1e0f0983dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8468e05640 1 -- 172.21.15.44:0/239204156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700042e0 con 0x55d4f0888790 2024-09-13T17:30:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8468e05640 1 -- 172.21.15.44:0/239204156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f847003d440 con 0x55d4f0888790 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f8468e05640 1 -- 172.21.15.44:0/239204156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470003a80 con 0x55d4f0888790 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f847e8d5740 1 -- 172.21.15.44:0/239204156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 msgr2=0x55d4f089ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.996+0000 7f847e8d5740 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f089ab50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f84700318e0 tx=0x7f847002a410 comp rx=0 tx=0).stop 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- 172.21.15.44:0/239204156 shutdown_connections 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f08890b0 0x55d4f088ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0888790 0x55d4f089ab50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 --2- 172.21.15.44:0/239204156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f0884d00 0x55d4f089a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- 172.21.15.44:0/239204156 >> 172.21.15.44:0/239204156 conn(0x55d4f0893050 msgr2=0x55d4f08aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- 172.21.15.44:0/239204156 shutdown_connections 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- 172.21.15.44:0/239204156 wait complete. 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- start start 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 Processor -- start 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 0x55d4f089a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0888790 0x55d4f0896bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f08890b0 0x55d4f0897110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f0897650 con 0x55d4f0884d00 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f08977c0 con 0x55d4f08890b0 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f0897980 con 0x55d4f0888790 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f847e8d5740 10 client.0 _ll_register_callbacks cb 0x7f846400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f8479150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 0x55d4f089a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f8479150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 0x55d4f089a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52812/0 (socket says 172.21.15.44:52812) 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.997+0000 7f8479150640 1 -- 172.21.15.44:0/3980490538 learned_addr learned my addr 172.21.15.44:0/3980490538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479951640 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0888790 0x55d4f0896bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479150640 1 -- 172.21.15.44:0/3980490538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0888790 msgr2=0x55d4f0896bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479150640 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0888790 0x55d4f0896bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479150640 1 -- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f08890b0 msgr2=0x55d4f0897110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479150640 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f08890b0 0x55d4f0897110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479150640 1 -- 172.21.15.44:0/3980490538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4f07ea3c0 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f8479150640 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 0x55d4f089a640 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f846400c010 tx=0x7f8464005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc1ad3f35a3f308 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f845d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f845d7fa640 1 -- 172.21.15.44:0/3980490538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640054c0 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f845d7fa640 1 -- 172.21.15.44:0/3980490538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f8464005620 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84640035f0 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f845d7fa640 1 -- 172.21.15.44:0/3980490538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8464003760 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f847e8d5740 10 client.16755 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4f07b1e80 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.998+0000 7f845d7fa640 1 -- 172.21.15.44:0/3980490538 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84640035f0 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f845d7fa640 1 -- 172.21.15.44:0/3980490538 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8464033070 con 0x55d4f0884d00 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f847e8d5740 10 client.16755 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f847e8d5740 10 client.16755 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:30:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f847e8d5740 20 client.16755 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f0893950 0x55d4f08fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f847e8d5740 4 client.16755 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d4f04df4e0 con 0x55d4f0893950 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f8479951640 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f0893950 0x55d4f08fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f845d7fa640 10 client.16755 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:34.999+0000 7f8479951640 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f0893950 0x55d4f08fca90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f865392b99a7e9c server_cookie=83d776e7de232832 in_seq=0 out_seq=0 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f845d7fa640 1 -- 172.21.15.44:0/3980490538 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d4f04df4e0 con 0x55d4f0893950 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f845d7fa640 10 client.16755 handle_command_reply: tid=0 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 client.16755 shutdown 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 msgr2=0x55d4f089a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 0x55d4f089a640 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f846400c010 tx=0x7f8464005bf0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 shutdown_connections 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55d4f0893950 0x55d4f08fca90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4f0888790 0x55d4f0896bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f845d7fa640 0 client.16755 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:30:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4f08890b0 0x55d4f0897110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 --2- 172.21.15.44:0/3980490538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4f0884d00 0x55d4f089a640 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 >> 172.21.15.44:0/3980490538 conn(0x55d4f0893050 msgr2=0x55d4f088eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 shutdown_connections 2024-09-13T17:30:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 1 -- 172.21.15.44:0/3980490538 wait complete. 2024-09-13T17:30:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847e8d5740 20 client.16755 trim_cache size 0 max 16384 2024-09-13T17:30:35.008 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84600a1280 msgr2=0x7f84600a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847bbb5640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84600a1280 0x7f84600a3740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f846c095ef0 tx=0x7f846c06bad0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 msgr2=0x7f84740ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847bbb5640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 0x7f84740ff5b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f846c009fd0 tx=0x7f846c00b260 comp rx=0 tx=0).stop 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 shutdown_connections 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.000+0000 7f847bbb5640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84600a1280 0x7f84600a3740 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.001+0000 7f847bbb5640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f847410a350 0x7f84740ff5b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.001+0000 7f847bbb5640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8474101800 0x7f84740ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.001+0000 7f847bbb5640 1 --2- 172.21.15.44:0/1915394614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8474100e50 0x7f84740feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.001+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 >> 172.21.15.44:0/1915394614 conn(0x7f84740fa930 msgr2=0x7f84740fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.001+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 shutdown_connections 2024-09-13T17:30:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.001+0000 7f847bbb5640 1 -- 172.21.15.44:0/1915394614 wait complete. 2024-09-13T17:30:35.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:30:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 -- 172.21.15.44:0/2783338183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc0fee80 msgr2=0x7f84fc0ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 --2- 172.21.15.44:0/2783338183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc0fee80 0x7f84fc0ff260 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009930 tx=0x7f84f802a420 comp rx=0 tx=0).stop 2024-09-13T17:30:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 -- 172.21.15.44:0/2783338183 shutdown_connections 2024-09-13T17:30:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 --2- 172.21.15.44:0/2783338183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84fc0ff830 0x7f84fc103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 --2- 172.21.15.44:0/2783338183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84fc103fd0 0x7f84fc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 --2- 172.21.15.44:0/2783338183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc0fee80 0x7f84fc0ff260 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 -- 172.21.15.44:0/2783338183 >> 172.21.15.44:0/2783338183 conn(0x7f84fc0fa8d0 msgr2=0x7f84fc0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 -- 172.21.15.44:0/2783338183 shutdown_connections 2024-09-13T17:30:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.115+0000 7f8503ae5640 1 -- 172.21.15.44:0/2783338183 wait complete. 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 Processor -- start 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 -- start start 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84fc0fee80 0x7f84fc196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 0x7f84fc196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84fc19aac0 0x7f84fc19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19b4b0 con 0x7f84fc103fd0 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19b620 con 0x7f84fc0fee80 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8503ae5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19b790 con 0x7f84fc19aac0 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 0x7f84fc196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 0x7f84fc196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52820/0 (socket says 172.21.15.44:52820) 2024-09-13T17:30:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 -- 172.21.15.44:0/3266391501 learned_addr learned my addr 172.21.15.44:0/3266391501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501881640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84fc0fee80 0x7f84fc196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8502082640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84fc19aac0 0x7f84fc19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 -- 172.21.15.44:0/3266391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84fc19aac0 msgr2=0x7f84fc19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84fc19aac0 0x7f84fc19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 -- 172.21.15.44:0/3266391501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84fc0fee80 msgr2=0x7f84fc196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84fc0fee80 0x7f84fc196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.116+0000 7f8501080640 1 -- 172.21.15.44:0/3266391501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009560 con 0x7f84fc103fd0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.117+0000 7f8501080640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 0x7f84fc196740 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c970 tx=0x7f84ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.117+0000 7f84f2ffd640 1 -- 172.21.15.44:0/3266391501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec007bb0 con 0x7f84fc103fd0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.117+0000 7f84f2ffd640 1 -- 172.21.15.44:0/3266391501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ec007d10 con 0x7f84fc103fd0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.117+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc1a2d70 con 0x7f84fc103fd0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.117+0000 7f84f2ffd640 1 -- 172.21.15.44:0/3266391501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec018660 con 0x7f84fc103fd0 2024-09-13T17:30:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.117+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc1a31d0 con 0x7f84fc103fd0 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.119+0000 7f84f2ffd640 1 -- 172.21.15.44:0/3266391501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f84ec00f450 con 0x7f84fc103fd0 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.119+0000 7f8506803740 1 Processor -- start 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 -- start start 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x56186711ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x56186712e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561867086cc0 con 0x56186712bd60 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561867084450 con 0x56186711cf90 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8506803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186707fd40 con 0x56186711c670 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x56186712e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x56186712e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52834/0 (socket says 172.21.15.44:52834) 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 -- 172.21.15.44:0/1490289099 learned_addr learned my addr 172.21.15.44:0/1490289099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8502082640 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x56186711ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501881640 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f84f2ffd640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84e80a1200 0x7f84e80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 -- 172.21.15.44:0/1490289099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 msgr2=0x56186711ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x56186711ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 -- 172.21.15.44:0/1490289099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 msgr2=0x56186712b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8501080640 1 -- 172.21.15.44:0/1490289099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561867047620 con 0x56186712bd60 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8502082640 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x56186711ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f8502082640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84e80a1200 0x7f84e80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.120+0000 7f84f2ffd640 1 -- 172.21.15.44:0/3266391501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f84ec0cd160 con 0x7f84fc103fd0 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.121+0000 7f8501080640 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x56186712e150 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f84ec04ab00 tx=0x7f84ec021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7056dd3b6c53212d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.121+0000 7f8502082640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84e80a1200 0x7f84e80a36c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f84f4005fd0 tx=0x7f84f4004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.121+0000 7f84e7fff640 1 -- 172.21.15.44:0/1490289099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec099cc0 con 0x56186712bd60 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.121+0000 7f84e7fff640 1 -- 172.21.15.44:0/1490289099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ec099a30 con 0x56186712bd60 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.121+0000 7f84e7fff640 1 -- 172.21.15.44:0/1490289099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0d0cc0 con 0x56186712bd60 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- 172.21.15.44:0/1490289099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 msgr2=0x56186712e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x56186712e150 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f84ec04ab00 tx=0x7f84ec021000 comp rx=0 tx=0).stop 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- 172.21.15.44:0/1490289099 shutdown_connections 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x56186712e150 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- 172.21.15.44:0/1490289099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x56186711ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- 172.21.15.44:0/1490289099 >> 172.21.15.44:0/1490289099 conn(0x561867118d00 msgr2=0x5618671190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- 172.21.15.44:0/1490289099 shutdown_connections 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- 172.21.15.44:0/1490289099 wait complete. 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- start start 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 Processor -- start 2024-09-13T17:30:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x561867128650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x561867128b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186703e180 con 0x56186711c670 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561866d58430 con 0x56186711cf90 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.122+0000 7f8506803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618670035e0 con 0x56186712bd60 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8506803740 10 client.0 _ll_register_callbacks cb 0x56186706faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52848/0 (socket says 172.21.15.44:52848) 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 -- 172.21.15.44:0/2127665776 learned_addr learned my addr 172.21.15.44:0/2127665776 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8502082640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x561867128b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501080640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x561867128650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 -- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 msgr2=0x561867128b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x561867128b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 -- 172.21.15.44:0/2127665776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 msgr2=0x561867128650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x561867128650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 -- 172.21.15.44:0/2127665776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc1a31d0 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8502082640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x561867128b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501080640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x561867128650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8501881640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712b380 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f84f8005940 tx=0x7f84f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cbaad5134cc0a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f84e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f84e57fa640 1 -- 172.21.15.44:0/2127665776 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8004520 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f84e57fa640 1 -- 172.21.15.44:0/2127665776 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f84f8031b60 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618670a9b80 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f84e57fa640 1 -- 172.21.15.44:0/2127665776 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84f8031d20 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8506803740 10 client.16767 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.123+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56186707e3c0 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f84e57fa640 1 -- 172.21.15.44:0/2127665776 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5618670a9b80 con 0x56186711c670 2024-09-13T17:30:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f84e57fa640 1 -- 172.21.15.44:0/2127665776 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84f8046500 con 0x56186711c670 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8506803740 10 client.16767 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8506803740 10 client.16767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8506803740 20 client.16767 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8506803740 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56186713b710 0x56186713db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8506803740 4 client.16767 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618670721f0 con 0x56186713b710 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.124+0000 7f8501080640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56186713b710 0x56186713db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8501080640 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56186713b710 0x56186713db00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=527b8b5e1008be60 server_cookie=754cfd704df39496 in_seq=0 out_seq=0 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f84e57fa640 10 client.16767 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f84e57fa640 1 -- 172.21.15.44:0/2127665776 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5618670721f0 con 0x56186713b710 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f84e57fa640 10 client.16767 handle_command_reply: tid=0 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 client.16767 shutdown 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 msgr2=0x56186712b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712b380 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f84f8005940 tx=0x7f84f8004270 comp rx=0 tx=0).stop 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 shutdown_connections 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56186713b710 0x56186713db00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x561867128b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x561867128650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 --2- 172.21.15.44:0/2127665776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712b380 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f84e57fa640 0 client.16767 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 >> 172.21.15.44:0/2127665776 conn(0x561867118d00 msgr2=0x56186711ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 shutdown_connections 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.125+0000 7f8506803740 1 -- 172.21.15.44:0/2127665776 wait complete. 2024-09-13T17:30:35.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.126+0000 7f8506803740 20 client.16767 trim_cache size 0 max 16384 2024-09-13T17:30:35.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 Processor -- start 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 -- start start 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x5618671237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618670a9bb0 con 0x56186711c670 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186711dc90 con 0x56186711cf90 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8506803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618671146b0 con 0x56186712bd60 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52856/0 (socket says 172.21.15.44:52856) 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 -- 172.21.15.44:0/3573828260 learned_addr learned my addr 172.21.15.44:0/3573828260 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 -- 172.21.15.44:0/3573828260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 msgr2=0x5618671237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8502082640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x5618671237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501080640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x5618671237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 -- 172.21.15.44:0/3573828260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 msgr2=0x56186712b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 -- 172.21.15.44:0/3573828260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561867020770 con 0x56186711c670 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8502082640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x5618671237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501080640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.137+0000 7f8501881640 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712ae20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009850 tx=0x7f84f802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d612487b3c8944a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f84f0e05640 1 -- 172.21.15.44:0/3573828260 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8031400 con 0x56186711c670 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f84f0e05640 1 -- 172.21.15.44:0/3573828260 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84f8041070 con 0x56186711c670 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f84f0e05640 1 -- 172.21.15.44:0/3573828260 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8003850 con 0x56186711c670 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 -- 172.21.15.44:0/3573828260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 msgr2=0x56186712ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712ae20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009850 tx=0x7f84f802a420 comp rx=0 tx=0).stop 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 -- 172.21.15.44:0/3573828260 shutdown_connections 2024-09-13T17:30:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186712bd60 0x5618671237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x56186712b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 --2- 172.21.15.44:0/3573828260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186711c670 0x56186712ae20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 -- 172.21.15.44:0/3573828260 >> 172.21.15.44:0/3573828260 conn(0x561867118d00 msgr2=0x561867126f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 -- 172.21.15.44:0/3573828260 shutdown_connections 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 -- 172.21.15.44:0/3573828260 wait complete. 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.138+0000 7f8506803740 1 -- start start 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 Processor -- start 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x561867131080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x5618671315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x561867132460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561867132a20 con 0x56186712bd60 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561867132b90 con 0x56186711cf90 2024-09-13T17:30:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561867132d50 con 0x56186711c670 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8506803740 10 client.0 _ll_register_callbacks cb 0x7f84ec0d0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x561867132460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x561867132460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52866/0 (socket says 172.21.15.44:52866) 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 -- 172.21.15.44:0/1669303100 learned_addr learned my addr 172.21.15.44:0/1669303100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8501881640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x561867131080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8501080640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x5618671315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 -- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 msgr2=0x561867131080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x561867131080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 -- 172.21.15.44:0/1669303100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 msgr2=0x5618671315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x5618671315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.139+0000 7f8502082640 1 -- 172.21.15.44:0/1669303100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56186707e3c0 con 0x56186712bd60 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8502082640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x561867132460 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f84f40106e0 tx=0x7f84f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4e902db880b927 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f84e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:35.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f84e57fa640 1 -- 172.21.15.44:0/1669303100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4024070 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84ec099a30 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f84e57fa640 1 -- 172.21.15.44:0/1669303100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f84f401f6a0 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f84e57fa640 1 -- 172.21.15.44:0/1669303100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84f401f2e0 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 10 client.16773 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561867045e80 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f84e57fa640 1 -- 172.21.15.44:0/1669303100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ec099a30 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f84e57fa640 1 -- 172.21.15.44:0/1669303100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84f401d960 con 0x56186712bd60 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 10 client.16773 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 10 client.16773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 20 client.16773 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561867192ed0 0x5618671952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 4 client.16773 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.140+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561866d734e0 con 0x561867192ed0 2024-09-13T17:30:35.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8501881640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561867192ed0 0x5618671952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8501881640 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561867192ed0 0x5618671952f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c94b8a85d7bbba0e server_cookie=67c75fac8598a00f in_seq=0 out_seq=0 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f84e57fa640 10 client.16773 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f84e57fa640 1 -- 172.21.15.44:0/1669303100 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561866d734e0 con 0x561867192ed0 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f84e57fa640 10 client.16773 handle_command_reply: tid=0 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 client.16773 shutdown 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 msgr2=0x561867132460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x561867132460 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f84f40106e0 tx=0x7f84f4004ab0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 shutdown_connections 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x561867192ed0 0x5618671952f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56186712bd60 0x561867132460 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56186711cf90 0x5618671315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 --2- 172.21.15.44:0/1669303100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56186711c670 0x561867131080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f84e57fa640 0 client.16773 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 >> 172.21.15.44:0/1669303100 conn(0x561867118d00 msgr2=0x561867126ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 shutdown_connections 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.141+0000 7f8506803740 1 -- 172.21.15.44:0/1669303100 wait complete. 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8506803740 20 client.16773 trim_cache size 0 max 16384 2024-09-13T17:30:35.150 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84e80a1200 msgr2=0x7f84e80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84e80a1200 0x7f84e80a36c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f84f4005fd0 tx=0x7f84f4004a40 comp rx=0 tx=0).stop 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 msgr2=0x7f84fc196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 0x7f84fc196740 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c970 tx=0x7f84ec00ce30 comp rx=0 tx=0).stop 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 shutdown_connections 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f84e80a1200 0x7f84e80a36c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84fc19aac0 0x7f84fc19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84fc103fd0 0x7f84fc196740 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 --2- 172.21.15.44:0/3266391501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84fc0fee80 0x7f84fc196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 >> 172.21.15.44:0/3266391501 conn(0x7f84fc0fa8d0 msgr2=0x7f84fc0688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 shutdown_connections 2024-09-13T17:30:35.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.142+0000 7f8503ae5640 1 -- 172.21.15.44:0/3266391501 wait complete. 2024-09-13T17:30:35.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:30:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1473170881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e80ff830 msgr2=0x7f95e8103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1473170881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e80ff830 0x7f95e8103870 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009930 tx=0x7f95d802a420 comp rx=0 tx=0).stop 2024-09-13T17:30:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1473170881 shutdown_connections 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1473170881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8103fd0 0x7f95e810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1473170881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e80ff830 0x7f95e8103870 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1473170881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e80fee80 0x7f95e80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1473170881 >> 172.21.15.44:0/1473170881 conn(0x7f95e80fa8d0 msgr2=0x7f95e80fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1473170881 shutdown_connections 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.257+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1473170881 wait complete. 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 Processor -- start 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 -- start start 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e80fee80 0x7f95e8196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 0x7f95e81966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e819aaa0 0x7f95e819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819b490 con 0x7f95e8103fd0 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819b600 con 0x7f95e819aaa0 2024-09-13T17:30:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ef6aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819b770 con 0x7f95e80fee80 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 0x7f95e81966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 0x7f95e81966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52894/0 (socket says 172.21.15.44:52894) 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 -- 172.21.15.44:0/1830227570 learned_addr learned my addr 172.21.15.44:0/1830227570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95edc47640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e819aaa0 0x7f95e819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ed446640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e80fee80 0x7f95e8196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 -- 172.21.15.44:0/1830227570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e80fee80 msgr2=0x7f95e8196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e80fee80 0x7f95e8196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 -- 172.21.15.44:0/1830227570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e819aaa0 msgr2=0x7f95e819af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e819aaa0 0x7f95e819af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 -- 172.21.15.44:0/1830227570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d8009560 con 0x7f95e8103fd0 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.258+0000 7f95ecc45640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 0x7f95e81966b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005ab0 tx=0x7f95d8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.259+0000 7f95de7fc640 1 -- 172.21.15.44:0/1830227570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8032e10 con 0x7f95e8103fd0 2024-09-13T17:30:35.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.259+0000 7f95de7fc640 1 -- 172.21.15.44:0/1830227570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95d80046c0 con 0x7f95e8103fd0 2024-09-13T17:30:35.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.259+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e81a2d50 con 0x7f95e8103fd0 2024-09-13T17:30:35.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.259+0000 7f95de7fc640 1 -- 172.21.15.44:0/1830227570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8003830 con 0x7f95e8103fd0 2024-09-13T17:30:35.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.259+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e81a31d0 con 0x7f95e8103fd0 2024-09-13T17:30:35.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.261+0000 7f95de7fc640 1 -- 172.21.15.44:0/1830227570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f95d8004830 con 0x7f95e8103fd0 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.261+0000 7f95f23ce740 1 Processor -- start 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.261+0000 7f95f23ce740 1 -- start start 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.261+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d663b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.261+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d670d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95f23ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d5cdcc0 con 0x55940d6640b0 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95f23ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d5cb450 con 0x55940d66e950 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95f23ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d5c6d40 con 0x55940d663790 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52906/0 (socket says 172.21.15.44:52906) 2024-09-13T17:30:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 -- 172.21.15.44:0/1488318205 learned_addr learned my addr 172.21.15.44:0/1488318205 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95edc47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d663b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ecc45640 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d670d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95de7fc640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95d40a1280 0x7f95d40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 -- 172.21.15.44:0/1488318205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 msgr2=0x55940d663b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d663b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 -- 172.21.15.44:0/1488318205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 msgr2=0x55940d670d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d670d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 -- 172.21.15.44:0/1488318205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55940d58e620 con 0x55940d6640b0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95edc47640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95d40a1280 0x7f95d40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95de7fc640 1 -- 172.21.15.44:0/1830227570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f95d80edad0 con 0x7f95e8103fd0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95ed446640 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66e410 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f95e400ad00 tx=0x7f95e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea874fd90902c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95d37fe640 1 -- 172.21.15.44:0/1488318205 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400a800 con 0x55940d6640b0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95d37fe640 1 -- 172.21.15.44:0/1488318205 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95e4018ad0 con 0x55940d6640b0 2024-09-13T17:30:35.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.262+0000 7f95d37fe640 1 -- 172.21.15.44:0/1488318205 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4010b40 con 0x55940d6640b0 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 -- 172.21.15.44:0/1488318205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 msgr2=0x55940d66e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95edc47640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95d40a1280 0x7f95d40a3740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e0004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66e410 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f95e400ad00 tx=0x7f95e400f790 comp rx=0 tx=0).stop 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 -- 172.21.15.44:0/1488318205 shutdown_connections 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66e410 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 --2- 172.21.15.44:0/1488318205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d663b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 -- 172.21.15.44:0/1488318205 >> 172.21.15.44:0/1488318205 conn(0x55940d65fd00 msgr2=0x55940d6600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 -- 172.21.15.44:0/1488318205 shutdown_connections 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.263+0000 7f95f23ce740 1 -- 172.21.15.44:0/1488318205 wait complete. 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 -- start start 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 Processor -- start 2024-09-13T17:30:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d66df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d66b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d585180 con 0x55940d6640b0 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d29f430 con 0x55940d66e950 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d54a5e0 con 0x55940d663790 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95f23ce740 10 client.0 _ll_register_callbacks cb 0x55940d5b6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52918/0 (socket says 172.21.15.44:52918) 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 -- 172.21.15.44:0/610157184 learned_addr learned my addr 172.21.15.44:0/610157184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 -- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 msgr2=0x55940d66df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d66df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95edc47640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d66b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 -- 172.21.15.44:0/610157184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 msgr2=0x55940d66b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d66b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95ecc45640 1 -- 172.21.15.44:0/610157184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e400a960 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.264+0000 7f95edc47640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d66b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95ecc45640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66b1e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95d8043000 tx=0x7f95d80b6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da45f068e262836 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95d0ff9640 1 -- 172.21.15.44:0/610157184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80eb380 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95d0ff9640 1 -- 172.21.15.44:0/610157184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f95d80bafc0 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55940d5f0b80 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95d0ff9640 1 -- 172.21.15.44:0/610157184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95d8004d20 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 10 client.16785 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55940d5c53c0 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95d0ff9640 1 -- 172.21.15.44:0/610157184 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95d80d2070 con 0x55940d6640b0 2024-09-13T17:30:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95d0ff9640 1 -- 172.21.15.44:0/610157184 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95d80b9db0 con 0x55940d6640b0 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 10 client.16785 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 20 client.16785 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d67e030 0x55940d680450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 4 client.16785 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.265+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55940d5b91f0 con 0x55940d67e030 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.266+0000 7f95ed446640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d67e030 0x55940d680450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.266+0000 7f95ed446640 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d67e030 0x55940d680450 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c393d5052ee4626 server_cookie=5ec38e6df66add99 in_seq=0 out_seq=0 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.266+0000 7f95d0ff9640 10 client.16785 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95d0ff9640 1 -- 172.21.15.44:0/610157184 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55940d5b91f0 con 0x55940d67e030 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95d0ff9640 10 client.16785 handle_command_reply: tid=0 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 client.16785 shutdown 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 msgr2=0x55940d66b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66b1e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95d8043000 tx=0x7f95d80b6c00 comp rx=0 tx=0).stop 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 shutdown_connections 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d67e030 0x55940d680450 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d66e950 0x55940d66b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66b1e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95d0ff9640 0 client.16785 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 --2- 172.21.15.44:0/610157184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d663790 0x55940d66df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 >> 172.21.15.44:0/610157184 conn(0x55940d65fd00 msgr2=0x55940d665a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 shutdown_connections 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 1 -- 172.21.15.44:0/610157184 wait complete. 2024-09-13T17:30:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.267+0000 7f95f23ce740 20 client.16785 trim_cache size 0 max 16384 2024-09-13T17:30:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 Processor -- start 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 -- start start 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d66a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d66def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d66c350 con 0x55940d6640b0 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d671070 con 0x55940d663790 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95f23ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d6855f0 con 0x55940d66e950 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52920/0 (socket says 172.21.15.44:52920) 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 -- 172.21.15.44:0/3809853712 learned_addr learned my addr 172.21.15.44:0/3809853712 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ed446640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d66a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95edc47640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d66def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 -- 172.21.15.44:0/3809853712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 msgr2=0x55940d66def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d66def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 -- 172.21.15.44:0/3809853712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 msgr2=0x55940d66a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d66a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 -- 172.21.15.44:0/3809853712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55940d567770 con 0x55940d6640b0 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ed446640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d66a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95edc47640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d66def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.279+0000 7f95ecc45640 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66d9b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f95d80ba850 tx=0x7f95d80b71d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508788e5a23c5c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95d3fff640 1 -- 172.21.15.44:0/3809853712 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8004d20 con 0x55940d6640b0 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95d3fff640 1 -- 172.21.15.44:0/3809853712 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95d80d2050 con 0x55940d6640b0 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95d3fff640 1 -- 172.21.15.44:0/3809853712 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80ba8c0 con 0x55940d6640b0 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 -- 172.21.15.44:0/3809853712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 msgr2=0x55940d66d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66d9b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f95d80ba850 tx=0x7f95d80b71d0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 -- 172.21.15.44:0/3809853712 shutdown_connections 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d66def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d66d9b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 --2- 172.21.15.44:0/3809853712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d66a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 -- 172.21.15.44:0/3809853712 >> 172.21.15.44:0/3809853712 conn(0x55940d65fd00 msgr2=0x55940d669a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 -- 172.21.15.44:0/3809853712 shutdown_connections 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 -- 172.21.15.44:0/3809853712 wait complete. 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.280+0000 7f95f23ce740 1 -- start start 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 Processor -- start 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d6755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d675b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d673ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d6743a0 con 0x55940d6640b0 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d674510 con 0x55940d663790 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940d6746d0 con 0x55940d66e950 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95f23ce740 10 client.0 _ll_register_callbacks cb 0x7f95e400a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d675b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d675b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52924/0 (socket says 172.21.15.44:52924) 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 -- 172.21.15.44:0/2914924557 learned_addr learned my addr 172.21.15.44:0/2914924557 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ed446640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d6755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95edc47640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d673ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 -- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 msgr2=0x55940d673ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d673ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 -- 172.21.15.44:0/2914924557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 msgr2=0x55940d6755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d6755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95ecc45640 1 -- 172.21.15.44:0/2914924557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55940d5c53c0 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.281+0000 7f95edc47640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d673ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95ecc45640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d675b20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f95d80b7260 tx=0x7f95d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fafd402c27b947 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95d0ff9640 1 -- 172.21.15.44:0/2914924557 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80ba690 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95d0ff9640 1 -- 172.21.15.44:0/2914924557 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f95d80bafc0 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95e4018ad0 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95d0ff9640 1 -- 172.21.15.44:0/2914924557 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95d80d3230 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 10 client.16791 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55940d58ce80 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95d0ff9640 1 -- 172.21.15.44:0/2914924557 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95e4018ad0 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95d0ff9640 1 -- 172.21.15.44:0/2914924557 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95d80b66b0 con 0x55940d6640b0 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 10 client.16791 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 20 client.16791 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d6d5320 0x55940d6d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 4 client.16791 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.282+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55940d2ba4e0 con 0x55940d6d5320 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95ed446640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d6d5320 0x55940d6d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95ed446640 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d6d5320 0x55940d6d7710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17482be62953fdaa server_cookie=9121a34d2a4bfbab in_seq=0 out_seq=0 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95d0ff9640 10 client.16791 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95d0ff9640 1 -- 172.21.15.44:0/2914924557 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55940d2ba4e0 con 0x55940d6d5320 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95d0ff9640 10 client.16791 handle_command_reply: tid=0 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 client.16791 shutdown 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 msgr2=0x55940d675b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d675b20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f95d80b7260 tx=0x7f95d80045e0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 shutdown_connections 2024-09-13T17:30:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55940d6d5320 0x55940d6d7710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55940d66e950 0x55940d673ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55940d6640b0 0x55940d675b20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95d0ff9640 0 client.16791 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 --2- 172.21.15.44:0/2914924557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55940d663790 0x55940d6755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 >> 172.21.15.44:0/2914924557 conn(0x55940d65fd00 msgr2=0x55940d65eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 shutdown_connections 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.283+0000 7f95f23ce740 1 -- 172.21.15.44:0/2914924557 wait complete. 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95f23ce740 20 client.16791 trim_cache size 0 max 16384 2024-09-13T17:30:35.293 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95d40a1280 msgr2=0x7f95d40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95d40a1280 0x7f95d40a3740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e0004a40 comp rx=0 tx=0).stop 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 msgr2=0x7f95e81966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 0x7f95e81966b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005ab0 tx=0x7f95d8032c20 comp rx=0 tx=0).stop 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 shutdown_connections 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95d40a1280 0x7f95d40a3740 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e819aaa0 0x7f95e819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e8103fd0 0x7f95e81966b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 --2- 172.21.15.44:0/1830227570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e80fee80 0x7f95e8196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:35.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 >> 172.21.15.44:0/1830227570 conn(0x7f95e80fa8d0 msgr2=0x7f95e80fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:35.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 shutdown_connections 2024-09-13T17:30:35.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:35.284+0000 7f95ef6aa640 1 -- 172.21.15.44:0/1830227570 wait complete. 2024-09-13T17:30:35.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:35 smithi044 ceph-mon[31175]: pgmap v119: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-09-13T17:30:35.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:35 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3232937021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:30:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:35 smithi070 ceph-mon[34502]: pgmap v119: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-09-13T17:30:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:35 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3232937021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:30:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:35 smithi100 ceph-mon[35055]: pgmap v119: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-09-13T17:30:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:35 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3232937021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:30:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 -- 172.21.15.44:0/1983581560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 msgr2=0x7fa5900681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 --2- 172.21.15.44:0/1983581560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5900681d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009900 tx=0x7fa58c02a410 comp rx=0 tx=0).stop 2024-09-13T17:30:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 -- 172.21.15.44:0/1983581560 shutdown_connections 2024-09-13T17:30:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 --2- 172.21.15.44:0/1983581560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59010c660 0x7fa59010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 --2- 172.21.15.44:0/1983581560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5900681d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 --2- 172.21.15.44:0/1983581560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa590100ca0 0x7fa590067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 -- 172.21.15.44:0/1983581560 >> 172.21.15.44:0/1983581560 conn(0x7fa590074940 msgr2=0x7fa590074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 -- 172.21.15.44:0/1983581560 shutdown_connections 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 -- 172.21.15.44:0/1983581560 wait complete. 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 Processor -- start 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.267+0000 7fa59822b640 1 -- start start 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa59822b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5901984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa59822b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa590100ca0 0x7fa5901989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa59822b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59010c660 0x7fa59019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa59822b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019d3d0 con 0x7fa590067d70 2024-09-13T17:30:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa59822b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019d540 con 0x7fa59010c660 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa59822b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019d700 con 0x7fa590100ca0 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5901984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5901984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52946/0 (socket says 172.21.15.44:52946) 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 -- 172.21.15.44:0/2550938728 learned_addr learned my addr 172.21.15.44:0/2550938728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa5967c8640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59010c660 0x7fa59019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa5957c6640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa590100ca0 0x7fa5901989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 -- 172.21.15.44:0/2550938728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa590100ca0 msgr2=0x7fa5901989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa590100ca0 0x7fa5901989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 -- 172.21.15.44:0/2550938728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59010c660 msgr2=0x7fa59019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59010c660 0x7fa59019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 -- 172.21.15.44:0/2550938728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c009560 con 0x7fa590067d70 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa5957c6640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa590100ca0 0x7fa5901989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.268+0000 7fa595fc7640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5901984a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa58400e9f0 tx=0x7fa58400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.269+0000 7fa582ffd640 1 -- 172.21.15.44:0/2550938728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584009700 con 0x7fa590067d70 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.269+0000 7fa582ffd640 1 -- 172.21.15.44:0/2550938728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa58400cd80 con 0x7fa590067d70 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.269+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59019d9e0 con 0x7fa590067d70 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.269+0000 7fa582ffd640 1 -- 172.21.15.44:0/2550938728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840187e0 con 0x7fa590067d70 2024-09-13T17:30:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.269+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901a5260 con 0x7fa590067d70 2024-09-13T17:30:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.271+0000 7fa582ffd640 1 -- 172.21.15.44:0/2550938728 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa584010430 con 0x7fa590067d70 2024-09-13T17:30:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.271+0000 7fa59b02c740 1 Processor -- start 2024-09-13T17:30:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 -- start start 2024-09-13T17:30:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15b330 0x55b74f15b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15bc50 0x55b74f165fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f0c6c30 con 0x55b74f1664f0 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f0c43c0 con 0x55b74f15bc50 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa59b02c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f0bfcb0 con 0x55b74f15b330 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15bc50 0x55b74f165fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f1688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52958/0 (socket says 172.21.15.44:52958) 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 -- 172.21.15.44:0/1101247112 learned_addr learned my addr 172.21.15.44:0/1101247112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5967c8640 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15b330 0x55b74f15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa582ffd640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa57c0a1280 0x7fa57c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 -- 172.21.15.44:0/1101247112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15b330 msgr2=0x55b74f15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15b330 0x55b74f15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 -- 172.21.15.44:0/1101247112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15bc50 msgr2=0x55b74f165fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15bc50 0x55b74f165fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5957c6640 1 -- 172.21.15.44:0/1101247112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b74f066910 con 0x55b74f1664f0 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.272+0000 7fa5967c8640 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15b330 0x55b74f15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.273+0000 7fa582ffd640 1 -- 172.21.15.44:0/2550938728 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa584014070 con 0x7fa590067d70 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.273+0000 7fa5967c8640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa57c0a1280 0x7fa57c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.273+0000 7fa5957c6640 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f1688e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa58c02a930 tx=0x7fa58c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e016cc007116895 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.273+0000 7fa57bfff640 1 -- 172.21.15.44:0/1101247112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c004030 con 0x55b74f1664f0 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.273+0000 7fa57bfff640 1 -- 172.21.15.44:0/1101247112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa58c031960 con 0x55b74f1664f0 2024-09-13T17:30:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.273+0000 7fa57bfff640 1 -- 172.21.15.44:0/1101247112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003ad0 con 0x55b74f1664f0 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 -- 172.21.15.44:0/1101247112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 msgr2=0x55b74f1688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa5967c8640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa57c0a1280 0x7fa57c0a3740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa590068810 tx=0x7fa588014040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f1688e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa58c02a930 tx=0x7fa58c005e80 comp rx=0 tx=0).stop 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 -- 172.21.15.44:0/1101247112 shutdown_connections 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f1688e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15bc50 0x55b74f165fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1101247112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15b330 0x55b74f15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 -- 172.21.15.44:0/1101247112 >> 172.21.15.44:0/1101247112 conn(0x55b74f1578c0 msgr2=0x55b74f157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 -- 172.21.15.44:0/1101247112 shutdown_connections 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 -- 172.21.15.44:0/1101247112 wait complete. 2024-09-13T17:30:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 -- start start 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.274+0000 7fa59b02c740 1 Processor -- start 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 0x55b74f165ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f1664f0 0x55b74f1637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f07de80 con 0x55b74f15b330 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f07db20 con 0x55b74f1664f0 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f043260 con 0x55b74f15bc50 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa59b02c740 10 client.0 _ll_register_callbacks cb 0x55b74f078df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 0x55b74f165ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 0x55b74f165ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52974/0 (socket says 172.21.15.44:52974) 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 -- 172.21.15.44:0/798362029 learned_addr learned my addr 172.21.15.44:0/798362029 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa5967c8640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f1664f0 0x55b74f1637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa5957c6640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 -- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 msgr2=0x55b74f1631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 -- 172.21.15.44:0/798362029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f1664f0 msgr2=0x55b74f1637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f1664f0 0x55b74f1637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 -- 172.21.15.44:0/798362029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59010f6a0 con 0x55b74f15b330 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa595fc7640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 0x55b74f165ec0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c6a0 tx=0x7fa5840d0f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59c1228dc6c08ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa5797fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.275+0000 7fa5797fa640 1 -- 172.21.15.44:0/798362029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584099ac0 con 0x55b74f15b330 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa5797fa640 1 -- 172.21.15.44:0/798362029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa58409a860 con 0x55b74f15b330 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b74f0e8680 con 0x55b74f15b330 2024-09-13T17:30:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa5797fa640 1 -- 172.21.15.44:0/798362029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa58409a160 con 0x55b74f15b330 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 10 client.16803 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b74f0be330 con 0x55b74f15b330 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa5797fa640 1 -- 172.21.15.44:0/798362029 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b74f0e8680 con 0x55b74f15b330 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa5797fa640 1 -- 172.21.15.44:0/798362029 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa5840106e0 con 0x55b74f15b330 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 10 client.16803 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 20 client.16803 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f175e00 0x55b74f1781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 4 client.16803 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.276+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b74f0b35d0 con 0x55b74f175e00 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.277+0000 7fa5957c6640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f175e00 0x55b74f1781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.277+0000 7fa5957c6640 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f175e00 0x55b74f1781f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525fccd35582cabb server_cookie=bb7281c6c78525fd in_seq=0 out_seq=0 2024-09-13T17:30:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.277+0000 7fa5797fa640 10 client.16803 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa5797fa640 1 -- 172.21.15.44:0/798362029 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b74f0b35d0 con 0x55b74f175e00 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa5797fa640 10 client.16803 handle_command_reply: tid=0 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 client.16803 shutdown 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 msgr2=0x55b74f165ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 0x55b74f165ec0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c6a0 tx=0x7fa5840d0f30 comp rx=0 tx=0).stop 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 shutdown_connections 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f175e00 0x55b74f1781f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f1664f0 0x55b74f1637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 --2- 172.21.15.44:0/798362029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15b330 0x55b74f165ec0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa5797fa640 0 client.16803 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 >> 172.21.15.44:0/798362029 conn(0x55b74f1578c0 msgr2=0x55b74f156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 shutdown_connections 2024-09-13T17:30:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 1 -- 172.21.15.44:0/798362029 wait complete. 2024-09-13T17:30:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.278+0000 7fa59b02c740 20 client.16803 trim_cache size 0 max 16384 2024-09-13T17:30:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 Processor -- start 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 -- start start 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f157300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f16d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f172bc0 con 0x55b74f1664f0 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f178520 con 0x55b74f15b330 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa59b02c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f17cf50 con 0x55b74f15bc50 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f16d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f16d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52982/0 (socket says 172.21.15.44:52982) 2024-09-13T17:30:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 -- 172.21.15.44:0/1753947890 learned_addr learned my addr 172.21.15.44:0/1753947890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa595fc7640 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5957c6640 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 -- 172.21.15.44:0/1753947890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 msgr2=0x55b74f1659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 -- 172.21.15.44:0/1753947890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 msgr2=0x55b74f157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5967c8640 1 -- 172.21.15.44:0/1753947890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b74f060300 con 0x55b74f1664f0 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.291+0000 7fa5957c6640 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa5967c8640 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f16d1b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa5880105f0 tx=0x7fa588017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbc9fed1e5e29e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa580ff9640 1 -- 172.21.15.44:0/1753947890 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588023040 con 0x55b74f1664f0 2024-09-13T17:30:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa580ff9640 1 -- 172.21.15.44:0/1753947890 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa58801c910 con 0x55b74f1664f0 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa580ff9640 1 -- 172.21.15.44:0/1753947890 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58801d8b0 con 0x55b74f1664f0 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 -- 172.21.15.44:0/1753947890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 msgr2=0x55b74f16d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f16d1b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa5880105f0 tx=0x7fa588017000 comp rx=0 tx=0).stop 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 -- 172.21.15.44:0/1753947890 shutdown_connections 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f15bc50 0x55b74f1659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f1664f0 0x55b74f16d1b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 --2- 172.21.15.44:0/1753947890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 -- 172.21.15.44:0/1753947890 >> 172.21.15.44:0/1753947890 conn(0x55b74f1578c0 msgr2=0x55b74f161a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 -- 172.21.15.44:0/1753947890 shutdown_connections 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.292+0000 7fa59b02c740 1 -- 172.21.15.44:0/1753947890 wait complete. 2024-09-13T17:30:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 -- start start 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 Processor -- start 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f16a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 0x55b74f16a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f1664f0 0x55b74f1cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f1cc7b0 con 0x55b74f15bc50 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f1cc8f0 con 0x55b74f15b330 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b74f1ccab0 con 0x55b74f1664f0 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa59b02c740 10 client.0 _ll_register_callbacks cb 0x7fa584005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa5957c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 0x55b74f16a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f16a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa595fc7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f16a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:48532/0 (socket says 172.21.15.44:48532) 2024-09-13T17:30:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa5957c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 0x55b74f16a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52990/0 (socket says 172.21.15.44:52990) 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.293+0000 7fa595fc7640 1 -- 172.21.15.44:0/2106027507 learned_addr learned my addr 172.21.15.44:0/2106027507 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa5967c8640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f1664f0 0x55b74f1cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa595fc7640 1 -- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f1664f0 msgr2=0x55b74f1cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa595fc7640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f1664f0 0x55b74f1cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa595fc7640 1 -- 172.21.15.44:0/2106027507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 msgr2=0x55b74f16a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa595fc7640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 0x55b74f16a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa595fc7640 1 -- 172.21.15.44:0/2106027507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa588004800 con 0x55b74f15b330 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa5957c6640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 0x55b74f16a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa595fc7640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f16a460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa584099e60 tx=0x7fa5840ca870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29d8c72406cfb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa5797fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa5797fa640 1 -- 172.21.15.44:0/2106027507 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584099ac0 con 0x55b74f15b330 2024-09-13T17:30:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa5797fa640 1 -- 172.21.15.44:0/2106027507 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa58409a860 con 0x55b74f15b330 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa58c031960 con 0x55b74f15b330 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa5797fa640 1 -- 172.21.15.44:0/2106027507 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa584099f30 con 0x55b74f15b330 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa59b02c740 10 client.26240 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.294+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b74f0858e0 con 0x55b74f15b330 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa5797fa640 1 -- 172.21.15.44:0/2106027507 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa5840a6020 con 0x55b74f15b330 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa5797fa640 1 -- 172.21.15.44:0/2106027507 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa5840d0890 con 0x55b74f15b330 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa59b02c740 10 client.26240 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa59b02c740 10 client.26240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa59b02c740 20 client.26240 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa59b02c740 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f1cde40 0x55b74f1d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa59b02c740 4 client.26240 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:30:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b74edb3300 con 0x55b74f1cde40 2024-09-13T17:30:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa5957c6640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f1cde40 0x55b74f1d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa5957c6640 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f1cde40 0x55b74f1d0260 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85033dd31bcd550 server_cookie=c904453b4aac92bc in_seq=0 out_seq=0 2024-09-13T17:30:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.295+0000 7fa5797fa640 10 client.26240 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.304 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:30:36.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:30:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa5797fa640 1 -- 172.21.15.44:0/2106027507 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b74edb3300 con 0x55b74f1cde40 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa5797fa640 10 client.26240 handle_command_reply: tid=0 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 client.26240 shutdown 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 msgr2=0x55b74f16a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f16a460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa584099e60 tx=0x7fa5840ca870 comp rx=0 tx=0).stop 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 shutdown_connections 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b74f1cde40 0x55b74f1d0260 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b74f1664f0 0x55b74f1cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74f15bc50 0x55b74f16a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 --2- 172.21.15.44:0/2106027507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74f15b330 0x55b74f16a460 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa5797fa640 0 client.26240 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 >> 172.21.15.44:0/2106027507 conn(0x55b74f1578c0 msgr2=0x55b74f15cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 shutdown_connections 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 1 -- 172.21.15.44:0/2106027507 wait complete. 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.312+0000 7fa59b02c740 20 client.26240 trim_cache size 0 max 16384 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:30:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:30:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa57c0a1280 msgr2=0x7fa57c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa57c0a1280 0x7fa57c0a3740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa590068810 tx=0x7fa588014040 comp rx=0 tx=0).stop 2024-09-13T17:30:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 msgr2=0x7fa5901984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5901984a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa58400e9f0 tx=0x7fa58400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 shutdown_connections 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa57c0a1280 0x7fa57c0a3740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa590100ca0 0x7fa5901989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59010c660 0x7fa59019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 --2- 172.21.15.44:0/2550938728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590067d70 0x7fa5901984a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 >> 172.21.15.44:0/2550938728 conn(0x7fa590074940 msgr2=0x7fa59010d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 shutdown_connections 2024-09-13T17:30:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:36.313+0000 7fa59822b640 1 -- 172.21.15.44:0/2550938728 wait complete. 2024-09-13T17:30:36.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 -- 172.21.15.44:0/3657631285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440101800 msgr2=0x7f6440109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 --2- 172.21.15.44:0/3657631285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440101800 0x7f6440109ce0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6434009900 tx=0x7f643402a410 comp rx=0 tx=0).stop 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 -- 172.21.15.44:0/3657631285 shutdown_connections 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 --2- 172.21.15.44:0/3657631285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f644010a350 0x7f644010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 --2- 172.21.15.44:0/3657631285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440101800 0x7f6440109ce0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 --2- 172.21.15.44:0/3657631285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440100e50 0x7f6440101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 -- 172.21.15.44:0/3657631285 >> 172.21.15.44:0/3657631285 conn(0x7f64400fa930 msgr2=0x7f64400fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 -- 172.21.15.44:0/3657631285 shutdown_connections 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.412+0000 7f6447918640 1 -- 172.21.15.44:0/3657631285 wait complete. 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 Processor -- start 2024-09-13T17:30:36.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 -- start start 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 0x7f64401961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440101800 0x7f6440196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f644010a350 0x7f644019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644019b100 con 0x7f6440100e50 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644019b270 con 0x7f6440101800 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6447918640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644019b430 con 0x7f644010a350 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 0x7f64401961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 0x7f64401961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53002/0 (socket says 172.21.15.44:53002) 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 -- 172.21.15.44:0/690057299 learned_addr learned my addr 172.21.15.44:0/690057299 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6444eb3640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440101800 0x7f6440196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6445eb5640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f644010a350 0x7f644019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 -- 172.21.15.44:0/690057299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f644010a350 msgr2=0x7f644019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f644010a350 0x7f644019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 -- 172.21.15.44:0/690057299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440101800 msgr2=0x7f6440196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440101800 0x7f6440196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f64456b4640 1 -- 172.21.15.44:0/690057299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009560 con 0x7f6440100e50 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.413+0000 7f6444eb3640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440101800 0x7f6440196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.414+0000 7f64456b4640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 0x7f64401961d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f643800e9f0 tx=0x7f643800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.414+0000 7f643e7fc640 1 -- 172.21.15.44:0/690057299 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438009700 con 0x7f6440100e50 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.414+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644019b710 con 0x7f6440100e50 2024-09-13T17:30:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.414+0000 7f643e7fc640 1 -- 172.21.15.44:0/690057299 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f643800cd80 con 0x7f6440100e50 2024-09-13T17:30:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.414+0000 7f643e7fc640 1 -- 172.21.15.44:0/690057299 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64380187e0 con 0x7f6440100e50 2024-09-13T17:30:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.414+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64401a2f90 con 0x7f6440100e50 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.416+0000 7f643e7fc640 1 -- 172.21.15.44:0/690057299 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6438010430 con 0x7f6440100e50 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.416+0000 7f644a63b740 1 Processor -- start 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.416+0000 7f644a63b740 1 -- start start 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f644a63b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b3627bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f644a63b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 0x560b36286850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f644a63b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b36286d90 0x560b36289180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f644a63b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b361e2c60 con 0x560b3627c4f0 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f644a63b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b361e03f0 con 0x560b3627bbd0 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f644a63b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b361dbce0 con 0x560b36286d90 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 0x560b36286850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f6445eb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b3627bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f6444eb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b36286d90 0x560b36289180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 0x560b36286850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53016/0 (socket says 172.21.15.44:53016) 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 -- 172.21.15.44:0/4114597547 learned_addr learned my addr 172.21.15.44:0/4114597547 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f643e7fc640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f64280a1200 0x7f64280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f6445eb5640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f64280a1200 0x7f64280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f643e7fc640 1 -- 172.21.15.44:0/690057299 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6438014070 con 0x7f6440100e50 2024-09-13T17:30:36.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 -- 172.21.15.44:0/4114597547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b36286d90 msgr2=0x560b36289180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b36286d90 0x560b36289180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 -- 172.21.15.44:0/4114597547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 msgr2=0x560b3627bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b3627bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 -- 172.21.15.44:0/4114597547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b36197970 con 0x560b3627c4f0 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f6444eb3640 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b36286d90 0x560b36289180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f6445eb5640 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b3627bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.417+0000 7f64456b4640 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 0x560b36286850 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6438096750 tx=0x7f6438099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1980a099b15d9654 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f64277fe640 1 -- 172.21.15.44:0/4114597547 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643809a250 con 0x560b3627c4f0 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f6445eb5640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f64280a1200 0x7f64280a36c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f64300094d0 tx=0x7f6430004ab0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f64277fe640 1 -- 172.21.15.44:0/4114597547 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f643809e080 con 0x560b3627c4f0 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f64277fe640 1 -- 172.21.15.44:0/4114597547 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438010b70 con 0x560b3627c4f0 2024-09-13T17:30:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 -- 172.21.15.44:0/4114597547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 msgr2=0x560b36286850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 0x560b36286850 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6438096750 tx=0x7f6438099d90 comp rx=0 tx=0).stop 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 -- 172.21.15.44:0/4114597547 shutdown_connections 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b36286d90 0x560b36289180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627c4f0 0x560b36286850 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 --2- 172.21.15.44:0/4114597547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b3627bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.418+0000 7f644a63b740 1 -- 172.21.15.44:0/4114597547 >> 172.21.15.44:0/4114597547 conn(0x560b36278160 msgr2=0x560b36278540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 -- 172.21.15.44:0/4114597547 shutdown_connections 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 -- 172.21.15.44:0/4114597547 wait complete. 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 -- start start 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 Processor -- start 2024-09-13T17:30:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b36286790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b36283a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b36284070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36199f80 con 0x560b3627bbd0 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36199c20 con 0x560b36286d90 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36166ab0 con 0x560b3627c4f0 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f644a63b740 10 client.0 _ll_register_callbacks cb 0x560b361d5350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b36286790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b36286790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53030/0 (socket says 172.21.15.44:53030) 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f64456b4640 1 -- 172.21.15.44:0/4015254419 learned_addr learned my addr 172.21.15.44:0/4015254419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.419+0000 7f6445eb5640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b36284070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6444eb3640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b36283a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f64456b4640 1 -- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 msgr2=0x560b36283a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f64456b4640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b36283a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f64456b4640 1 -- 172.21.15.44:0/4015254419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 msgr2=0x560b36284070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f64456b4640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b36284070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f64456b4640 1 -- 172.21.15.44:0/4015254419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64401a2f90 con 0x560b3627bbd0 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6444eb3640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b36283a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f64456b4640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b36286790 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f64380ca940 tx=0x7f64380d0930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c955ca931c33b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6424ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6424ff9640 1 -- 172.21.15.44:0/4015254419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64380a26d0 con 0x560b3627bbd0 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6424ff9640 1 -- 172.21.15.44:0/4015254419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f64380106e0 con 0x560b3627bbd0 2024-09-13T17:30:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b35ecf590 con 0x560b3627bbd0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6424ff9640 1 -- 172.21.15.44:0/4015254419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64380108e0 con 0x560b3627bbd0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f644a63b740 10 client.16821 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b361da360 con 0x560b3627bbd0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.420+0000 7f6424ff9640 1 -- 172.21.15.44:0/4015254419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64380b6080 con 0x560b3627bbd0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f6424ff9640 1 -- 172.21.15.44:0/4015254419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f64380997d0 con 0x560b3627bbd0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f644a63b740 10 client.16821 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f644a63b740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f644a63b740 20 client.16821 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f644a63b740 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362967d0 0x560b36298bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f644a63b740 4 client.16821 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b35ee0860 con 0x560b362967d0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f6444eb3640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362967d0 0x560b36298bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f6424ff9640 10 client.16821 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.421+0000 7f6444eb3640 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362967d0 0x560b36298bc0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e1a494f87442a3 server_cookie=dbc95ab791a3d320 in_seq=0 out_seq=0 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f6424ff9640 1 -- 172.21.15.44:0/4015254419 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560b35ee0860 con 0x560b362967d0 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f6424ff9640 10 client.16821 handle_command_reply: tid=0 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 client.16821 shutdown 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 msgr2=0x560b36286790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b36286790 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f64380ca940 tx=0x7f64380d0930 comp rx=0 tx=0).stop 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 shutdown_connections 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362967d0 0x560b36298bc0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b36283a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b36284070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f6424ff9640 0 client.16821 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 --2- 172.21.15.44:0/4015254419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b36286790 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 >> 172.21.15.44:0/4015254419 conn(0x560b36278160 msgr2=0x560b3627d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 shutdown_connections 2024-09-13T17:30:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.422+0000 7f644a63b740 1 -- 172.21.15.44:0/4015254419 wait complete. 2024-09-13T17:30:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.423+0000 7f644a63b740 20 client.16821 trim_cache size 0 max 16384 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 Processor -- start 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 -- start start 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b362862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 0x560b3628df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3629dda0 con 0x560b36286d90 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3626f6e0 con 0x560b3627bbd0 2024-09-13T17:30:36.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f644a63b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b362843a0 con 0x560b3627c4f0 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f6445eb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 0x560b3628df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f6445eb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 0x560b3628df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53038/0 (socket says 172.21.15.44:53038) 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f6445eb5640 1 -- 172.21.15.44:0/3595507590 learned_addr learned my addr 172.21.15.44:0/3595507590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.437+0000 7f64456b4640 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b362862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6444eb3640 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6445eb5640 1 -- 172.21.15.44:0/3595507590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 msgr2=0x560b3628da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6445eb5640 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6445eb5640 1 -- 172.21.15.44:0/3595507590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 msgr2=0x560b362862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6445eb5640 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b362862f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6445eb5640 1 -- 172.21.15.44:0/3595507590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b3617c1c0 con 0x560b36286d90 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6444eb3640 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:36.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6445eb5640 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 0x560b3628df40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6430012540 tx=0x7f6430009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1268558bb5ef715b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6427fff640 1 -- 172.21.15.44:0/3595507590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300109a0 con 0x560b36286d90 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6427fff640 1 -- 172.21.15.44:0/3595507590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f643002a470 con 0x560b36286d90 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f6427fff640 1 -- 172.21.15.44:0/3595507590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300216b0 con 0x560b36286d90 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f644a63b740 1 -- 172.21.15.44:0/3595507590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 msgr2=0x560b3628df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.438+0000 7f644a63b740 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 0x560b3628df40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6430012540 tx=0x7f6430009a70 comp rx=0 tx=0).stop 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- 172.21.15.44:0/3595507590 shutdown_connections 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b36286d90 0x560b3628df40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 --2- 172.21.15.44:0/3595507590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b3627bbd0 0x560b362862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- 172.21.15.44:0/3595507590 >> 172.21.15.44:0/3595507590 conn(0x560b36278160 msgr2=0x560b36278540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- 172.21.15.44:0/3595507590 shutdown_connections 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- 172.21.15.44:0/3595507590 wait complete. 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- start start 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 Processor -- start 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b3628cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b362ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b362ed1c0 con 0x560b3627bbd0 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b362ed300 con 0x560b36286d90 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b362ed440 con 0x560b3627c4f0 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f644a63b740 10 client.0 _ll_register_callbacks cb 0x7f64300099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:36.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b3628cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f64456b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b3628cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53044/0 (socket says 172.21.15.44:53044) 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.439+0000 7f64456b4640 1 -- 172.21.15.44:0/2773399988 learned_addr learned my addr 172.21.15.44:0/2773399988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6445eb5640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b362ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6444eb3640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f64456b4640 1 -- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 msgr2=0x560b3628d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f64456b4640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f64456b4640 1 -- 172.21.15.44:0/2773399988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 msgr2=0x560b362ecc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f64456b4640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b362ecc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f64456b4640 1 -- 172.21.15.44:0/2773399988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64300121d0 con 0x560b3627bbd0 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f64456b4640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b3628cc20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6438096cc0 tx=0x7f64380966c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e673b8c055016605 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6424ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6424ff9640 1 -- 172.21.15.44:0/2773399988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438099bb0 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6424ff9640 1 -- 172.21.15.44:0/2773399988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f64380a26d0 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f643809e080 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6424ff9640 1 -- 172.21.15.44:0/2773399988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64380cd330 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f644a63b740 10 client.16827 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b361a1990 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.440+0000 7f6424ff9640 1 -- 172.21.15.44:0/2773399988 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f643809e080 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f6424ff9640 1 -- 172.21.15.44:0/2773399988 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f64380b2070 con 0x560b3627bbd0 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f644a63b740 10 client.16827 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f644a63b740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f644a63b740 20 client.16827 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f644a63b740 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362ee660 0x560b362f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f644a63b740 4 client.16827 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560b35f33eb0 con 0x560b362ee660 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f6444eb3640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362ee660 0x560b362f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f6424ff9640 10 client.16827 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.441+0000 7f6444eb3640 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362ee660 0x560b362f0a50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e9290acf69dac server_cookie=79d24ceb542883c7 in_seq=0 out_seq=0 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f6424ff9640 1 -- 172.21.15.44:0/2773399988 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560b35f33eb0 con 0x560b362ee660 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f6424ff9640 10 client.16827 handle_command_reply: tid=0 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 client.16827 shutdown 2024-09-13T17:30:36.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 msgr2=0x560b3628cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b3628cc20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6438096cc0 tx=0x7f64380966c0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 shutdown_connections 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b362ee660 0x560b362f0a50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3627c4f0 0x560b3628d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f6424ff9640 0 client.16827 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b36286d90 0x560b362ecc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 --2- 172.21.15.44:0/2773399988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b3627bbd0 0x560b3628cc20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.442+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 >> 172.21.15.44:0/2773399988 conn(0x560b36278160 msgr2=0x560b3627ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 shutdown_connections 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f644a63b740 1 -- 172.21.15.44:0/2773399988 wait complete. 2024-09-13T17:30:36.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f644a63b740 20 client.16827 trim_cache size 0 max 16384 2024-09-13T17:30:36.453 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:30:36.453 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:30:36.453 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "c1b07303-7ab7-4d46-a269-437577f19a51", 2024-09-13T17:30:36.453 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:30:36.453 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f64280a1200 msgr2=0x7f64280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f64280a1200 0x7f64280a36c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f64300094d0 tx=0x7f6430004ab0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 msgr2=0x7f64401961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 0x7f64401961d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f643800e9f0 tx=0x7f643800eeb0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 shutdown_connections 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f64280a1200 0x7f64280a36c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f644010a350 0x7f644019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6440101800 0x7f6440196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 --2- 172.21.15.44:0/690057299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6440100e50 0x7f64401961d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 >> 172.21.15.44:0/690057299 conn(0x7f64400fa930 msgr2=0x7f64400fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 shutdown_connections 2024-09-13T17:30:36.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.443+0000 7f6447918640 1 -- 172.21.15.44:0/690057299 wait complete. 2024-09-13T17:30:36.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:30:36.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 -- 172.21.15.44:0/3419952399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 msgr2=0x7fa3740ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 --2- 172.21.15.44:0/3419952399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa3740ff240 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa364009930 tx=0x7fa36402a420 comp rx=0 tx=0).stop 2024-09-13T17:30:36.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 -- 172.21.15.44:0/3419952399 shutdown_connections 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 --2- 172.21.15.44:0/3419952399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa374103fb0 0x7fa37410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 --2- 172.21.15.44:0/3419952399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740ff810 0x7fa374103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 --2- 172.21.15.44:0/3419952399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa3740ff240 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 -- 172.21.15.44:0/3419952399 >> 172.21.15.44:0/3419952399 conn(0x7fa374075c60 msgr2=0x7fa3740fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 -- 172.21.15.44:0/3419952399 shutdown_connections 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 -- 172.21.15.44:0/3419952399 wait complete. 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 Processor -- start 2024-09-13T17:30:36.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.600+0000 7fa379ca3640 1 -- start start 2024-09-13T17:30:36.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa379ca3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa374195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa379ca3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa374103fb0 0x7fa374195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa379ca3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374199a70 0x7fa374199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa379ca3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419a460 con 0x7fa3740fee60 2024-09-13T17:30:36.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa379ca3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419a5d0 con 0x7fa374199a70 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa379ca3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419a740 con 0x7fa374103fb0 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa374195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa374195140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53062/0 (socket says 172.21.15.44:53062) 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 -- 172.21.15.44:0/2798480068 learned_addr learned my addr 172.21.15.44:0/2798480068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa373fff640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374199a70 0x7fa374199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa36affd640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa374103fb0 0x7fa374195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 -- 172.21.15.44:0/2798480068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa374103fb0 msgr2=0x7fa374195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa374103fb0 0x7fa374195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 -- 172.21.15.44:0/2798480068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374199a70 msgr2=0x7fa374199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374199a70 0x7fa374199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 -- 172.21.15.44:0/2798480068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c009490 con 0x7fa3740fee60 2024-09-13T17:30:36.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.601+0000 7fa3737fe640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa374195140 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa36402a940 tx=0x7fa364005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.602+0000 7fa3717fa640 1 -- 172.21.15.44:0/2798480068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364004c50 con 0x7fa3740fee60 2024-09-13T17:30:36.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.602+0000 7fa3717fa640 1 -- 172.21.15.44:0/2798480068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa364004db0 con 0x7fa3740fee60 2024-09-13T17:30:36.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.602+0000 7fa3717fa640 1 -- 172.21.15.44:0/2798480068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36403c9c0 con 0x7fa3740fee60 2024-09-13T17:30:36.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.602+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa364009560 con 0x7fa3740fee60 2024-09-13T17:30:36.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.602+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3741a1fb0 con 0x7fa3740fee60 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.604+0000 7fa3717fa640 1 -- 172.21.15.44:0/2798480068 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa364003ca0 con 0x7fa3740fee60 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.604+0000 7fa37c9cc740 1 Processor -- start 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.604+0000 7fa37c9cc740 1 -- start start 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb912f20 con 0x55c6fb9b51e0 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb9106b0 con 0x55c6fb9aa940 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb90bfa0 con 0x55c6fb9aa020 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa3717fa640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3600a1280 0x7fa3600a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53072/0 (socket says 172.21.15.44:53072) 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa373fff640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3600a1280 0x7fa3600a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 -- 172.21.15.44:0/3944400489 learned_addr learned my addr 172.21.15.44:0/3944400489 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa3737fe640 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa373fff640 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa3717fa640 1 -- 172.21.15.44:0/2798480068 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa3640ee1d0 con 0x7fa3740fee60 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 -- 172.21.15.44:0/3944400489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 msgr2=0x55c6fb9aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 -- 172.21.15.44:0/3944400489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa940 msgr2=0x55c6fb9b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 -- 172.21.15.44:0/3944400489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6fb8c6840 con 0x55c6fb9b51e0 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa3737fe640 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:36.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa373fff640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3600a1280 0x7fa3600a3740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa36c005fd0 tx=0x7fa36c009d30 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.605+0000 7fa36affd640 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b75d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009df0 tx=0x7fa35c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92cc816d15814b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa369ffb640 1 -- 172.21.15.44:0/3944400489 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004230 con 0x55c6fb9b51e0 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa369ffb640 1 -- 172.21.15.44:0/3944400489 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa35c004390 con 0x55c6fb9b51e0 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa369ffb640 1 -- 172.21.15.44:0/3944400489 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c019740 con 0x55c6fb9b51e0 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3944400489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 msgr2=0x55c6fb9b75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b75d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009df0 tx=0x7fa35c011810 comp rx=0 tx=0).stop 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3944400489 shutdown_connections 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b75d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3944400489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3944400489 >> 172.21.15.44:0/3944400489 conn(0x55c6fb9a6590 msgr2=0x55c6fb9a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3944400489 shutdown_connections 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.606+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3944400489 wait complete. 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 -- start start 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 Processor -- start 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb8ad2a0 con 0x55c6fb9aa940 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb5e4430 con 0x55c6fb9b51e0 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb88f680 con 0x55c6fb9aa020 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa37c9cc740 10 client.0 _ll_register_callbacks cb 0x55c6fb905610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa36affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa36affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53084/0 (socket says 172.21.15.44:53084) 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa36affd640 1 -- 172.21.15.44:0/1094360835 learned_addr learned my addr 172.21.15.44:0/1094360835 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.607+0000 7fa373fff640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa3737fe640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa36affd640 1 -- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 msgr2=0x55c6fb9b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa36affd640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa36affd640 1 -- 172.21.15.44:0/1094360835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9b51e0 msgr2=0x55c6fb9b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa36affd640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa36affd640 1 -- 172.21.15.44:0/1094360835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c009a20 con 0x55c6fb9aa940 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa36affd640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b1d60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa35c000c00 tx=0x7fa35c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890cc0b7a44ff3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa352ffd640 1 -- 172.21.15.44:0/1094360835 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c003900 con 0x55c6fb9aa940 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa352ffd640 1 -- 172.21.15.44:0/1094360835 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa35c003a60 con 0x55c6fb9aa940 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6fb5ff4e0 con 0x55c6fb9aa940 2024-09-13T17:30:36.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa352ffd640 1 -- 172.21.15.44:0/1094360835 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35c003c20 con 0x55c6fb9aa940 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa37c9cc740 10 client.16839 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c6fb90a620 con 0x55c6fb9aa940 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.608+0000 7fa352ffd640 1 -- 172.21.15.44:0/1094360835 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c6fb5ff4e0 con 0x55c6fb9aa940 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa352ffd640 1 -- 172.21.15.44:0/1094360835 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa35c004b80 con 0x55c6fb9aa940 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa37c9cc740 10 client.16839 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa37c9cc740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa37c9cc740 20 client.16839 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fb9c4ef0 0x55c6fb9c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa37c9cc740 4 client.16839 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6fb6107e0 con 0x55c6fb9c4ef0 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa3737fe640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fb9c4ef0 0x55c6fb9c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa3737fe640 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fb9c4ef0 0x55c6fb9c7310 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2d9b6c85b93b82 server_cookie=6921a6991bfc274f in_seq=0 out_seq=0 2024-09-13T17:30:36.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.609+0000 7fa352ffd640 10 client.16839 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa352ffd640 1 -- 172.21.15.44:0/1094360835 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c6fb6107e0 con 0x55c6fb9c4ef0 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa352ffd640 10 client.16839 handle_command_reply: tid=0 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 client.16839 shutdown 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 msgr2=0x55c6fb9b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b1d60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa35c000c00 tx=0x7fa35c003dc0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 shutdown_connections 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fb9c4ef0 0x55c6fb9c7310 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9aa940 0x55c6fb9b1d60 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa352ffd640 0 client.16839 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/1094360835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa020 0x55c6fb9b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 >> 172.21.15.44:0/1094360835 conn(0x55c6fb9a6590 msgr2=0x55c6fb9a3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 shutdown_connections 2024-09-13T17:30:36.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 1 -- 172.21.15.44:0/1094360835 wait complete. 2024-09-13T17:30:36.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.610+0000 7fa37c9cc740 20 client.16839 trim_cache size 0 max 16384 2024-09-13T17:30:36.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.624+0000 7fa37c9cc740 1 Processor -- start 2024-09-13T17:30:36.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.624+0000 7fa37c9cc740 1 -- start start 2024-09-13T17:30:36.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.624+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.624+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.624+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb99f7e0 con 0x55c6fb9b51e0 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb9a9dd0 con 0x55c6fb9aa020 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb9b0290 con 0x55c6fb9aa940 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa3737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53088/0 (socket says 172.21.15.44:53088) 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 -- 172.21.15.44:0/3756649164 learned_addr learned my addr 172.21.15.44:0/3756649164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa36affd640 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 -- 172.21.15.44:0/3756649164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 msgr2=0x55c6fb9ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 -- 172.21.15.44:0/3756649164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 msgr2=0x55c6fb9ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 -- 172.21.15.44:0/3756649164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6fb8ac790 con 0x55c6fb9b51e0 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa36affd640 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa3737fe640 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa373fff640 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b0610 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa36c012710 tx=0x7fa36c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84af58fc153b423 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa36a7fc640 1 -- 172.21.15.44:0/3756649164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c004040 con 0x55c6fb9b51e0 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa36a7fc640 1 -- 172.21.15.44:0/3756649164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa36c004630 con 0x55c6fb9b51e0 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.625+0000 7fa36a7fc640 1 -- 172.21.15.44:0/3756649164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c02aaf0 con 0x55c6fb9b51e0 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3756649164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 msgr2=0x55c6fb9b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b0610 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa36c012710 tx=0x7fa36c01f000 comp rx=0 tx=0).stop 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3756649164 shutdown_connections 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9b0610 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/3756649164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3756649164 >> 172.21.15.44:0/3756649164 conn(0x55c6fb9a6590 msgr2=0x55c6fb9a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3756649164 shutdown_connections 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 -- 172.21.15.44:0/3756649164 wait complete. 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 -- start start 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.626+0000 7fa37c9cc740 1 Processor -- start 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9bc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb9bb2d0 con 0x55c6fb9b51e0 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb9bb440 con 0x55c6fb9aa020 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fba1b400 con 0x55c6fb9aa940 2024-09-13T17:30:36.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa37c9cc740 10 client.0 _ll_register_callbacks cb 0x7fa364004f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9babd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53092/0 (socket says 172.21.15.44:53092) 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa3737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9bc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 -- 172.21.15.44:0/4113506924 learned_addr learned my addr 172.21.15.44:0/4113506924 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa36affd640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 -- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 msgr2=0x55c6fb9ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 -- 172.21.15.44:0/4113506924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 msgr2=0x55c6fb9bc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9bc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 -- 172.21.15.44:0/4113506924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa364009560 con 0x55c6fb9b51e0 2024-09-13T17:30:36.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa3737fe640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9bc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa373fff640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9babd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0049d0 tx=0x7fa36c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dd4fe533dca2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.627+0000 7fa352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa352ffd640 1 -- 172.21.15.44:0/4113506924 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0041c0 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa352ffd640 1 -- 172.21.15.44:0/4113506924 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa36c004320 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa35c004390 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa352ffd640 1 -- 172.21.15.44:0/4113506924 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa36c02a690 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 10 client.16845 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fa36c01a040 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa352ffd640 1 -- 172.21.15.44:0/4113506924 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa35c004390 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa352ffd640 1 -- 172.21.15.44:0/4113506924 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa36c0336c0 con 0x55c6fb9b51e0 2024-09-13T17:30:36.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 10 client.16845 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 20 client.16845 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fba1c9b0 0x55c6fba1edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 4 client.16845 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6fb663e30 con 0x55c6fba1c9b0 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.628+0000 7fa3737fe640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fba1c9b0 0x55c6fba1edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa352ffd640 10 client.16845 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa3737fe640 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fba1c9b0 0x55c6fba1edd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3698ab35ebbbff server_cookie=b20f4cd2ccbef1d4 in_seq=0 out_seq=0 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa352ffd640 1 -- 172.21.15.44:0/4113506924 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c6fb663e30 con 0x55c6fba1c9b0 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa352ffd640 10 client.16845 handle_command_reply: tid=0 2024-09-13T17:30:36.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 client.16845 shutdown 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 msgr2=0x55c6fb9babd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9babd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0049d0 tx=0x7fa36c010f90 comp rx=0 tx=0).stop 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 shutdown_connections 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c6fba1c9b0 0x55c6fba1edd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6fb9aa940 0x55c6fb9ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa352ffd640 0 client.16845 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6fb9b51e0 0x55c6fb9babd0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 --2- 172.21.15.44:0/4113506924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6fb9aa020 0x55c6fb9bc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 >> 172.21.15.44:0/4113506924 conn(0x55c6fb9a6590 msgr2=0x55c6fb9ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 shutdown_connections 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 1 -- 172.21.15.44:0/4113506924 wait complete. 2024-09-13T17:30:36.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.629+0000 7fa37c9cc740 20 client.16845 trim_cache size 0 max 16384 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: "c1b07303-7ab7-4d46-a269-437577f19a51": { 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "c1b07303-7ab7-4d46-a269-437577f19a51", 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:30:36.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:30:36.641 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:30:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3600a1280 msgr2=0x7fa3600a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3600a1280 0x7fa3600a3740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa36c005fd0 tx=0x7fa36c009d30 comp rx=0 tx=0).stop 2024-09-13T17:30:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 msgr2=0x7fa374195140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa374195140 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa36402a940 tx=0x7fa364005610 comp rx=0 tx=0).stop 2024-09-13T17:30:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 shutdown_connections 2024-09-13T17:30:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa3600a1280 0x7fa3600a3740 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa374103fb0 0x7fa374195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374199a70 0x7fa374199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 --2- 172.21.15.44:0/2798480068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3740fee60 0x7fa374195140 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:36.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 >> 172.21.15.44:0/2798480068 conn(0x7fa374075c60 msgr2=0x7fa374102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:36.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 shutdown_connections 2024-09-13T17:30:36.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:30:36.630+0000 7fa379ca3640 1 -- 172.21.15.44:0/2798480068 wait complete. 2024-09-13T17:30:36.643 INFO:tasks.cephfs.filesystem:scrub status for tag:c1b07303-7ab7-4d46-a269-437577f19a51 - {'path': '/', 'tag': 'c1b07303-7ab7-4d46-a269-437577f19a51', 'options': 'recursive,force'} 2024-09-13T17:30:37.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:37 smithi044 ceph-mon[31175]: pgmap v120: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-13T17:30:37.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:37 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:30:37.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:37 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:37.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:37 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:30:37.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:37 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:37 smithi070 ceph-mon[34502]: pgmap v120: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-13T17:30:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:37 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:30:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:37 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:37 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:30:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:37 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:37 smithi100 ceph-mon[35055]: pgmap v120: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-13T17:30:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:37 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:30:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:37 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:37 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:30:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:37 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:30:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:39 smithi044 ceph-mon[31175]: pgmap v121: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-13T17:30:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:39 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:30:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:39 smithi070 ceph-mon[34502]: pgmap v121: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-13T17:30:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:39 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:30:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:39 smithi100 ceph-mon[35055]: pgmap v121: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-13T17:30:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:39 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:30:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:41 smithi044 ceph-mon[31175]: pgmap v122: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:30:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:41 smithi070 ceph-mon[34502]: pgmap v122: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:30:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:41 smithi100 ceph-mon[35055]: pgmap v122: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:30:43.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:43 smithi044 ceph-mon[31175]: pgmap v123: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:30:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:43 smithi070 ceph-mon[34502]: pgmap v123: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:30:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:43 smithi100 ceph-mon[35055]: pgmap v123: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:30:45.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:45 smithi044 ceph-mon[31175]: pgmap v124: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-13T17:30:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:45 smithi070 ceph-mon[34502]: pgmap v124: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-13T17:30:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:45 smithi100 ceph-mon[35055]: pgmap v124: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-13T17:30:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:47 smithi044 ceph-mon[31175]: pgmap v125: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-13T17:30:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:47 smithi070 ceph-mon[34502]: pgmap v125: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-13T17:30:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:47 smithi100 ceph-mon[35055]: pgmap v125: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-13T17:30:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.413+0000 7fc72b8c9640 1 -- 172.21.15.44:0/711185247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240feff0 msgr2=0x7fc7240ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.413+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/711185247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240feff0 0x7fc7240ff430 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc714009930 tx=0x7fc71402a420 comp rx=0 tx=0).stop 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 -- 172.21.15.44:0/711185247 shutdown_connections 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/711185247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7240ff970 0x7fc72410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/711185247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240feff0 0x7fc7240ff430 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/711185247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724104c30 0x7fc724105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 -- 172.21.15.44:0/711185247 >> 172.21.15.44:0/711185247 conn(0x7fc724074940 msgr2=0x7fc724074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 -- 172.21.15.44:0/711185247 shutdown_connections 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 -- 172.21.15.44:0/711185247 wait complete. 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 Processor -- start 2024-09-13T17:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.414+0000 7fc72b8c9640 1 -- start start 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc72b8c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7240feff0 0x7fc724196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc72b8c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 0x7fc7241966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc72b8c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc724104c30 0x7fc72419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc72b8c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72419b0c0 con 0x7fc7240ff970 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc72b8c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72419b230 con 0x7fc7240feff0 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc72b8c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72419b3f0 con 0x7fc724104c30 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 0x7fc7241966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 0x7fc7241966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48674/0 (socket says 172.21.15.44:48674) 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 -- 172.21.15.44:0/4126953938 learned_addr learned my addr 172.21.15.44:0/4126953938 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc729665640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7240feff0 0x7fc724196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc729e66640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc724104c30 0x7fc72419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 -- 172.21.15.44:0/4126953938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc724104c30 msgr2=0x7fc72419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc724104c30 0x7fc72419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 -- 172.21.15.44:0/4126953938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7240feff0 msgr2=0x7fc724196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7240feff0 0x7fc724196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 -- 172.21.15.44:0/4126953938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714009560 con 0x7fc7240ff970 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc729665640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7240feff0 0x7fc724196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc728e64640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 0x7fc7241966d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ab0 tx=0x7fc714032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.415+0000 7fc71a7fc640 1 -- 172.21.15.44:0/4126953938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140044f0 con 0x7fc7240ff970 2024-09-13T17:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.416+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72419b670 con 0x7fc7240ff970 2024-09-13T17:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.416+0000 7fc71a7fc640 1 -- 172.21.15.44:0/4126953938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc714032e10 con 0x7fc7240ff970 2024-09-13T17:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.416+0000 7fc71a7fc640 1 -- 172.21.15.44:0/4126953938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714003850 con 0x7fc7240ff970 2024-09-13T17:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.416+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7241a2f50 con 0x7fc7240ff970 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.418+0000 7fc71a7fc640 1 -- 172.21.15.44:0/4126953938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc714005210 con 0x7fc7240ff970 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.418+0000 7fc72e5e8740 1 Processor -- start 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 -- start start 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 0x55701ba013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba03cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b95dc30 con 0x55701b9f2210 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b95b3c0 con 0x55701b9f2b30 2024-09-13T17:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc71a7fc640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7100a1200 0x7fc7100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b956cb0 con 0x55701ba01900 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729665640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 0x55701ba013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48688/0 (socket says 172.21.15.44:48688) 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729665640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 0x55701ba013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:45422/0 (socket says 172.21.15.44:45422) 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 -- 172.21.15.44:0/2205505758 learned_addr learned my addr 172.21.15.44:0/2205505758 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc728e64640 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba03cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7100a1200 0x7fc7100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc71a7fc640 1 -- 172.21.15.44:0/4126953938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc7140edba0 con 0x7fc7240ff970 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 -- 172.21.15.44:0/2205505758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 msgr2=0x55701ba03cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba03cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 -- 172.21.15.44:0/2205505758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 msgr2=0x55701ba013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 0x55701ba013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729e66640 1 -- 172.21.15.44:0/2205505758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55701b8fd910 con 0x55701b9f2210 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.419+0000 7fc729665640 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 0x55701ba013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.420+0000 7fc729e66640 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f25f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc71c012d80 tx=0x7fc71c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b0241f0968aff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.420+0000 7fc729e66640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7100a1200 0x7fc7100a36c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc71c010c40 tx=0x7fc71c017000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.420+0000 7fc70f7fe640 1 -- 172.21.15.44:0/2205505758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c012890 con 0x55701b9f2210 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.420+0000 7fc70f7fe640 1 -- 172.21.15.44:0/2205505758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc71c009d80 con 0x55701b9f2210 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.420+0000 7fc70f7fe640 1 -- 172.21.15.44:0/2205505758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003d70 con 0x55701b9f2210 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2205505758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 msgr2=0x55701b9f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f25f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc71c012d80 tx=0x7fc71c010740 comp rx=0 tx=0).stop 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2205505758 shutdown_connections 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba03cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2b30 0x55701ba013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2205505758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f25f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2205505758 >> 172.21.15.44:0/2205505758 conn(0x55701b9ee8c0 msgr2=0x55701b9eeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2205505758 shutdown_connections 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2205505758 wait complete. 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 -- start start 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.421+0000 7fc72e5e8740 1 Processor -- start 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2210 0x55701ba01360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701b9fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b914e80 con 0x55701b9f2b30 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b914b20 con 0x55701ba01900 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b8da260 con 0x55701b9f2210 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc72e5e8740 10 client.0 _ll_register_callbacks cb 0x55701b90fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701b9fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701b9fe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48692/0 (socket says 172.21.15.44:48692) 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 -- 172.21.15.44:0/2932131953 learned_addr learned my addr 172.21.15.44:0/2932131953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc729665640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2210 0x55701ba01360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc729e66640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 -- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2210 msgr2=0x55701ba01360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2210 0x55701ba01360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 -- 172.21.15.44:0/2932131953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 msgr2=0x55701b9fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 -- 172.21.15.44:0/2932131953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c01a040 con 0x55701b9f2b30 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc729665640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2210 0x55701ba01360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc728e64640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701b9fe600 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc714000c00 tx=0x7fc714043510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b5e994951ed305 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc70cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.422+0000 7fc70cff9640 1 -- 172.21.15.44:0/2932131953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714004e20 con 0x55701b9f2b30 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc70cff9640 1 -- 172.21.15.44:0/2932131953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc7140ba9e0 con 0x55701b9f2b30 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55701b97f680 con 0x55701b9f2b30 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc70cff9640 1 -- 172.21.15.44:0/2932131953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7140be760 con 0x55701b9f2b30 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 10 client.16857 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55701b955330 con 0x55701b9f2b30 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc70cff9640 1 -- 172.21.15.44:0/2932131953 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55701b97f680 con 0x55701b9f2b30 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc70cff9640 1 -- 172.21.15.44:0/2932131953 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7140046c0 con 0x55701b9f2b30 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 10 client.16857 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 20 client.16857 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba11170 0x55701ba13590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 4 client.16857 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55701b94a5d0 con 0x55701ba11170 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.423+0000 7fc729665640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba11170 0x55701ba13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc729665640 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba11170 0x55701ba13590 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a004fcf0d0a3e7 server_cookie=8ad4922ee842a889 in_seq=0 out_seq=0 2024-09-13T17:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc70cff9640 10 client.16857 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc70cff9640 1 -- 172.21.15.44:0/2932131953 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55701b94a5d0 con 0x55701ba11170 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc70cff9640 10 client.16857 handle_command_reply: tid=0 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc72e5e8740 1 client.16857 shutdown 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 msgr2=0x55701b9fe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.424+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701b9fe600 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc714000c00 tx=0x7fc714043510 comp rx=0 tx=0).stop 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 shutdown_connections 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba11170 0x55701ba13590 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701b9fe600 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc70cff9640 0 client.16857 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2932131953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2210 0x55701ba01360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 >> 172.21.15.44:0/2932131953 conn(0x55701b9ee8c0 msgr2=0x55701b9f4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 shutdown_connections 2024-09-13T17:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2932131953 wait complete. 2024-09-13T17:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.425+0000 7fc72e5e8740 20 client.16857 trim_cache size 0 max 16384 2024-09-13T17:30:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.437+0000 7fc72e5e8740 1 Processor -- start 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.437+0000 7fc72e5e8740 1 -- start start 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2b30 0x55701b9f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b9ff5c0 con 0x55701b9f2210 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b94a600 con 0x55701ba01900 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701b9f38a0 con 0x55701b9f2b30 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48700/0 (socket says 172.21.15.44:48700) 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 -- 172.21.15.44:0/1665759628 learned_addr learned my addr 172.21.15.44:0/1665759628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc728e64640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2b30 0x55701b9f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729e66640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 -- 172.21.15.44:0/1665759628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2b30 msgr2=0x55701b9f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2b30 0x55701b9f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 -- 172.21.15.44:0/1665759628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 msgr2=0x55701b9fceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 -- 172.21.15.44:0/1665759628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55701b8f7300 con 0x55701b9f2210 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc728e64640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2b30 0x55701b9f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:30:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729e66640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc729665640 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f98c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55701ba13e60 tx=0x7fc720005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f47b6b91d67a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc70ffff640 1 -- 172.21.15.44:0/1665759628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000fc60 con 0x55701b9f2210 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc70ffff640 1 -- 172.21.15.44:0/1665759628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc720018040 con 0x55701b9f2210 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.438+0000 7fc70ffff640 1 -- 172.21.15.44:0/1665759628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720004b90 con 0x55701b9f2210 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 -- 172.21.15.44:0/1665759628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 msgr2=0x55701b9f98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f98c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55701ba13e60 tx=0x7fc720005f90 comp rx=0 tx=0).stop 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 -- 172.21.15.44:0/1665759628 shutdown_connections 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701b9f2b30 0x55701b9f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701ba01900 0x55701b9fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/1665759628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2210 0x55701b9f98c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 -- 172.21.15.44:0/1665759628 >> 172.21.15.44:0/1665759628 conn(0x55701b9ee8c0 msgr2=0x55701ba17eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 -- 172.21.15.44:0/1665759628 shutdown_connections 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 -- 172.21.15.44:0/1665759628 wait complete. 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 -- start start 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.439+0000 7fc72e5e8740 1 Processor -- start 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2210 0x55701ba05820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701ba05d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba67630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701ba06480 con 0x55701b9f2b30 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701ba67b70 con 0x55701b9f2210 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701ba67ce0 con 0x55701ba01900 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc72e5e8740 10 client.0 _ll_register_callbacks cb 0x7fc71c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:30:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701ba05d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701ba05d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48706/0 (socket says 172.21.15.44:48706) 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 -- 172.21.15.44:0/2368159399 learned_addr learned my addr 172.21.15.44:0/2368159399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc729665640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2210 0x55701ba05820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc729e66640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba67630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 -- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 msgr2=0x55701ba67630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba67630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 -- 172.21.15.44:0/2368159399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2210 msgr2=0x55701ba05820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2210 0x55701ba05820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 -- 172.21.15.44:0/2368159399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720011800 con 0x55701b9f2b30 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc728e64640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701ba05d60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc714030040 tx=0x7fc7140ba470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3500ac13aa4071cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.440+0000 7fc70cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:30:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc70cff9640 1 -- 172.21.15.44:0/2368159399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140be760 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc70cff9640 1 -- 172.21.15.44:0/2368159399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc7140c2db0 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc71c009d80 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc70cff9640 1 -- 172.21.15.44:0/2368159399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc714043320 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 10 client.16863 fetch_fsmap learned FSMap version 25 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55701b91c8e0 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc70cff9640 1 -- 172.21.15.44:0/2368159399 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc71c009d80 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc70cff9640 1 -- 172.21.15.44:0/2368159399 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7140c2920 con 0x55701b9f2b30 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 10 client.16863 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 20 client.16863 populate_metadata read hostname 'smithi044' 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba68f50 0x55701ba6b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 4 client.16863 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:30:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.441+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55701b64a300 con 0x55701ba68f50 2024-09-13T17:30:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.442+0000 7fc729665640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba68f50 0x55701ba6b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:30:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.442+0000 7fc729665640 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba68f50 0x55701ba6b340 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f39c4b0b65ae6da server_cookie=1b69ffd62a8f1547 in_seq=0 out_seq=0 2024-09-13T17:30:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.442+0000 7fc70cff9640 10 client.16863 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.456+0000 7fc70cff9640 1 -- 172.21.15.44:0/2368159399 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55701b64a300 con 0x55701ba68f50 2024-09-13T17:30:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.456+0000 7fc70cff9640 10 client.16863 handle_command_reply: tid=0 2024-09-13T17:30:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.456+0000 7fc72e5e8740 1 client.16863 shutdown 2024-09-13T17:30:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.456+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 msgr2=0x55701ba05d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.456+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701ba05d60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc714030040 tx=0x7fc7140ba470 comp rx=0 tx=0).stop 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 shutdown_connections 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55701ba68f50 0x55701ba6b340 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55701ba01900 0x55701ba67630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701b9f2b30 0x55701ba05d60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc70cff9640 0 client.16863 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 --2- 172.21.15.44:0/2368159399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55701b9f2210 0x55701ba05820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 >> 172.21.15.44:0/2368159399 conn(0x55701b9ee8c0 msgr2=0x55701b9fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 shutdown_connections 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 1 -- 172.21.15.44:0/2368159399 wait complete. 2024-09-13T17:30:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72e5e8740 20 client.16863 trim_cache size 0 max 16384 2024-09-13T17:30:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:30:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:30:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:30:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7100a1200 msgr2=0x7fc7100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7100a1200 0x7fc7100a36c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc71c010c40 tx=0x7fc71c017000 comp rx=0 tx=0).stop 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 msgr2=0x7fc7241966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 0x7fc7241966d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ab0 tx=0x7fc714032c20 comp rx=0 tx=0).stop 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 shutdown_connections 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc7100a1200 0x7fc7100a36c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc724104c30 0x7fc72419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7240ff970 0x7fc7241966d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 --2- 172.21.15.44:0/4126953938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7240feff0 0x7fc724196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:30:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.457+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 >> 172.21.15.44:0/4126953938 conn(0x7fc724074940 msgr2=0x7fc724067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:30:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.458+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 shutdown_connections 2024-09-13T17:30:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:30:49.458+0000 7fc72b8c9640 1 -- 172.21.15.44:0/4126953938 wait complete. 2024-09-13T17:30:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:49 smithi044 ceph-mon[31175]: pgmap v126: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 18 op/s 2024-09-13T17:30:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:49 smithi070 ceph-mon[34502]: pgmap v126: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 18 op/s 2024-09-13T17:30:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:49 smithi100 ceph-mon[35055]: pgmap v126: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 18 op/s 2024-09-13T17:30:51.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:51 smithi044 ceph-mon[31175]: pgmap v127: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:30:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:51 smithi070 ceph-mon[34502]: pgmap v127: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:30:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:51 smithi100 ceph-mon[35055]: pgmap v127: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:30:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:53 smithi044 ceph-mon[31175]: pgmap v128: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:30:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:53 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:30:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:53 smithi070 ceph-mon[34502]: pgmap v128: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:30:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:53 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:30:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:53 smithi100 ceph-mon[35055]: pgmap v128: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-13T17:30:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:53 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:30:54.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:30:54.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:30:54.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:54 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:30:54.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:30:54.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:30:54.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:54 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:30:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:30:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:30:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:54 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:30:56.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:55 smithi044 ceph-mon[31175]: pgmap v129: 129 pgs: 129 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 18 op/s 2024-09-13T17:30:56.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:55 smithi070 ceph-mon[34502]: pgmap v129: 129 pgs: 129 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 18 op/s 2024-09-13T17:30:56.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:55 smithi100 ceph-mon[35055]: pgmap v129: 129 pgs: 129 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 18 op/s 2024-09-13T17:30:58.076 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:57 smithi100 ceph-mon[35055]: pgmap v130: 129 pgs: 129 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-13T17:30:58.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:57 smithi044 ceph-mon[31175]: pgmap v130: 129 pgs: 129 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-13T17:30:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:57 smithi070 ceph-mon[34502]: pgmap v130: 129 pgs: 129 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-13T17:31:00.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:30:59 smithi044 ceph-mon[31175]: pgmap v131: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 18 op/s 2024-09-13T17:31:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:30:59 smithi070 ceph-mon[34502]: pgmap v131: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 18 op/s 2024-09-13T17:31:00.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:30:59 smithi100 ceph-mon[35055]: pgmap v131: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 18 op/s 2024-09-13T17:31:02.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:01 smithi044 ceph-mon[31175]: pgmap v132: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:31:02.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:01 smithi070 ceph-mon[34502]: pgmap v132: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:31:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:01 smithi100 ceph-mon[35055]: pgmap v132: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 -- 172.21.15.44:0/4122994901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c1008a0 msgr2=0x7f9f5c1048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/4122994901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c1008a0 0x7f9f5c1048e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009930 tx=0x7f9f5002a420 comp rx=0 tx=0).stop 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 -- 172.21.15.44:0/4122994901 shutdown_connections 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/4122994901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f5c105040 0x7f9f5c10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/4122994901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c1008a0 0x7f9f5c1048e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/4122994901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c0ffef0 0x7f9f5c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 -- 172.21.15.44:0/4122994901 >> 172.21.15.44:0/4122994901 conn(0x7f9f5c0fb960 msgr2=0x7f9f5c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 -- 172.21.15.44:0/4122994901 shutdown_connections 2024-09-13T17:31:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.557+0000 7f9f60ffb640 1 -- 172.21.15.44:0/4122994901 wait complete. 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.558+0000 7f9f60ffb640 1 Processor -- start 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.558+0000 7f9f60ffb640 1 -- start start 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f60ffb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c0ffef0 0x7f9f5c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f60ffb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 0x7f9f5c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f60ffb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f5c073230 0x7f9f5c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f60ffb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c1a4940 con 0x7f9f5c105040 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f60ffb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c1a4ab0 con 0x7f9f5c073230 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f60ffb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c1a4c70 con 0x7f9f5c0ffef0 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 0x7f9f5c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 0x7f9f5c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45692/0 (socket says 172.21.15.44:45692) 2024-09-13T17:31:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2817000668 learned_addr learned my addr 172.21.15.44:0/2817000668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c0ffef0 0x7f9f5c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f5c073230 0x7f9f5c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2817000668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c0ffef0 msgr2=0x7f9f5c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c0ffef0 0x7f9f5c0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2817000668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f5c073230 msgr2=0x7f9f5c1a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f5c073230 0x7f9f5c1a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2817000668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009560 con 0x7f9f5c105040 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.559+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 0x7f9f5c072cf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005ab0 tx=0x7f9f50032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.560+0000 7f9f4b7fe640 1 -- 172.21.15.44:0/2817000668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50032e10 con 0x7f9f5c105040 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.560+0000 7f9f4b7fe640 1 -- 172.21.15.44:0/2817000668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f500046c0 con 0x7f9f5c105040 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.560+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c1a4ef0 con 0x7f9f5c105040 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.560+0000 7f9f4b7fe640 1 -- 172.21.15.44:0/2817000668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50003850 con 0x7f9f5c105040 2024-09-13T17:31:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.560+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c1a5310 con 0x7f9f5c105040 2024-09-13T17:31:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.562+0000 7f9f4b7fe640 1 -- 172.21.15.44:0/2817000668 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9f50004830 con 0x7f9f5c105040 2024-09-13T17:31:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.562+0000 7f9f63df3740 1 Processor -- start 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 -- start start 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33185710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df331928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df330f0c30 con 0x55df331904f0 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df330ee3c0 con 0x55df33185c50 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f63df3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df330e9cb0 con 0x55df33185330 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f4b7fe640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f440a1230 0x7f9f440a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df331928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f440a1230 0x7f9f440a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df331928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45708/0 (socket says 172.21.15.44:45708) 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f59d9b640 1 -- 172.21.15.44:0/3874897819 learned_addr learned my addr 172.21.15.44:0/3874897819 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f4b7fe640 1 -- 172.21.15.44:0/2817000668 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9f500edc60 con 0x7f9f5c105040 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.563+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33185710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f59d9b640 1 -- 172.21.15.44:0/3874897819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 msgr2=0x55df33185710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33185710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f59d9b640 1 -- 172.21.15.44:0/3874897819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 msgr2=0x55df3318ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f59d9b640 1 -- 172.21.15.44:0/3874897819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df33090910 con 0x55df331904f0 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f440a1230 0x7f9f440a36f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f54005fd0 tx=0x7f9f54009d50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df331928e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9f50000c00 tx=0x7f9f500b01a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef88c06738001a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f48ff9640 1 -- 172.21.15.44:0/3874897819 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500ba590 con 0x55df331904f0 2024-09-13T17:31:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f48ff9640 1 -- 172.21.15.44:0/3874897819 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f500ba3b0 con 0x55df331904f0 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.564+0000 7f9f48ff9640 1 -- 172.21.15.44:0/3874897819 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50043320 con 0x55df331904f0 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 -- 172.21.15.44:0/3874897819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 msgr2=0x55df331928e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df331928e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9f50000c00 tx=0x7f9f500b01a0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 -- 172.21.15.44:0/3874897819 shutdown_connections 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df331928e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 --2- 172.21.15.44:0/3874897819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33185710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 -- 172.21.15.44:0/3874897819 >> 172.21.15.44:0/3874897819 conn(0x55df331818c0 msgr2=0x55df33181ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 -- 172.21.15.44:0/3874897819 shutdown_connections 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 -- 172.21.15.44:0/3874897819 wait complete. 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 -- start start 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.565+0000 7f9f63df3740 1 Processor -- start 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 0x55df3318fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df331904f0 0x55df3318d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df330a7e80 con 0x55df33185330 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df330a7b20 con 0x55df33185c50 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3306d260 con 0x55df331904f0 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f63df3740 10 client.0 _ll_register_callbacks cb 0x55df330a2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 0x55df3318fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 0x55df3318fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45718/0 (socket says 172.21.15.44:45718) 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 -- 172.21.15.44:0/1474608313 learned_addr learned my addr 172.21.15.44:0/1474608313 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df331904f0 0x55df3318d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 -- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df331904f0 msgr2=0x55df3318d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df331904f0 0x55df3318d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 -- 172.21.15.44:0/1474608313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 msgr2=0x55df3318d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 -- 172.21.15.44:0/1474608313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c1a5310 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df331904f0 0x55df3318d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 0x55df3318fea0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bef0 tx=0x7f9f4c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c12af5b0d2395ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.566+0000 7f9f41ffb640 1 -- 172.21.15.44:0/1474608313 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0108a0 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f41ffb640 1 -- 172.21.15.44:0/1474608313 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9f4c010a00 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df33112680 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f41ffb640 1 -- 172.21.15.44:0/1474608313 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c010bc0 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 10 client.16875 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55df330e8330 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f41ffb640 1 -- 172.21.15.44:0/1474608313 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55df33112680 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f41ffb640 1 -- 172.21.15.44:0/1474608313 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f4c01a3e0 con 0x55df33185330 2024-09-13T17:31:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 10 client.16875 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 20 client.16875 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df3319fd00 0x55df331a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 4 client.16875 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.567+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df330dd5d0 con 0x55df3319fd00 2024-09-13T17:31:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.568+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df3319fd00 0x55df331a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.568+0000 7f9f41ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.568+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df3319fd00 0x55df331a2120 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1dbac8d52e9353 server_cookie=8c2627ac895718aa in_seq=0 out_seq=0 2024-09-13T17:31:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f41ffb640 1 -- 172.21.15.44:0/1474608313 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55df330dd5d0 con 0x55df3319fd00 2024-09-13T17:31:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f41ffb640 10 client.16875 handle_command_reply: tid=0 2024-09-13T17:31:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 client.16875 shutdown 2024-09-13T17:31:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 msgr2=0x55df3318fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 0x55df3318fea0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bef0 tx=0x7f9f4c012820 comp rx=0 tx=0).stop 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 shutdown_connections 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df3319fd00 0x55df331a2120 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df331904f0 0x55df3318d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 --2- 172.21.15.44:0/1474608313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185330 0x55df3318fea0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f41ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 >> 172.21.15.44:0/1474608313 conn(0x55df331818c0 msgr2=0x55df33180b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 shutdown_connections 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 1 -- 172.21.15.44:0/1474608313 wait complete. 2024-09-13T17:31:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.569+0000 7f9f63df3740 20 client.16875 trim_cache size 0 max 16384 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 Processor -- start 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 -- start start 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df3318f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df3318c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df331a7370 con 0x55df331904f0 2024-09-13T17:31:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df331861d0 con 0x55df33185c50 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f63df3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df33186800 con 0x55df33185330 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df3318c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df3318c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45720/0 (socket says 172.21.15.44:45720) 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f5ad9d640 1 -- 172.21.15.44:0/2457109694 learned_addr learned my addr 172.21.15.44:0/2457109694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.582+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df3318f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f5ad9d640 1 -- 172.21.15.44:0/2457109694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 msgr2=0x55df3318f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df3318f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f5ad9d640 1 -- 172.21.15.44:0/2457109694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 msgr2=0x55df3318bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f5ad9d640 1 -- 172.21.15.44:0/2457109694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3308a300 con 0x55df331904f0 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df3318c010 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9f54014240 tx=0x7f9f5400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa4d71dcdbe5be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f497fa640 1 -- 172.21.15.44:0/2457109694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54004240 con 0x55df331904f0 2024-09-13T17:31:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f497fa640 1 -- 172.21.15.44:0/2457109694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f5401a040 con 0x55df331904f0 2024-09-13T17:31:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f497fa640 1 -- 172.21.15.44:0/2457109694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5402b990 con 0x55df331904f0 2024-09-13T17:31:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f63df3740 1 -- 172.21.15.44:0/2457109694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 msgr2=0x55df3318c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.583+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df3318c010 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9f54014240 tx=0x7f9f5400b000 comp rx=0 tx=0).stop 2024-09-13T17:31:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- 172.21.15.44:0/2457109694 shutdown_connections 2024-09-13T17:31:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df331904f0 0x55df3318c010 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df33185c50 0x55df3318bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2457109694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df3318f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- 172.21.15.44:0/2457109694 >> 172.21.15.44:0/2457109694 conn(0x55df331818c0 msgr2=0x55df331884b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- 172.21.15.44:0/2457109694 shutdown_connections 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- 172.21.15.44:0/2457109694 wait complete. 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- start start 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 Processor -- start 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33193cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 0x55df33194210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df331904f0 0x55df33194750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df331f6310 con 0x55df33185c50 2024-09-13T17:31:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df331f6480 con 0x55df331904f0 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df331f6640 con 0x55df33185330 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f63df3740 10 client.0 _ll_register_callbacks cb 0x7f9f5002ef80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 0x55df33194210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 0x55df33194210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45732/0 (socket says 172.21.15.44:45732) 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2058506803 learned_addr learned my addr 172.21.15.44:0/2058506803 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33193cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.584+0000 7f9f5ad9d640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df331904f0 0x55df33194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 msgr2=0x55df33193cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33193cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2058506803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df331904f0 msgr2=0x55df33194750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df331904f0 0x55df33194750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f59d9b640 1 -- 172.21.15.44:0/2058506803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df33090910 con 0x55df33185c50 2024-09-13T17:31:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f59d9b640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 0x55df33194210 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9f500d66d0 tx=0x7f9f50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c29bbe6df38da7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f41ffb640 1 -- 172.21.15.44:0/2058506803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500baa50 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f41ffb640 1 -- 172.21.15.44:0/2058506803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f9f500bb150 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f500ba3b0 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f41ffb640 1 -- 172.21.15.44:0/2058506803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f500eb310 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f63df3740 10 client.16881 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9f540096b0 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.585+0000 7f9f41ffb640 1 -- 172.21.15.44:0/2058506803 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f50086020 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f41ffb640 1 -- 172.21.15.44:0/2058506803 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f5002e260 con 0x55df33185c50 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f63df3740 10 client.16881 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f63df3740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f63df3740 20 client.16881 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df331f7a70 0x55df331f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f63df3740 4 client.16881 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:31:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df32ddd300 con 0x55df331f7a70 2024-09-13T17:31:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df331f7a70 0x55df331f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f5a59c640 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df331f7a70 0x55df331f9e90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c45be413cc0502 server_cookie=77b79dbf10ce1941 in_seq=0 out_seq=0 2024-09-13T17:31:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.586+0000 7f9f41ffb640 10 client.16881 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f41ffb640 1 -- 172.21.15.44:0/2058506803 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df32ddd300 con 0x55df331f7a70 2024-09-13T17:31:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f41ffb640 10 client.16881 handle_command_reply: tid=0 2024-09-13T17:31:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 client.16881 shutdown 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 msgr2=0x55df33194210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 0x55df33194210 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9f500d66d0 tx=0x7f9f50009260 comp rx=0 tx=0).stop 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 shutdown_connections 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55df331f7a70 0x55df331f9e90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55df331904f0 0x55df33194750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df33185c50 0x55df33194210 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f41ffb640 0 client.16881 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 --2- 172.21.15.44:0/2058506803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df33185330 0x55df33193cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 >> 172.21.15.44:0/2058506803 conn(0x55df331818c0 msgr2=0x55df3318ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 shutdown_connections 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 1 -- 172.21.15.44:0/2058506803 wait complete. 2024-09-13T17:31:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.601+0000 7f9f63df3740 20 client.16881 trim_cache size 0 max 16384 2024-09-13T17:31:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:31:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:31:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:31:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f440a1230 msgr2=0x7f9f440a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f440a1230 0x7f9f440a36f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f54005fd0 tx=0x7f9f54009d50 comp rx=0 tx=0).stop 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 msgr2=0x7f9f5c072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 0x7f9f5c072cf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005ab0 tx=0x7f9f50032c20 comp rx=0 tx=0).stop 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 shutdown_connections 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f440a1230 0x7f9f440a36f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f5c073230 0x7f9f5c1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f5c105040 0x7f9f5c072cf0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 --2- 172.21.15.44:0/2817000668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c0ffef0 0x7f9f5c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 >> 172.21.15.44:0/2817000668 conn(0x7f9f5c0fb960 msgr2=0x7f9f5c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 shutdown_connections 2024-09-13T17:31:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:02.602+0000 7f9f60ffb640 1 -- 172.21.15.44:0/2817000668 wait complete. 2024-09-13T17:31:04.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:03 smithi044 ceph-mon[31175]: pgmap v133: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:31:04.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:03 smithi070 ceph-mon[34502]: pgmap v133: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:31:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:03 smithi100 ceph-mon[35055]: pgmap v133: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-13T17:31:05.542 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.542 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.542 INFO:tasks.workunit.client.0.smithi044.stdout:127/600: open fd = 3 2024-09-13T17:31:05.542 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.542 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.542 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:128/600: open fd = 3 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:129/600: open fd = 3 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:130/600: open fd = 3 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:131/600: open fd = 3 2024-09-13T17:31:05.543 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:132/600: open fd = 3 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:133/600: open fd = 3 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:134/600: open fd = 3 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.544 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:135/600: open fd = 3 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:136/600: open fd = 3 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:137/600: open fd = 3 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.545 INFO:tasks.workunit.client.0.smithi044.stdout:138/600: open fd = 3 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:139/600: open fd = 3 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:140/600: open fd = 3 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:141/600: open fd = 3 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.546 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:142/600: open fd = 3 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:143/600: open fd = 3 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:144/600: open fd = 3 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:145/600: open fd = 3 2024-09-13T17:31:05.547 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:146/600: open fd = 3 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:147/600: open fd = 3 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:148/600: open fd = 3 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.548 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:149/600: open fd = 3 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:150/600: open fd = 3 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:151/600: open fd = 3 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.549 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:152/600: open fd = 3 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:153/600: open fd = 3 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:154/600: open fd = 3 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:155/600: open fd = 3 2024-09-13T17:31:05.550 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:156/600: open fd = 3 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:157/600: open fd = 3 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:158/600: open fd = 3 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.551 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:159/600: open fd = 3 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:160/600: open fd = 3 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:161/600: open fd = 3 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:162/600: open fd = 3 2024-09-13T17:31:05.552 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:163/600: open fd = 3 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:164/600: open fd = 3 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:165/600: open fd = 3 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.553 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:166/600: open fd = 3 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:167/600: open fd = 3 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.554 INFO:tasks.workunit.client.0.smithi044.stdout:168/600: open fd = 3 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:169/600: open fd = 3 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:170/600: open fd = 3 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:171/600: open fd = 3 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.555 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:172/600: open fd = 3 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:173/600: open fd = 3 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:174/600: open fd = 3 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:175/600: open fd = 3 2024-09-13T17:31:05.556 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:176/600: open fd = 3 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:177/600: open fd = 3 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:178/600: open fd = 3 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.557 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:179/600: open fd = 3 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:180/600: open fd = 3 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:181/600: open fd = 3 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:182/600: open fd = 3 2024-09-13T17:31:05.558 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:183/600: open fd = 3 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:184/600: open fd = 3 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:185/600: open fd = 3 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.559 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:186/600: open fd = 3 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:187/600: open fd = 3 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:188/600: open fd = 3 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:05.560 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:31:06.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:05 smithi044 ceph-mon[31175]: pgmap v134: 129 pgs: 129 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-09-13T17:31:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:05 smithi070 ceph-mon[34502]: pgmap v134: 129 pgs: 129 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-09-13T17:31:06.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:05 smithi100 ceph-mon[35055]: pgmap v134: 129 pgs: 129 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-09-13T17:31:06.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:31:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 -- 172.21.15.44:0/396735486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd454100ee0 msgr2=0x7fd4541012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 --2- 172.21.15.44:0/396735486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd454100ee0 0x7fd4541012c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ca0 tx=0x7fd44402a950 comp rx=0 tx=0).stop 2024-09-13T17:31:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 -- 172.21.15.44:0/396735486 shutdown_connections 2024-09-13T17:31:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 --2- 172.21.15.44:0/396735486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd45410a330 0x7fd45410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 --2- 172.21.15.44:0/396735486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd454101800 0x7fd454109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 --2- 172.21.15.44:0/396735486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd454100ee0 0x7fd4541012c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 -- 172.21.15.44:0/396735486 >> 172.21.15.44:0/396735486 conn(0x7fd4540fa8d0 msgr2=0x7fd4540fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 -- 172.21.15.44:0/396735486 shutdown_connections 2024-09-13T17:31:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 -- 172.21.15.44:0/396735486 wait complete. 2024-09-13T17:31:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.763+0000 7fd4593b5640 1 Processor -- start 2024-09-13T17:31:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 -- start start 2024-09-13T17:31:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd454100ee0 0x7fd4541961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd454101800 0x7fd4541966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 0x7fd45419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45419b0d0 con 0x7fd45410a330 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45419b240 con 0x7fd454100ee0 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4593b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45419b400 con 0x7fd454101800 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 0x7fd45419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 0x7fd45419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45736/0 (socket says 172.21.15.44:45736) 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 -- 172.21.15.44:0/4152417873 learned_addr learned my addr 172.21.15.44:0/4152417873 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4527fc640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd454101800 0x7fd4541966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd452ffd640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd454100ee0 0x7fd4541961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 -- 172.21.15.44:0/4152417873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd454101800 msgr2=0x7fd4541966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd454101800 0x7fd4541966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 -- 172.21.15.44:0/4152417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd454100ee0 msgr2=0x7fd4541961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd454100ee0 0x7fd4541961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.764+0000 7fd4537fe640 1 -- 172.21.15.44:0/4152417873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444009900 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.765+0000 7fd4537fe640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 0x7fd45419aad0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d950 tx=0x7fd44c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.765+0000 7fd44bfff640 1 -- 172.21.15.44:0/4152417873 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c009a30 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.765+0000 7fd44bfff640 1 -- 172.21.15.44:0/4152417873 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd44c0044b0 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.765+0000 7fd44bfff640 1 -- 172.21.15.44:0/4152417873 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c003b80 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.765+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45419b6e0 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.765+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4541a2f60 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd44bfff640 1 -- 172.21.15.44:0/4152417873 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd44c003d00 con 0x7fd45410a330 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 Processor -- start 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 -- start start 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e43c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e446a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e446fd0 0x55f18e4493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e3a6f20 con 0x55f18e43c730 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e3a46b0 con 0x55f18e446fd0 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.768+0000 7fd45c0da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e39ffa0 con 0x55f18e43be10 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e446a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e446a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45738/0 (socket says 172.21.15.44:45738) 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 -- 172.21.15.44:0/845757156 learned_addr learned my addr 172.21.15.44:0/845757156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4527fc640 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e43c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd44bfff640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4400a1200 0x7fd4400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4527fc640 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e446fd0 0x55f18e4493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 -- 172.21.15.44:0/845757156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 msgr2=0x55f18e43c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e43c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 -- 172.21.15.44:0/845757156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e446fd0 msgr2=0x55f18e4493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e446fd0 0x55f18e4493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 -- 172.21.15.44:0/845757156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18e35a840 con 0x55f18e43c730 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4527fc640 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e446fd0 0x55f18e4493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4537fe640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4400a1200 0x7fd4400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd44bfff640 1 -- 172.21.15.44:0/4152417873 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd44c0cc9b0 con 0x7fd45410a330 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd452ffd640 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e446a90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd44402ae70 tx=0x7fd444005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4642cef47f5ffe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4497fa640 1 -- 172.21.15.44:0/845757156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444040040 con 0x55f18e43c730 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4497fa640 1 -- 172.21.15.44:0/845757156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd444003970 con 0x55f18e43c730 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.769+0000 7fd4497fa640 1 -- 172.21.15.44:0/845757156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444030710 con 0x55f18e43c730 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 -- 172.21.15.44:0/845757156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 msgr2=0x55f18e446a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd4537fe640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4400a1200 0x7fd4400a36c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005fd0 tx=0x7fd44c00dfa0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e446a90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd44402ae70 tx=0x7fd444005c50 comp rx=0 tx=0).stop 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 -- 172.21.15.44:0/845757156 shutdown_connections 2024-09-13T17:31:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e446fd0 0x55f18e4493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e446a90 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 --2- 172.21.15.44:0/845757156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e43c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 -- 172.21.15.44:0/845757156 >> 172.21.15.44:0/845757156 conn(0x55f18e438380 msgr2=0x55f18e438760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 -- 172.21.15.44:0/845757156 shutdown_connections 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.770+0000 7fd45c0da740 1 -- 172.21.15.44:0/845757156 wait complete. 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 -- start start 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 Processor -- start 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e4469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e443d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 0x55f18e444330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e3412a0 con 0x55f18e446fd0 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e078430 con 0x55f18e43c730 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e323680 con 0x55f18e43be10 2024-09-13T17:31:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd45c0da740 10 client.0 _ll_register_callbacks cb 0x55f18e399610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 0x55f18e444330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 0x55f18e444330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45744/0 (socket says 172.21.15.44:45744) 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 -- 172.21.15.44:0/51658039 learned_addr learned my addr 172.21.15.44:0/51658039 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd452ffd640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e443d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e4469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 -- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 msgr2=0x55f18e4469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e4469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 -- 172.21.15.44:0/51658039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 msgr2=0x55f18e443d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e443d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.771+0000 7fd4527fc640 1 -- 172.21.15.44:0/51658039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45410d430 con 0x55f18e446fd0 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd4527fc640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 0x55f18e444330 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0129b0 tx=0x7fd43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7af14422ffdb2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd43a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd43a7fc640 1 -- 172.21.15.44:0/51658039 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c004020 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd43a7fc640 1 -- 172.21.15.44:0/51658039 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd43c004180 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f18e0934e0 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd43a7fc640 1 -- 172.21.15.44:0/51658039 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd43c004340 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 10 client.16893 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f18e39e620 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd43a7fc640 1 -- 172.21.15.44:0/51658039 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f18e0934e0 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd43a7fc640 1 -- 172.21.15.44:0/51658039 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd43c01e070 con 0x55f18e446fd0 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 10 client.16893 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 20 client.16893 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e456d10 0x55f18e459130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 4 client.16893 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.772+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f18e0a47e0 con 0x55f18e456d10 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.773+0000 7fd452ffd640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e456d10 0x55f18e459130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.773+0000 7fd452ffd640 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e456d10 0x55f18e459130 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423ac358b6f40b9d server_cookie=486abb820ccf0a75 in_seq=0 out_seq=0 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.773+0000 7fd43a7fc640 10 client.16893 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd43a7fc640 1 -- 172.21.15.44:0/51658039 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f18e0a47e0 con 0x55f18e456d10 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd43a7fc640 10 client.16893 handle_command_reply: tid=0 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 client.16893 shutdown 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 msgr2=0x55f18e444330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 0x55f18e444330 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0129b0 tx=0x7fd43c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 shutdown_connections 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e456d10 0x55f18e459130 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e446fd0 0x55f18e444330 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e443d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 --2- 172.21.15.44:0/51658039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e43be10 0x55f18e4469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd43a7fc640 0 client.16893 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:06.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 >> 172.21.15.44:0/51658039 conn(0x55f18e438380 msgr2=0x55f18e437510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 shutdown_connections 2024-09-13T17:31:06.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 1 -- 172.21.15.44:0/51658039 wait complete. 2024-09-13T17:31:06.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.774+0000 7fd45c0da740 20 client.16893 trim_cache size 0 max 16384 2024-09-13T17:31:06.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.788+0000 7fd45c0da740 1 Processor -- start 2024-09-13T17:31:06.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.788+0000 7fd45c0da740 1 -- start start 2024-09-13T17:31:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 0x55f18e43ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e43f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e443ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd45c0da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e0934e0 con 0x55f18e43be10 2024-09-13T17:31:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd45c0da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e455680 con 0x55f18e43c730 2024-09-13T17:31:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd45c0da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e439430 con 0x55f18e446fd0 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 0x55f18e43ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 0x55f18e43ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45760/0 (socket says 172.21.15.44:45760) 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 -- 172.21.15.44:0/3059159150 learned_addr learned my addr 172.21.15.44:0/3059159150 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd452ffd640 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e43f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e443ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 -- 172.21.15.44:0/3059159150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 msgr2=0x55f18e443ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e443ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 -- 172.21.15.44:0/3059159150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 msgr2=0x55f18e43f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e43f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 -- 172.21.15.44:0/3059159150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18e340790 con 0x55f18e43be10 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd452ffd640 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e43f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd4527fc640 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 0x55f18e43ef90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0107a0 tx=0x7fd43c012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599c5f266be59725 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.789+0000 7fd449ffb640 1 -- 172.21.15.44:0/3059159150 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c005510 con 0x55f18e43be10 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd449ffb640 1 -- 172.21.15.44:0/3059159150 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd43c018070 con 0x55f18e43be10 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd449ffb640 1 -- 172.21.15.44:0/3059159150 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c010ee0 con 0x55f18e43be10 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 -- 172.21.15.44:0/3059159150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 msgr2=0x55f18e43ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 0x55f18e43ef90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0107a0 tx=0x7fd43c012690 comp rx=0 tx=0).stop 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 -- 172.21.15.44:0/3059159150 shutdown_connections 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e443ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43c730 0x55f18e43f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 --2- 172.21.15.44:0/3059159150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43be10 0x55f18e43ef90 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 -- 172.21.15.44:0/3059159150 >> 172.21.15.44:0/3059159150 conn(0x55f18e438380 msgr2=0x55f18e45da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 -- 172.21.15.44:0/3059159150 shutdown_connections 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 -- 172.21.15.44:0/3059159150 wait complete. 2024-09-13T17:31:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.790+0000 7fd45c0da740 1 -- start start 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 Processor -- start 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43be10 0x55f18e44a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e44a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e4a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e4a57d0 con 0x55f18e43c730 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e4a5940 con 0x55f18e43be10 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18e4a5b00 con 0x55f18e446fd0 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd45c0da740 10 client.0 _ll_register_callbacks cb 0x7fd43c00b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e44a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e44a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45762/0 (socket says 172.21.15.44:45762) 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 -- 172.21.15.44:0/2344942250 learned_addr learned my addr 172.21.15.44:0/2344942250 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd4527fc640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43be10 0x55f18e44a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd4527fc640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e4a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 -- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 msgr2=0x55f18e4a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e4a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 -- 172.21.15.44:0/2344942250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43be10 msgr2=0x55f18e44a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43be10 0x55f18e44a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 -- 172.21.15.44:0/2344942250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f18e39e620 con 0x55f18e43c730 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd4527fc640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e4a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd452ffd640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e44a940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd444009430 tx=0x7fd44402b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db167fd257ed1b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.791+0000 7fd43a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd43a7fc640 1 -- 172.21.15.44:0/2344942250 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444040040 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd43a7fc640 1 -- 172.21.15.44:0/2344942250 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd44402fd50 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd444003970 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd43a7fc640 1 -- 172.21.15.44:0/2344942250 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd444058380 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 10 client.16899 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f18e365ea0 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd43a7fc640 1 -- 172.21.15.44:0/2344942250 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd444003970 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd43a7fc640 1 -- 172.21.15.44:0/2344942250 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd44403a430 con 0x55f18e43c730 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 10 client.16899 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 20 client.16899 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e4ae230 0x55f18e4b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 4 client.16899 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f18e0f7e30 con 0x55f18e4ae230 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.792+0000 7fd4527fc640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e4ae230 0x55f18e4b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd4527fc640 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e4ae230 0x55f18e4b0650 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c4d5ecb8619bbc server_cookie=9a8999e515891541 in_seq=0 out_seq=0 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd43a7fc640 10 client.16899 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd43a7fc640 1 -- 172.21.15.44:0/2344942250 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f18e0f7e30 con 0x55f18e4ae230 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd43a7fc640 10 client.16899 handle_command_reply: tid=0 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 client.16899 shutdown 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 msgr2=0x55f18e44a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e44a940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd444009430 tx=0x7fd44402b0d0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 shutdown_connections 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f18e4ae230 0x55f18e4b0650 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f18e446fd0 0x55f18e4a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd43a7fc640 0 client.16899 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f18e43c730 0x55f18e44a940 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 --2- 172.21.15.44:0/2344942250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f18e43be10 0x55f18e44a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 >> 172.21.15.44:0/2344942250 conn(0x55f18e438380 msgr2=0x55f18e43ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 shutdown_connections 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 1 -- 172.21.15.44:0/2344942250 wait complete. 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.793+0000 7fd45c0da740 20 client.16899 trim_cache size 0 max 16384 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:31:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:31:06.803 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4400a1200 msgr2=0x7fd4400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4400a1200 0x7fd4400a36c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005fd0 tx=0x7fd44c00dfa0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 msgr2=0x7fd45419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 0x7fd45419aad0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d950 tx=0x7fd44c00de10 comp rx=0 tx=0).stop 2024-09-13T17:31:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 shutdown_connections 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4400a1200 0x7fd4400a36c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd454101800 0x7fd4541966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45410a330 0x7fd45419aad0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 --2- 172.21.15.44:0/4152417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd454100ee0 0x7fd4541961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 >> 172.21.15.44:0/4152417873 conn(0x7fd4540fa8d0 msgr2=0x7fd4540fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 shutdown_connections 2024-09-13T17:31:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.794+0000 7fd4593b5640 1 -- 172.21.15.44:0/4152417873 wait complete. 2024-09-13T17:31:06.807 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:31:06.807 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:31:06.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 -- 172.21.15.44:0/280056589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c103fd0 msgr2=0x7f4f6c10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 --2- 172.21.15.44:0/280056589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c103fd0 0x7f4f6c10bb20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009930 tx=0x7f4f6402a420 comp rx=0 tx=0).stop 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 -- 172.21.15.44:0/280056589 shutdown_connections 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 --2- 172.21.15.44:0/280056589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c103fd0 0x7f4f6c10bb20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 --2- 172.21.15.44:0/280056589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f6c0ff830 0x7f4f6c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 --2- 172.21.15.44:0/280056589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f6c0fee80 0x7f4f6c0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 -- 172.21.15.44:0/280056589 >> 172.21.15.44:0/280056589 conn(0x7f4f6c075c60 msgr2=0x7f4f6c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 -- 172.21.15.44:0/280056589 shutdown_connections 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 -- 172.21.15.44:0/280056589 wait complete. 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 Processor -- start 2024-09-13T17:31:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.909+0000 7f4f745aa640 1 -- start start 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f6c0fee80 0x7f4f6c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f6c103fd0 0x7f4f6c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 0x7f4f6c19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19d6c0 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19d830 con 0x7f4f6c103fd0 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19d9a0 con 0x7f4f6c0fee80 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 0x7f4f6c19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 0x7f4f6c19d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45782/0 (socket says 172.21.15.44:45782) 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 -- 172.21.15.44:0/2928607416 learned_addr learned my addr 172.21.15.44:0/2928607416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f71b45640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f6c103fd0 0x7f4f6c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72346640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f6c0fee80 0x7f4f6c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 -- 172.21.15.44:0/2928607416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f6c0fee80 msgr2=0x7f4f6c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f6c0fee80 0x7f4f6c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 -- 172.21.15.44:0/2928607416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f6c103fd0 msgr2=0x7f4f6c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f6c103fd0 0x7f4f6c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 -- 172.21.15.44:0/2928607416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f64009560 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f72b47640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 0x7f4f6c19d180 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009900 tx=0x7f4f64032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f640044f0 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f64005490 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c1a4f80 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64003850 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.910+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c1a5340 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.911+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c100350 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.912+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4f64004850 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.913+0000 7f4f5f7fe640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4f580a1200 0x7f4f580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:06.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.914+0000 7f4f72346640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4f580a1200 0x7f4f580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:06.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.914+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f4f640edb10 con 0x7f4f6c19ccd0 2024-09-13T17:31:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.914+0000 7f4f72346640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4f580a1200 0x7f4f580a36c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005fd0 tx=0x7f4f68009680 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:06.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:06.918+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4f640b6cf0 con 0x7f4f6c19ccd0 2024-09-13T17:31:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.127+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f6c0ff260 con 0x7f4f6c19ccd0 2024-09-13T17:31:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.127+0000 7f4f5f7fe640 1 -- 172.21.15.44:0/2928607416 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f4f640ba310 con 0x7f4f6c19ccd0 2024-09-13T17:31:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:31:07.129 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:31:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4f580a1200 msgr2=0x7f4f580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4f580a1200 0x7f4f580a36c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005fd0 tx=0x7f4f68009680 comp rx=0 tx=0).stop 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 msgr2=0x7f4f6c19d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 0x7f4f6c19d180 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009900 tx=0x7f4f64032ba0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 shutdown_connections 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4f580a1200 0x7f4f580a36c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f6c19ccd0 0x7f4f6c19d180 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f6c103fd0 0x7f4f6c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 --2- 172.21.15.44:0/2928607416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f6c0fee80 0x7f4f6c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 >> 172.21.15.44:0/2928607416 conn(0x7f4f6c075c60 msgr2=0x7f4f6c0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.129+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 shutdown_connections 2024-09-13T17:31:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.130+0000 7f4f745aa640 1 -- 172.21.15.44:0/2928607416 wait complete. 2024-09-13T17:31:07.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:31:07.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 -- 172.21.15.44:0/4135582990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3900fee80 msgr2=0x7fc3900ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 --2- 172.21.15.44:0/4135582990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3900fee80 0x7fc3900ff260 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009930 tx=0x7fc38c02a420 comp rx=0 tx=0).stop 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 -- 172.21.15.44:0/4135582990 shutdown_connections 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 --2- 172.21.15.44:0/4135582990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc390103fd0 0x7fc39010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 --2- 172.21.15.44:0/4135582990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3900ff830 0x7fc390103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 --2- 172.21.15.44:0/4135582990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3900fee80 0x7fc3900ff260 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 -- 172.21.15.44:0/4135582990 >> 172.21.15.44:0/4135582990 conn(0x7fc3900fa930 msgr2=0x7fc3900fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 -- 172.21.15.44:0/4135582990 shutdown_connections 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.245+0000 7fc397cd7640 1 -- 172.21.15.44:0/4135582990 wait complete. 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 Processor -- start 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 -- start start 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3900fee80 0x7fc390196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc390103fd0 0x7fc390196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 0x7fc39019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39019b540 con 0x7fc39019ab50 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39019b6b0 con 0x7fc3900fee80 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc397cd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39019b820 con 0x7fc390103fd0 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 0x7fc39019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 0x7fc39019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45804/0 (socket says 172.21.15.44:45804) 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 -- 172.21.15.44:0/1193968728 learned_addr learned my addr 172.21.15.44:0/1193968728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc395272640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc390103fd0 0x7fc390196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc395a73640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3900fee80 0x7fc390196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 -- 172.21.15.44:0/1193968728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc390103fd0 msgr2=0x7fc390196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc390103fd0 0x7fc390196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 -- 172.21.15.44:0/1193968728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3900fee80 msgr2=0x7fc390196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3900fee80 0x7fc390196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.246+0000 7fc396274640 1 -- 172.21.15.44:0/1193968728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009560 con 0x7fc39019ab50 2024-09-13T17:31:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.247+0000 7fc396274640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 0x7fc39019b000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc388009fd0 tx=0x7fc38800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.247+0000 7fc386ffd640 1 -- 172.21.15.44:0/1193968728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800bea0 con 0x7fc39019ab50 2024-09-13T17:31:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.247+0000 7fc386ffd640 1 -- 172.21.15.44:0/1193968728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc388009940 con 0x7fc39019ab50 2024-09-13T17:31:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.247+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3901a2e00 con 0x7fc39019ab50 2024-09-13T17:31:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.247+0000 7fc386ffd640 1 -- 172.21.15.44:0/1193968728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388010480 con 0x7fc39019ab50 2024-09-13T17:31:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.247+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3901a32b0 con 0x7fc39019ab50 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.249+0000 7fc386ffd640 1 -- 172.21.15.44:0/1193968728 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc388003970 con 0x7fc39019ab50 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.249+0000 7fc39a9f3740 1 Processor -- start 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 -- start start 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b57ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b58b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b58e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b4e6cc0 con 0x55f56b57c6b0 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b4e4450 con 0x55f56b57cfd0 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b4dfd40 con 0x55f56b58bd60 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b57ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b57ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45820/0 (socket says 172.21.15.44:45820) 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 -- 172.21.15.44:0/1749580867 learned_addr learned my addr 172.21.15.44:0/1749580867 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395a73640 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b58b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc386ffd640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc37c0a1280 0x7fc37c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 -- 172.21.15.44:0/1749580867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 msgr2=0x55f56b58e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b58e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 -- 172.21.15.44:0/1749580867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 msgr2=0x55f56b58b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b58b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 -- 172.21.15.44:0/1749580867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f56b4a7620 con 0x55f56b57c6b0 2024-09-13T17:31:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc396274640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc37c0a1280 0x7fc37c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc386ffd640 1 -- 172.21.15.44:0/1193968728 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc3880ccc00 con 0x7fc39019ab50 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.250+0000 7fc395272640 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b57ca90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc380009d90 tx=0x7fc380013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546724442d60a4d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc37bfff640 1 -- 172.21.15.44:0/1749580867 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380009e80 con 0x55f56b57c6b0 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc37bfff640 1 -- 172.21.15.44:0/1749580867 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3800044e0 con 0x55f56b57c6b0 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc37bfff640 1 -- 172.21.15.44:0/1749580867 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380010430 con 0x55f56b57c6b0 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1749580867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 msgr2=0x55f56b57ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc396274640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc37c0a1280 0x7fc37c0a3740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc388099980 tx=0x7fc38806cb80 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b57ca90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc380009d90 tx=0x7fc380013830 comp rx=0 tx=0).stop 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1749580867 shutdown_connections 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b58e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b58b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1749580867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b57ca90 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.251+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1749580867 >> 172.21.15.44:0/1749580867 conn(0x55f56b578d00 msgr2=0x55f56b5790e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1749580867 shutdown_connections 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1749580867 wait complete. 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 -- start start 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 Processor -- start 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b588710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b588c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b49e180 con 0x55f56b57cfd0 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b1b8430 con 0x55f56b57c6b0 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b4635e0 con 0x55f56b58bd60 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.252+0000 7fc39a9f3740 10 client.0 _ll_register_callbacks cb 0x55f56b4cfaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b588710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b588710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45836/0 (socket says 172.21.15.44:45836) 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 -- 172.21.15.44:0/3111670762 learned_addr learned my addr 172.21.15.44:0/3111670762 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395272640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b588c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395272640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 -- 172.21.15.44:0/3111670762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 msgr2=0x55f56b588c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b588c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 -- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 msgr2=0x55f56b58b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 -- 172.21.15.44:0/3111670762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3800099d0 con 0x55f56b57cfd0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395272640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc395a73640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b588710 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc38c000c00 tx=0x7fc38c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1bd8d36c029cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc3797fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc3797fa640 1 -- 172.21.15.44:0/3111670762 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c004430 con 0x55f56b57cfd0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc3797fa640 1 -- 172.21.15.44:0/3111670762 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc38c031960 con 0x55f56b57cfd0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f56b509b80 con 0x55f56b57cfd0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc3797fa640 1 -- 172.21.15.44:0/3111670762 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc38c031b20 con 0x55f56b57cfd0 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc39a9f3740 10 client.16917 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.253+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f56b4de3c0 con 0x55f56b57cfd0 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc3797fa640 1 -- 172.21.15.44:0/3111670762 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f56b509b80 con 0x55f56b57cfd0 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc3797fa640 1 -- 172.21.15.44:0/3111670762 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc38c045470 con 0x55f56b57cfd0 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc39a9f3740 10 client.16917 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc39a9f3740 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc39a9f3740 20 client.16917 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b59b760 0x55f56b59db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc39a9f3740 4 client.16917 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f56b4d21f0 con 0x55f56b59b760 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.254+0000 7fc395272640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b59b760 0x55f56b59db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc3797fa640 10 client.16917 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc395272640 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b59b760 0x55f56b59db80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=599707b5ae9bbf1b server_cookie=4b257a7666815d6f in_seq=0 out_seq=0 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc3797fa640 1 -- 172.21.15.44:0/3111670762 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f56b4d21f0 con 0x55f56b59b760 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc3797fa640 10 client.16917 handle_command_reply: tid=0 2024-09-13T17:31:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 client.16917 shutdown 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 msgr2=0x55f56b588710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b588710 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc38c000c00 tx=0x7fc38c004040 comp rx=0 tx=0).stop 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 shutdown_connections 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b59b760 0x55f56b59db80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b588c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b588710 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc3797fa640 0 client.16917 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/3111670762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 >> 172.21.15.44:0/3111670762 conn(0x55f56b578d00 msgr2=0x55f56b577eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 shutdown_connections 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.255+0000 7fc39a9f3740 1 -- 172.21.15.44:0/3111670762 wait complete. 2024-09-13T17:31:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.256+0000 7fc39a9f3740 20 client.16917 trim_cache size 0 max 16384 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.266+0000 7fc39a9f3740 1 Processor -- start 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.266+0000 7fc39a9f3740 1 -- start start 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b5786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b592a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b592f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b509b80 con 0x55f56b57c6b0 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b58e810 con 0x55f56b57cfd0 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b5896d0 con 0x55f56b58bd60 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b5786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b5786a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45848/0 (socket says 172.21.15.44:45848) 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 -- 172.21.15.44:0/1884607708 learned_addr learned my addr 172.21.15.44:0/1884607708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395a73640 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b592a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 -- 172.21.15.44:0/1884607708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 msgr2=0x55f56b592f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b592f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 -- 172.21.15.44:0/1884607708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 msgr2=0x55f56b592a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b592a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 -- 172.21.15.44:0/1884607708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f56b480770 con 0x55f56b57c6b0 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395a73640 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b592a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc395272640 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b5786a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc380005e00 tx=0x7fc380005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6788e8eefcc75488 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc384ff9640 1 -- 172.21.15.44:0/1884607708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800036f0 con 0x55f56b57c6b0 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc384ff9640 1 -- 172.21.15.44:0/1884607708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc380022040 con 0x55f56b57c6b0 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.267+0000 7fc384ff9640 1 -- 172.21.15.44:0/1884607708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380005630 con 0x55f56b57c6b0 2024-09-13T17:31:07.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1884607708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 msgr2=0x55f56b5786a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b5786a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc380005e00 tx=0x7fc380005f90 comp rx=0 tx=0).stop 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1884607708 shutdown_connections 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b592f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57cfd0 0x55f56b592a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/1884607708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57c6b0 0x55f56b5786a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1884607708 >> 172.21.15.44:0/1884607708 conn(0x55f56b578d00 msgr2=0x55f56b586fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1884607708 shutdown_connections 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 -- 172.21.15.44:0/1884607708 wait complete. 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 -- start start 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.268+0000 7fc39a9f3740 1 Processor -- start 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b58faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b590030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b5f1c60 con 0x55f56b57cfd0 2024-09-13T17:31:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b5f1da0 con 0x55f56b57c6b0 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f56b5f1ee0 con 0x55f56b58bd60 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 10 client.0 _ll_register_callbacks cb 0x7fc3800097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b58faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b58faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45852/0 (socket says 172.21.15.44:45852) 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 -- 172.21.15.44:0/172879450 learned_addr learned my addr 172.21.15.44:0/172879450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395272640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395272640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b590030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 -- 172.21.15.44:0/172879450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 msgr2=0x55f56b590030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b590030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 -- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 msgr2=0x55f56b58f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 -- 172.21.15.44:0/172879450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc380024040 con 0x55f56b57cfd0 2024-09-13T17:31:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395272640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b590030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc395a73640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b58faf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc38c045800 tx=0x7fc38c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a769453b457f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc3797fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc3797fa640 1 -- 172.21.15.44:0/172879450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c031500 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc3797fa640 1 -- 172.21.15.44:0/172879450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc38c000e40 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.269+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3800044e0 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc3797fa640 1 -- 172.21.15.44:0/172879450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc38c009780 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 10 client.16923 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f56b4a5e80 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc3797fa640 1 -- 172.21.15.44:0/172879450 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3800044e0 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc3797fa640 1 -- 172.21.15.44:0/172879450 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc38c003d10 con 0x55f56b57cfd0 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 10 client.16923 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 10 client.16923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 20 client.16923 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b5f33e0 0x55f56b5f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 4 client.16923 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f56b1d34e0 con 0x55f56b5f33e0 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.270+0000 7fc395272640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b5f33e0 0x55f56b5f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc3797fa640 10 client.16923 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc395272640 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b5f33e0 0x55f56b5f5800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afa821fd50014d26 server_cookie=d5f9370761362aa4 in_seq=0 out_seq=0 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc3797fa640 1 -- 172.21.15.44:0/172879450 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f56b1d34e0 con 0x55f56b5f33e0 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc3797fa640 10 client.16923 handle_command_reply: tid=0 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 client.16923 shutdown 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 msgr2=0x55f56b58faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b58faf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc38c045800 tx=0x7fc38c004360 comp rx=0 tx=0).stop 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 shutdown_connections 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f56b5f33e0 0x55f56b5f5800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f56b58bd60 0x55f56b590030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f56b57cfd0 0x55f56b58faf0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc3797fa640 0 client.16923 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 --2- 172.21.15.44:0/172879450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f56b57c6b0 0x55f56b58f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 >> 172.21.15.44:0/172879450 conn(0x55f56b578d00 msgr2=0x55f56b5837c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 shutdown_connections 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 1 -- 172.21.15.44:0/172879450 wait complete. 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.271+0000 7fc39a9f3740 20 client.16923 trim_cache size 0 max 16384 2024-09-13T17:31:07.279 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc37c0a1280 msgr2=0x7fc37c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc37c0a1280 0x7fc37c0a3740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc388099980 tx=0x7fc38806cb80 comp rx=0 tx=0).stop 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 msgr2=0x7fc39019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 0x7fc39019b000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc388009fd0 tx=0x7fc38800b260 comp rx=0 tx=0).stop 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 shutdown_connections 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc37c0a1280 0x7fc37c0a3740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc390103fd0 0x7fc390196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc39019ab50 0x7fc39019b000 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 --2- 172.21.15.44:0/1193968728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3900fee80 0x7fc390196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 >> 172.21.15.44:0/1193968728 conn(0x7fc3900fa930 msgr2=0x7fc3900fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 shutdown_connections 2024-09-13T17:31:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.272+0000 7fc397cd7640 1 -- 172.21.15.44:0/1193968728 wait complete. 2024-09-13T17:31:07.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 -- 172.21.15.44:0/3260268187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4100ee0 msgr2=0x7f7cd41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3260268187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4100ee0 0x7f7cd41012c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009900 tx=0x7f7cc402a480 comp rx=0 tx=0).stop 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 -- 172.21.15.44:0/3260268187 shutdown_connections 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3260268187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd410a330 0x7f7cd410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3260268187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd4101800 0x7f7cd4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3260268187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4100ee0 0x7f7cd41012c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.386+0000 7f7cd929c640 1 -- 172.21.15.44:0/3260268187 >> 172.21.15.44:0/3260268187 conn(0x7f7cd40fa8d0 msgr2=0x7f7cd40fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.387+0000 7f7cd929c640 1 -- 172.21.15.44:0/3260268187 shutdown_connections 2024-09-13T17:31:07.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.387+0000 7f7cd929c640 1 -- 172.21.15.44:0/3260268187 wait complete. 2024-09-13T17:31:07.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.387+0000 7f7cd929c640 1 Processor -- start 2024-09-13T17:31:07.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.387+0000 7f7cd929c640 1 -- start start 2024-09-13T17:31:07.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd4100ee0 0x7f7cd41961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 0x7f7cd41966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd410a330 0x7f7cd419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd419b0d0 con 0x7f7cd4101800 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd419b240 con 0x7f7cd4100ee0 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd419b400 con 0x7f7cd410a330 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 0x7f7cd41966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 0x7f7cd41966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45872/0 (socket says 172.21.15.44:45872) 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 -- 172.21.15.44:0/3181284121 learned_addr learned my addr 172.21.15.44:0/3181284121 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd4100ee0 0x7f7cd41961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd410a330 0x7f7cd419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 -- 172.21.15.44:0/3181284121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd410a330 msgr2=0x7f7cd419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd410a330 0x7f7cd419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 -- 172.21.15.44:0/3181284121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd4100ee0 msgr2=0x7f7cd41961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd4100ee0 0x7f7cd41961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 -- 172.21.15.44:0/3181284121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4009560 con 0x7f7cd4101800 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 0x7f7cd41966e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e9f0 tx=0x7f7cc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cbbfff640 1 -- 172.21.15.44:0/3181284121 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8009650 con 0x7f7cd4101800 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd419b6e0 con 0x7f7cd4101800 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cbbfff640 1 -- 172.21.15.44:0/3181284121 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cc800ccc0 con 0x7f7cd4101800 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cbbfff640 1 -- 172.21.15.44:0/3181284121 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8018660 con 0x7f7cd4101800 2024-09-13T17:31:07.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.388+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41a2f60 con 0x7f7cd4101800 2024-09-13T17:31:07.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.391+0000 7f7cdbfc4740 1 Processor -- start 2024-09-13T17:31:07.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.391+0000 7f7cbbfff640 1 -- 172.21.15.44:0/3181284121 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7cc8010430 con 0x7f7cd4101800 2024-09-13T17:31:07.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.391+0000 7f7cdbfc4740 1 -- start start 2024-09-13T17:31:07.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaef2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaefd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaeffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eae5ccc0 con 0x5584eaef2790 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eae5a450 con 0x5584eaefd950 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eae55d40 con 0x5584eaef30b0 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaef2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaef2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45886/0 (socket says 172.21.15.44:45886) 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 -- 172.21.15.44:0/1362655899 learned_addr learned my addr 172.21.15.44:0/1362655899 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaefd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaeffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 -- 172.21.15.44:0/1362655899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 msgr2=0x5584eaefd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaefd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 -- 172.21.15.44:0/1362655899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 msgr2=0x5584eaeffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaeffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 -- 172.21.15.44:0/1362655899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584eae1d620 con 0x5584eaef2790 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaeffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cbbfff640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7cbc0a1200 0x7f7cbc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaef2b70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009e60 tx=0x7f7ccc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cd93d95818712 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7cbc0a1200 0x7f7cbc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.392+0000 7f7cbbfff640 1 -- 172.21.15.44:0/3181284121 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7cc8014070 con 0x7f7cd4101800 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cb97fa640 1 -- 172.21.15.44:0/1362655899 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc004d20 con 0x5584eaef2790 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cb97fa640 1 -- 172.21.15.44:0/1362655899 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ccc004e80 con 0x5584eaef2790 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cb97fa640 1 -- 172.21.15.44:0/1362655899 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc012730 con 0x5584eaef2790 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/1362655899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 msgr2=0x5584eaef2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaef2b70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009e60 tx=0x7f7ccc010740 comp rx=0 tx=0).stop 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7cbc0a1200 0x7f7cbc0a36c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4000c00 tx=0x7f7cc40058d0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/1362655899 shutdown_connections 2024-09-13T17:31:07.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaefd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaeffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/1362655899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaef2b70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/1362655899 >> 172.21.15.44:0/1362655899 conn(0x5584eaeeed00 msgr2=0x5584eaeef0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/1362655899 shutdown_connections 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.393+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/1362655899 wait complete. 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 -- start start 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 Processor -- start 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef2790 0x5584eaefcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 0x5584eaefa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaefa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eae14180 con 0x5584eaef30b0 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eab2e430 con 0x5584eaefd950 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eadd95e0 con 0x5584eaef2790 2024-09-13T17:31:07.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cdbfc4740 10 client.0 _ll_register_callbacks cb 0x5584eae45aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 0x5584eaefa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 0x5584eaefa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45888/0 (socket says 172.21.15.44:45888) 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 -- 172.21.15.44:0/2728974607 learned_addr learned my addr 172.21.15.44:0/2728974607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef2790 0x5584eaefcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaefa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 -- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef2790 msgr2=0x5584eaefcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef2790 0x5584eaefcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 -- 172.21.15.44:0/2728974607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 msgr2=0x5584eaefa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaefa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd27fc640 1 -- 172.21.15.44:0/2728974607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc009ac0 con 0x5584eaef30b0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.394+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaefa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 0x5584eaefa2f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e9c0 tx=0x7f7cc8099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c244459d1b0b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:07.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/2728974607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc809a8c0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/2728974607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7cc809a1c0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584eae481f0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/2728974607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cc809dff0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 10 client.16935 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5584eae543c0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/2728974607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5584eae481f0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/2728974607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cc80968d0 con 0x5584eaef30b0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 10 client.16935 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 20 client.16935 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf0d2e0 0x5584eaf0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 4 client.16935 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.395+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584eab494e0 con 0x5584eaf0d2e0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.396+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf0d2e0 0x5584eaf0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.396+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf0d2e0 0x5584eaf0f700 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b94f19a18afdd874 server_cookie=40cd224733d78697 in_seq=0 out_seq=0 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.396+0000 7f7cb6ff5640 10 client.16935 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/2728974607 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5584eab494e0 con 0x5584eaf0d2e0 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cb6ff5640 10 client.16935 handle_command_reply: tid=0 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 client.16935 shutdown 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 msgr2=0x5584eaefa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 0x5584eaefa2f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e9c0 tx=0x7f7cc8099b20 comp rx=0 tx=0).stop 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 shutdown_connections 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf0d2e0 0x5584eaf0f700 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaefa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cb6ff5640 0 client.16935 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef30b0 0x5584eaefa2f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/2728974607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef2790 0x5584eaefcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 >> 172.21.15.44:0/2728974607 conn(0x5584eaeeed00 msgr2=0x5584eaeedeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 shutdown_connections 2024-09-13T17:31:07.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/2728974607 wait complete. 2024-09-13T17:31:07.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.397+0000 7f7cdbfc4740 20 client.16935 trim_cache size 0 max 16384 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 Processor -- start 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 -- start start 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf04b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaef8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eaeefe70 con 0x5584eaef2790 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eaef3750 con 0x5584eaefd950 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eaefaba0 con 0x5584eaef30b0 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45894/0 (socket says 172.21.15.44:45894) 2024-09-13T17:31:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/4038576815 learned_addr learned my addr 172.21.15.44:0/4038576815 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf04b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/4038576815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 msgr2=0x5584eaf04b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf04b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/4038576815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 msgr2=0x5584eaef8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaef8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/4038576815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584eadf6770 con 0x5584eaef2790 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.408+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf04b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04610 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4033000 tx=0x7f7cc403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3a7ee737dba1a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cb9ffb640 1 -- 172.21.15.44:0/4038576815 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4031ea0 con 0x5584eaef2790 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cb9ffb640 1 -- 172.21.15.44:0/4038576815 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cc4041070 con 0x5584eaef2790 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cb9ffb640 1 -- 172.21.15.44:0/4038576815 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4003be0 con 0x5584eaef2790 2024-09-13T17:31:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/4038576815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 msgr2=0x5584eaf04610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04610 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4033000 tx=0x7f7cc403f000 comp rx=0 tx=0).stop 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/4038576815 shutdown_connections 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf04b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaef8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/4038576815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04610 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/4038576815 >> 172.21.15.44:0/4038576815 conn(0x5584eaeeed00 msgr2=0x5584eaf090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/4038576815 shutdown_connections 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.409+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/4038576815 wait complete. 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 -- start start 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 Processor -- start 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf00ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaf01480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eaf01b80 con 0x5584eaef2790 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eaf01cf0 con 0x5584eaefd950 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584eaf67bf0 con 0x5584eaef30b0 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cdbfc4740 10 client.0 _ll_register_callbacks cb 0x7f7ccc009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45906/0 (socket says 172.21.15.44:45906) 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/601534692 learned_addr learned my addr 172.21.15.44:0/601534692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd37fe640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaf01480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf00ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 msgr2=0x5584eaf00ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf00ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/601534692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 msgr2=0x5584eaf01480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaf01480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 -- 172.21.15.44:0/601534692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd410d430 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cd2ffd640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04a40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006010 tx=0x7f7cc4054f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ff4a76cce8278d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.410+0000 7f7cb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/601534692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4031ea0 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/601534692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7cc4050070 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584eae7fb80 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/601534692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cc40537b0 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 10 client.16941 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5584eae1be80 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/601534692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5584eae7fb80 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/601534692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cc403d780 con 0x5584eaef2790 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 10 client.16941 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 10 client.16941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 20 client.16941 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf690f0 0x5584eaf6b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 4 client.16941 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.411+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584eab5a7e0 con 0x5584eaf690f0 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf690f0 0x5584eaf6b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cb6ff5640 10 client.16941 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cd27fc640 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf690f0 0x5584eaf6b4e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80881c224dbfd736 server_cookie=a22cbdb246039efe in_seq=0 out_seq=0 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cb6ff5640 1 -- 172.21.15.44:0/601534692 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584eab5a7e0 con 0x5584eaf690f0 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cb6ff5640 10 client.16941 handle_command_reply: tid=0 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 client.16941 shutdown 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 msgr2=0x5584eaf04a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04a40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006010 tx=0x7f7cc4054f30 comp rx=0 tx=0).stop 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 shutdown_connections 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5584eaf690f0 0x5584eaf6b4e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584eaef30b0 0x5584eaf00ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5584eaefd950 0x5584eaf01480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 --2- 172.21.15.44:0/601534692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584eaef2790 0x5584eaf04a40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cb6ff5640 0 client.16941 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 >> 172.21.15.44:0/601534692 conn(0x5584eaeeed00 msgr2=0x5584eaef8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 shutdown_connections 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.412+0000 7f7cdbfc4740 1 -- 172.21.15.44:0/601534692 wait complete. 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cdbfc4740 20 client.16941 trim_cache size 0 max 16384 2024-09-13T17:31:07.420 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7cbc0a1200 msgr2=0x7f7cbc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7cbc0a1200 0x7f7cbc0a36c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4000c00 tx=0x7f7cc40058d0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 msgr2=0x7f7cd41966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 0x7f7cd41966e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e9f0 tx=0x7f7cc800eeb0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 shutdown_connections 2024-09-13T17:31:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7cbc0a1200 0x7f7cbc0a36c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd410a330 0x7f7cd419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cd4101800 0x7f7cd41966e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 --2- 172.21.15.44:0/3181284121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd4100ee0 0x7f7cd41961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 >> 172.21.15.44:0/3181284121 conn(0x7f7cd40fa8d0 msgr2=0x7f7cd40fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 shutdown_connections 2024-09-13T17:31:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.413+0000 7f7cd929c640 1 -- 172.21.15.44:0/3181284121 wait complete. 2024-09-13T17:31:07.425 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:31:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 -- 172.21.15.44:0/1793991244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 msgr2=0x7fe53c0fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 --2- 172.21.15.44:0/1793991244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c0fee40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe538009900 tx=0x7fe53802a410 comp rx=0 tx=0).stop 2024-09-13T17:31:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 -- 172.21.15.44:0/1793991244 shutdown_connections 2024-09-13T17:31:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 --2- 172.21.15.44:0/1793991244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe53c0ff380 0x7fe53c10c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 --2- 172.21.15.44:0/1793991244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 0x7fe53c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 --2- 172.21.15.44:0/1793991244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c0fee40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 -- 172.21.15.44:0/1793991244 >> 172.21.15.44:0/1793991244 conn(0x7fe53c0fa930 msgr2=0x7fe53c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 -- 172.21.15.44:0/1793991244 shutdown_connections 2024-09-13T17:31:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 -- 172.21.15.44:0/1793991244 wait complete. 2024-09-13T17:31:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 Processor -- start 2024-09-13T17:31:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 -- start start 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.529+0000 7fe543a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe53c0ff380 0x7fe53c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe543a7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 0x7fe53c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe543a7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c19d320 con 0x7fe53c0fea60 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe543a7d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c19d490 con 0x7fe53c10c610 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe543a7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c19d650 con 0x7fe53c0ff380 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45918/0 (socket says 172.21.15.44:45918) 2024-09-13T17:31:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 -- 172.21.15.44:0/205476338 learned_addr learned my addr 172.21.15.44:0/205476338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541018640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe53c0ff380 0x7fe53c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe54201a640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 0x7fe53c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 -- 172.21.15.44:0/205476338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe53c0ff380 msgr2=0x7fe53c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe53c0ff380 0x7fe53c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 -- 172.21.15.44:0/205476338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 msgr2=0x7fe53c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 0x7fe53c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 -- 172.21.15.44:0/205476338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538009560 con 0x7fe53c0fea60 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe54201a640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 0x7fe53c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe541819640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c1983f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe538006010 tx=0x7fe5380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe532ffd640 1 -- 172.21.15.44:0/205476338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538004cc0 con 0x7fe53c0fea60 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe532ffd640 1 -- 172.21.15.44:0/205476338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe538003a80 con 0x7fe53c0fea60 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c19d8d0 con 0x7fe53c0fea60 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe532ffd640 1 -- 172.21.15.44:0/205476338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53803cad0 con 0x7fe53c0fea60 2024-09-13T17:31:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.530+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c1a51b0 con 0x7fe53c0fea60 2024-09-13T17:31:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe532ffd640 1 -- 172.21.15.44:0/205476338 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe538003bf0 con 0x7fe53c0fea60 2024-09-13T17:31:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 Processor -- start 2024-09-13T17:31:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 -- start start 2024-09-13T17:31:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff88790 0x56317ff88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff93410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff95d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317fef2cc0 con 0x56317ff93950 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317fef0450 con 0x56317ff890b0 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe546878740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317feebd40 con 0x56317ff88790 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.533+0000 7fe541018640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff95d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff95d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45920/0 (socket says 172.21.15.44:45920) 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 -- 172.21.15.44:0/370740944 learned_addr learned my addr 172.21.15.44:0/370740944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe54201a640 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff88790 0x56317ff88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541819640 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff93410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe532ffd640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5280a1280 0x7fe5280a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 -- 172.21.15.44:0/370740944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff88790 msgr2=0x56317ff88b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff88790 0x56317ff88b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 -- 172.21.15.44:0/370740944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 msgr2=0x56317ff93410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff93410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 -- 172.21.15.44:0/370740944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317feb3620 con 0x56317ff93950 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe54201a640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5280a1280 0x7fe5280a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe532ffd640 1 -- 172.21.15.44:0/205476338 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe5380edf60 con 0x7fe53c0fea60 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe541018640 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff95d40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00b740 tx=0x7fe52c00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4941e0fb632d0cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe527fff640 1 -- 172.21.15.44:0/370740944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c003dd0 con 0x56317ff93950 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe527fff640 1 -- 172.21.15.44:0/370740944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52c00be50 con 0x56317ff93950 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.534+0000 7fe527fff640 1 -- 172.21.15.44:0/370740944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c004e10 con 0x56317ff93950 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 -- 172.21.15.44:0/370740944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 msgr2=0x56317ff95d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff95d40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00b740 tx=0x7fe52c00bc00 comp rx=0 tx=0).stop 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe54201a640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5280a1280 0x7fe5280a3740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe53c068350 tx=0x7fe534004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 -- 172.21.15.44:0/370740944 shutdown_connections 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff95d40 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff93410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 --2- 172.21.15.44:0/370740944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff88790 0x56317ff88b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 -- 172.21.15.44:0/370740944 >> 172.21.15.44:0/370740944 conn(0x56317ff84d00 msgr2=0x56317ff850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 -- 172.21.15.44:0/370740944 shutdown_connections 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 -- 172.21.15.44:0/370740944 wait complete. 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 -- start start 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.535+0000 7fe546878740 1 Processor -- start 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 0x56317ff92eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317feaa180 con 0x56317ff88790 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317fbc4430 con 0x56317ff890b0 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317fe6f5e0 con 0x56317ff93950 2024-09-13T17:31:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe546878740 10 client.0 _ll_register_callbacks cb 0x56317fedbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 0x56317ff92eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 0x56317ff92eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45930/0 (socket says 172.21.15.44:45930) 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 -- 172.21.15.44:0/4001913652 learned_addr learned my addr 172.21.15.44:0/4001913652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe54201a640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541018640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 -- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 msgr2=0x56317ff904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 -- 172.21.15.44:0/4001913652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 msgr2=0x56317ff933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 -- 172.21.15.44:0/4001913652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c004310 con 0x56317ff88790 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe541819640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 0x56317ff92eb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe538000c00 tx=0x7fe538005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc08f2586e9d2bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.536+0000 7fe5257fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe5257fa640 1 -- 172.21.15.44:0/4001913652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380ba6b0 con 0x56317ff88790 2024-09-13T17:31:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe5257fa640 1 -- 172.21.15.44:0/4001913652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe5380bb450 con 0x56317ff88790 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56317ff15b80 con 0x56317ff88790 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe5257fa640 1 -- 172.21.15.44:0/4001913652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5380bad50 con 0x56317ff88790 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 10 client.16953 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56317feea3c0 con 0x56317ff88790 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe5257fa640 1 -- 172.21.15.44:0/4001913652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56317ff15b80 con 0x56317ff88790 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe5257fa640 1 -- 172.21.15.44:0/4001913652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe5380bafb0 con 0x56317ff88790 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 10 client.16953 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 20 client.16953 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317ffa2f10 0x56317ffa5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 4 client.16953 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.537+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56317fede1f0 con 0x56317ffa2f10 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.538+0000 7fe541018640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317ffa2f10 0x56317ffa5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.538+0000 7fe5257fa640 10 client.16953 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.538+0000 7fe541018640 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317ffa2f10 0x56317ffa5300 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789310b91c134466 server_cookie=dc0420e5b60aa6a8 in_seq=0 out_seq=0 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe5257fa640 1 -- 172.21.15.44:0/4001913652 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56317fede1f0 con 0x56317ffa2f10 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe5257fa640 10 client.16953 handle_command_reply: tid=0 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 client.16953 shutdown 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 msgr2=0x56317ff92eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 0x56317ff92eb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe538000c00 tx=0x7fe538005bc0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 shutdown_connections 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317ffa2f10 0x56317ffa5300 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff890b0 0x56317ff933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 --2- 172.21.15.44:0/4001913652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff88790 0x56317ff92eb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe5257fa640 0 client.16953 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 >> 172.21.15.44:0/4001913652 conn(0x56317ff84d00 msgr2=0x56317ff8aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 shutdown_connections 2024-09-13T17:31:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 1 -- 172.21.15.44:0/4001913652 wait complete. 2024-09-13T17:31:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.539+0000 7fe546878740 20 client.16953 trim_cache size 0 max 16384 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 Processor -- start 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 -- start start 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff8b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 0x56317ff8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff94d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317ff91660 con 0x56317ff890b0 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317ffaa430 con 0x56317ff88790 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe546878740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317ff96020 con 0x56317ff93950 2024-09-13T17:31:07.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe541018640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 0x56317ff8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe541018640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 0x56317ff8b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45940/0 (socket says 172.21.15.44:45940) 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.550+0000 7fe541018640 1 -- 172.21.15.44:0/797402570 learned_addr learned my addr 172.21.15.44:0/797402570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe54201a640 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff94d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541819640 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff8b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541018640 1 -- 172.21.15.44:0/797402570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 msgr2=0x56317ff94d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541018640 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff94d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541018640 1 -- 172.21.15.44:0/797402570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 msgr2=0x56317ff8b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541018640 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff8b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541018640 1 -- 172.21.15.44:0/797402570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317fe8c770 con 0x56317ff890b0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541819640 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff8b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe541018640 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 0x56317ff8b8a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004b10 tx=0x7fe52c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff05e1b5ba6081c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe530e05640 1 -- 172.21.15.44:0/797402570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c003d90 con 0x56317ff890b0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe530e05640 1 -- 172.21.15.44:0/797402570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52c0278d0 con 0x56317ff890b0 2024-09-13T17:31:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.551+0000 7fe530e05640 1 -- 172.21.15.44:0/797402570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c011910 con 0x56317ff890b0 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- 172.21.15.44:0/797402570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 msgr2=0x56317ff8b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 0x56317ff8b8a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004b10 tx=0x7fe52c00b390 comp rx=0 tx=0).stop 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- 172.21.15.44:0/797402570 shutdown_connections 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff93950 0x56317ff94d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff890b0 0x56317ff8b8a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- 172.21.15.44:0/797402570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff8b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- 172.21.15.44:0/797402570 >> 172.21.15.44:0/797402570 conn(0x56317ff84d00 msgr2=0x56317ff850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- 172.21.15.44:0/797402570 shutdown_connections 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- 172.21.15.44:0/797402570 wait complete. 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- start start 2024-09-13T17:31:07.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 Processor -- start 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff9a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff890b0 0x56317ff96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff97320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317ff979d0 con 0x56317ff93950 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317ff97b40 con 0x56317ff88790 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317fff9390 con 0x56317ff890b0 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.552+0000 7fe546878740 10 client.0 _ll_register_callbacks cb 0x7fe52c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:07.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff97320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff97320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45950/0 (socket says 172.21.15.44:45950) 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 -- 172.21.15.44:0/527256692 learned_addr learned my addr 172.21.15.44:0/527256692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe541018640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff890b0 0x56317ff96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe541819640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff9a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 -- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff890b0 msgr2=0x56317ff96de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff890b0 0x56317ff96de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 -- 172.21.15.44:0/527256692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 msgr2=0x56317ff9a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff9a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 -- 172.21.15.44:0/527256692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c021040 con 0x56317ff93950 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe541819640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff9a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe54201a640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff97320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe5340129f0 tx=0x7fe534004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fd57871f0712e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe5257fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe5257fa640 1 -- 172.21.15.44:0/527256692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5340218e0 con 0x56317ff93950 2024-09-13T17:31:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe52c00be50 con 0x56317ff93950 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe5257fa640 1 -- 172.21.15.44:0/527256692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe534018040 con 0x56317ff93950 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe5257fa640 1 -- 172.21.15.44:0/527256692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe534021a40 con 0x56317ff93950 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe546878740 10 client.16959 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.553+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56317feb1e80 con 0x56317ff93950 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe5257fa640 1 -- 172.21.15.44:0/527256692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52c00be50 con 0x56317ff93950 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe5257fa640 1 -- 172.21.15.44:0/527256692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe53402acd0 con 0x56317ff93950 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe546878740 10 client.16959 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe546878740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe546878740 20 client.16959 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe546878740 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317fffa9f0 0x56317fffce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe546878740 4 client.16959 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56317fbdf4e0 con 0x56317fffa9f0 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe541819640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317fffa9f0 0x56317fffce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe541819640 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317fffa9f0 0x56317fffce10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60ef890a9898963 server_cookie=93d99ae1972a8ce2 in_seq=0 out_seq=0 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.554+0000 7fe5257fa640 10 client.16959 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe5257fa640 1 -- 172.21.15.44:0/527256692 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56317fbdf4e0 con 0x56317fffa9f0 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe5257fa640 10 client.16959 handle_command_reply: tid=0 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 client.16959 shutdown 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 msgr2=0x56317ff97320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff97320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe5340129f0 tx=0x7fe534004ab0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 shutdown_connections 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56317fffa9f0 0x56317fffce10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317ff890b0 0x56317ff96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56317ff93950 0x56317ff97320 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 --2- 172.21.15.44:0/527256692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56317ff88790 0x56317ff9a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe5257fa640 0 client.16959 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 >> 172.21.15.44:0/527256692 conn(0x56317ff84d00 msgr2=0x56317ff850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 shutdown_connections 2024-09-13T17:31:07.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 1 -- 172.21.15.44:0/527256692 wait complete. 2024-09-13T17:31:07.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe546878740 20 client.16959 trim_cache size 0 max 16384 2024-09-13T17:31:07.563 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:07.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5280a1280 msgr2=0x7fe5280a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe543a7d640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5280a1280 0x7fe5280a3740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe53c068350 tx=0x7fe534004a40 comp rx=0 tx=0).stop 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 msgr2=0x7fe53c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.555+0000 7fe543a7d640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c1983f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe538006010 tx=0x7fe5380046c0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 shutdown_connections 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5280a1280 0x7fe5280a3740 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe53c0ff380 0x7fe53c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe53c10c610 0x7fe53c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 --2- 172.21.15.44:0/205476338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53c0fea60 0x7fe53c1983f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 >> 172.21.15.44:0/205476338 conn(0x7fe53c0fa930 msgr2=0x7fe53c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 shutdown_connections 2024-09-13T17:31:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:07.556+0000 7fe543a7d640 1 -- 172.21.15.44:0/205476338 wait complete. 2024-09-13T17:31:08.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:07 smithi044 ceph-mon[31175]: pgmap v135: 129 pgs: 129 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 55 KiB/s wr, 28 op/s 2024-09-13T17:31:08.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:07 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2928607416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:07 smithi070 ceph-mon[34502]: pgmap v135: 129 pgs: 129 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 55 KiB/s wr, 28 op/s 2024-09-13T17:31:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:07 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2928607416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:07 smithi100 ceph-mon[35055]: pgmap v135: 129 pgs: 129 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 55 KiB/s wr, 28 op/s 2024-09-13T17:31:08.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:07 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2928607416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:08.569 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:31:08.569 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:31:08.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.687+0000 7f96aa46b640 1 -- 172.21.15.44:0/2660578473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 msgr2=0x7f96a410cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 --2- 172.21.15.44:0/2660578473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a410cd70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f969c009d00 tx=0x7f969c02a9a0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 -- 172.21.15.44:0/2660578473 shutdown_connections 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 --2- 172.21.15.44:0/2660578473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a410cd70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 --2- 172.21.15.44:0/2660578473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a40ff480 0x7f96a4105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 --2- 172.21.15.44:0/2660578473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96a40feb60 0x7f96a40fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 -- 172.21.15.44:0/2660578473 >> 172.21.15.44:0/2660578473 conn(0x7f96a40741b0 msgr2=0x7f96a40745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 -- 172.21.15.44:0/2660578473 shutdown_connections 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 -- 172.21.15.44:0/2660578473 wait complete. 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 Processor -- start 2024-09-13T17:31:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.688+0000 7f96aa46b640 1 -- start start 2024-09-13T17:31:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96aa46b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96a40feb60 0x7f96a40ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96aa46b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a40ff480 0x7f96a4100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96aa46b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a4100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96aa46b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41a4650 con 0x7f96a4106420 2024-09-13T17:31:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96aa46b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41a47c0 con 0x7f96a40ff480 2024-09-13T17:31:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96aa46b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41a4980 con 0x7f96a40feb60 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a4100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a4100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45972/0 (socket says 172.21.15.44:45972) 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 -- 172.21.15.44:0/1728990341 learned_addr learned my addr 172.21.15.44:0/1728990341 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96a40feb60 0x7f96a40ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a40ff480 0x7f96a4100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 -- 172.21.15.44:0/1728990341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96a40feb60 msgr2=0x7f96a40ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96a40feb60 0x7f96a40ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 -- 172.21.15.44:0/1728990341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a40ff480 msgr2=0x7f96a4100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a40ff480 0x7f96a4100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 -- 172.21.15.44:0/1728990341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c009900 con 0x7f96a4106420 2024-09-13T17:31:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.689+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a4100550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cd0 tx=0x7f969c02f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.690+0000 7f96a17fa640 1 -- 172.21.15.44:0/1728990341 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c030a70 con 0x7f96a4106420 2024-09-13T17:31:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.690+0000 7f96a17fa640 1 -- 172.21.15.44:0/1728990341 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969c030bd0 con 0x7f96a4106420 2024-09-13T17:31:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.690+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a41a4c00 con 0x7f96a4106420 2024-09-13T17:31:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.690+0000 7f96a17fa640 1 -- 172.21.15.44:0/1728990341 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c03a720 con 0x7f96a4106420 2024-09-13T17:31:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.690+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a41a5100 con 0x7f96a4106420 2024-09-13T17:31:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.692+0000 7f96a17fa640 1 -- 172.21.15.44:0/1728990341 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f969c00a040 con 0x7f96a4106420 2024-09-13T17:31:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.692+0000 7f96ad26a740 1 Processor -- start 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 -- start start 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a17fa640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f968c0a1280 0x7f968c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b3c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6b3ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6a9cc60 con 0x55bac6b3c980 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6a9a3f0 con 0x55bac6b320e0 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96ad26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6a95ce0 con 0x55bac6b317c0 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6b3ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f968c0a1280 0x7f968c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6b3ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45978/0 (socket says 172.21.15.44:45978) 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 -- 172.21.15.44:0/2400045262 learned_addr learned my addr 172.21.15.44:0/2400045262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a17fa640 1 -- 172.21.15.44:0/1728990341 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f969c0ee460 con 0x7f96a4106420 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a3fff640 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a37fe640 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b3c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 -- 172.21.15.44:0/2400045262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 msgr2=0x55bac6b31ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b31ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 -- 172.21.15.44:0/2400045262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 msgr2=0x55bac6b3c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b3c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 -- 172.21.15.44:0/2400045262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bac6a51970 con 0x55bac6b3c980 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f968c0a1280 0x7f968c0a3740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f96a4101250 tx=0x7f96980127c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.693+0000 7f96a8a08640 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6b3ed70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f969c0c3180 tx=0x7f969c02a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aba58b11542e211 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f968a7fc640 1 -- 172.21.15.44:0/2400045262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0d3de0 con 0x55bac6b3c980 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f968a7fc640 1 -- 172.21.15.44:0/2400045262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969c0bb930 con 0x55bac6b3c980 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f968a7fc640 1 -- 172.21.15.44:0/2400045262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0b7940 con 0x55bac6b3c980 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 -- 172.21.15.44:0/2400045262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 msgr2=0x55bac6b3ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6b3ed70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f969c0c3180 tx=0x7f969c02a9a0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 -- 172.21.15.44:0/2400045262 shutdown_connections 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6b3ed70 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b3c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2400045262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b31ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 -- 172.21.15.44:0/2400045262 >> 172.21.15.44:0/2400045262 conn(0x55bac6b2dd50 msgr2=0x55bac6b2e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 -- 172.21.15.44:0/2400045262 shutdown_connections 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.694+0000 7f96ad26a740 1 -- 172.21.15.44:0/2400045262 wait complete. 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 -- start start 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 Processor -- start 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b317c0 0x55bac6b3c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b39560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b3c980 0x55bac6b39aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6a53f80 con 0x55bac6b320e0 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6a53c20 con 0x55bac6b317c0 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6a20ab0 con 0x55bac6b3c980 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96ad26a740 10 client.0 _ll_register_callbacks cb 0x55bac6a8f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b39560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b39560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45988/0 (socket says 172.21.15.44:45988) 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.695+0000 7f96a37fe640 1 -- 172.21.15.44:0/2072240750 learned_addr learned my addr 172.21.15.44:0/2072240750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a8a08640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b3c980 0x55bac6b39aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a3fff640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b317c0 0x55bac6b3c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a37fe640 1 -- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b3c980 msgr2=0x55bac6b39aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a37fe640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b3c980 0x55bac6b39aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a37fe640 1 -- 172.21.15.44:0/2072240750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b317c0 msgr2=0x55bac6b3c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a37fe640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b317c0 0x55bac6b3c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a37fe640 1 -- 172.21.15.44:0/2072240750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a41a5100 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96a37fe640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b39560 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f969800e790 tx=0x7f9698004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce6d2a2389bedd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96837fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96837fe640 1 -- 172.21.15.44:0/2072240750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800fb40 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96837fe640 1 -- 172.21.15.44:0/2072240750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f969800fca0 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bac6789590 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96837fe640 1 -- 172.21.15.44:0/2072240750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f969800fe60 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96ad26a740 10 client.16971 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.696+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bac6a94360 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96837fe640 1 -- 172.21.15.44:0/2072240750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bac6789590 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96837fe640 1 -- 172.21.15.44:0/2072240750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9698025020 con 0x55bac6b320e0 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96ad26a740 10 client.16971 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96ad26a740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96ad26a740 20 client.16971 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6b4c3f0 0x55bac6b4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96ad26a740 4 client.16971 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bac679a860 con 0x55bac6b4c3f0 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96a3fff640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6b4c3f0 0x55bac6b4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96837fe640 10 client.16971 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.697+0000 7f96a3fff640 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6b4c3f0 0x55bac6b4e810 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de40e3bd73531a8 server_cookie=90f08507799baae2 in_seq=0 out_seq=0 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96837fe640 1 -- 172.21.15.44:0/2072240750 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bac679a860 con 0x55bac6b4c3f0 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96837fe640 10 client.16971 handle_command_reply: tid=0 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 client.16971 shutdown 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 msgr2=0x55bac6b39560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b39560 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f969800e790 tx=0x7f9698004730 comp rx=0 tx=0).stop 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 shutdown_connections 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6b4c3f0 0x55bac6b4e810 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b3c980 0x55bac6b39aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b39560 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96837fe640 0 client.16971 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 --2- 172.21.15.44:0/2072240750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b317c0 0x55bac6b3c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 >> 172.21.15.44:0/2072240750 conn(0x55bac6b2dd50 msgr2=0x55bac6b331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 shutdown_connections 2024-09-13T17:31:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.698+0000 7f96ad26a740 1 -- 172.21.15.44:0/2072240750 wait complete. 2024-09-13T17:31:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.699+0000 7f96ad26a740 20 client.16971 trim_cache size 0 max 16384 2024-09-13T17:31:08.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 Processor -- start 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 -- start start 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b3bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b3c980 0x55bac6b43440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6789590 con 0x55bac6b320e0 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6b48ab0 con 0x55bac6b3c980 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96ad26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6b49120 con 0x55bac6b317c0 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b3bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b3bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45994/0 (socket says 172.21.15.44:45994) 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 -- 172.21.15.44:0/1934992167 learned_addr learned my addr 172.21.15.44:0/1934992167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a3fff640 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 -- 172.21.15.44:0/1934992167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 msgr2=0x55bac6b338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b3c980 0x55bac6b43440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 -- 172.21.15.44:0/1934992167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b3c980 msgr2=0x55bac6b43440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b3c980 0x55bac6b43440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a37fe640 1 -- 172.21.15.44:0/1934992167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bac6a361c0 con 0x55bac6b320e0 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.713+0000 7f96a3fff640 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b3bda0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f969800e760 tx=0x7f9698012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22edd36a4bef01c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f968affd640 1 -- 172.21.15.44:0/1934992167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969801cad0 con 0x55bac6b320e0 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f968affd640 1 -- 172.21.15.44:0/1934992167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969801c690 con 0x55bac6b320e0 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f968affd640 1 -- 172.21.15.44:0/1934992167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969801f650 con 0x55bac6b320e0 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 -- 172.21.15.44:0/1934992167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 msgr2=0x55bac6b3bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b3bda0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f969800e760 tx=0x7f9698012d10 comp rx=0 tx=0).stop 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 -- 172.21.15.44:0/1934992167 shutdown_connections 2024-09-13T17:31:08.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b3c980 0x55bac6b43440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b320e0 0x55bac6b3bda0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1934992167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 -- 172.21.15.44:0/1934992167 >> 172.21.15.44:0/1934992167 conn(0x55bac6b2dd50 msgr2=0x55bac6b331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 -- 172.21.15.44:0/1934992167 shutdown_connections 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.714+0000 7f96ad26a740 1 -- 172.21.15.44:0/1934992167 wait complete. 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 -- start start 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 Processor -- start 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b42c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6ba28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6ba2e00 con 0x55bac6b3c980 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6ba2f40 con 0x55bac6b320e0 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac6ba3080 con 0x55bac6b317c0 2024-09-13T17:31:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96ad26a740 10 client.0 _ll_register_callbacks cb 0x7f969800bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96a8a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6ba28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96a8a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6ba28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46000/0 (socket says 172.21.15.44:46000) 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96a8a08640 1 -- 172.21.15.44:0/1676030708 learned_addr learned my addr 172.21.15.44:0/1676030708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96a37fe640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b42c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.715+0000 7f96a3fff640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96a8a08640 1 -- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 msgr2=0x55bac6b426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96a8a08640 1 -- 172.21.15.44:0/1676030708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 msgr2=0x55bac6b42c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b42c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96a8a08640 1 -- 172.21.15.44:0/1676030708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bac6a94360 con 0x55bac6b3c980 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96a8a08640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6ba28c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f969c0bb1c0 tx=0x7f969c0d3e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff89cda7623964b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96837fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:08.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96837fe640 1 -- 172.21.15.44:0/1676030708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c02ecd0 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96837fe640 1 -- 172.21.15.44:0/1676030708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f969c0b5e40 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f969c0bb930 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96837fe640 1 -- 172.21.15.44:0/1676030708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f969c0c36b0 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96ad26a740 10 client.16977 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bac6a5b990 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96837fe640 1 -- 172.21.15.44:0/1676030708 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969c0bb930 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96837fe640 1 -- 172.21.15.44:0/1676030708 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f969c04b060 con 0x55bac6b3c980 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96ad26a740 10 client.16977 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96ad26a740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.716+0000 7f96ad26a740 20 client.16977 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.717+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6ba4390 0x55bac6ba6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.717+0000 7f96ad26a740 4 client.16977 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.717+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bac67edeb0 con 0x55bac6ba4390 2024-09-13T17:31:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.717+0000 7f96a3fff640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6ba4390 0x55bac6ba6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.717+0000 7f96837fe640 10 client.16977 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.717+0000 7f96a3fff640 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6ba4390 0x55bac6ba6780 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48412126b6cb9957 server_cookie=dda589ac47508535 in_seq=0 out_seq=0 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96837fe640 1 -- 172.21.15.44:0/1676030708 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bac67edeb0 con 0x55bac6ba4390 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96837fe640 10 client.16977 handle_command_reply: tid=0 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 client.16977 shutdown 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 msgr2=0x55bac6ba28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6ba28c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f969c0bb1c0 tx=0x7f969c0d3e60 comp rx=0 tx=0).stop 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 shutdown_connections 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bac6ba4390 0x55bac6ba6780 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bac6b3c980 0x55bac6ba28c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bac6b320e0 0x55bac6b42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 --2- 172.21.15.44:0/1676030708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bac6b317c0 0x55bac6b426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96837fe640 0 client.16977 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.718+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 >> 172.21.15.44:0/1676030708 conn(0x55bac6b2dd50 msgr2=0x55bac6b38d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 shutdown_connections 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96ad26a740 1 -- 172.21.15.44:0/1676030708 wait complete. 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96ad26a740 20 client.16977 trim_cache size 0 max 16384 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "a3ee3870-71f0-4b83-9cc1-213c739b86dd", 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:31:08.726 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f968c0a1280 msgr2=0x7f968c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f968c0a1280 0x7f968c0a3740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f96a4101250 tx=0x7f96980127c0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 msgr2=0x7f96a4100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a4100550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cd0 tx=0x7f969c02f950 comp rx=0 tx=0).stop 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 shutdown_connections 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f968c0a1280 0x7f968c0a3740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a4106420 0x7f96a4100550 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a40ff480 0x7f96a4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 --2- 172.21.15.44:0/1728990341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96a40feb60 0x7f96a40ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 >> 172.21.15.44:0/1728990341 conn(0x7f96a40741b0 msgr2=0x7f96a4103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 shutdown_connections 2024-09-13T17:31:08.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.719+0000 7f96aa46b640 1 -- 172.21.15.44:0/1728990341 wait complete. 2024-09-13T17:31:08.731 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:31:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 -- 172.21.15.44:0/2603436485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab4103fb0 msgr2=0x7faab410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 --2- 172.21.15.44:0/2603436485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab4103fb0 0x7faab410bb00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7faaac009930 tx=0x7faaac02a420 comp rx=0 tx=0).stop 2024-09-13T17:31:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 -- 172.21.15.44:0/2603436485 shutdown_connections 2024-09-13T17:31:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 --2- 172.21.15.44:0/2603436485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab4103fb0 0x7faab410bb00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 --2- 172.21.15.44:0/2603436485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab40ff810 0x7faab4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 --2- 172.21.15.44:0/2603436485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faab40fee60 0x7faab40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 -- 172.21.15.44:0/2603436485 >> 172.21.15.44:0/2603436485 conn(0x7faab4075c60 msgr2=0x7faab40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 -- 172.21.15.44:0/2603436485 shutdown_connections 2024-09-13T17:31:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 -- 172.21.15.44:0/2603436485 wait complete. 2024-09-13T17:31:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.836+0000 7faab8e41640 1 Processor -- start 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 -- start start 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 0x7faab41962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4103fb0 0x7faab4196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faab419ab00 0x7faab419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab419b4c0 con 0x7faab40fee60 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab419b600 con 0x7faab4103fb0 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab8e41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab419b7c0 con 0x7faab419ab00 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 0x7faab41962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 0x7faab41962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46030/0 (socket says 172.21.15.44:46030) 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 -- 172.21.15.44:0/1184378355 learned_addr learned my addr 172.21.15.44:0/1184378355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab2d9d640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faab419ab00 0x7faab419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab1d9b640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4103fb0 0x7faab4196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 -- 172.21.15.44:0/1184378355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faab419ab00 msgr2=0x7faab419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faab419ab00 0x7faab419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 -- 172.21.15.44:0/1184378355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4103fb0 msgr2=0x7faab4196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4103fb0 0x7faab4196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 -- 172.21.15.44:0/1184378355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac009560 con 0x7faab40fee60 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.837+0000 7faab259c640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 0x7faab41962e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7faaa400d770 tx=0x7faaa400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.838+0000 7faaa37fe640 1 -- 172.21.15.44:0/1184378355 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4009c30 con 0x7faab40fee60 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.838+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab41a2dd0 con 0x7faab40fee60 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.838+0000 7faaa37fe640 1 -- 172.21.15.44:0/1184378355 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa400de80 con 0x7faab40fee60 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.838+0000 7faaa37fe640 1 -- 172.21.15.44:0/1184378355 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4004d40 con 0x7faab40fee60 2024-09-13T17:31:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.838+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab41a31d0 con 0x7faab40fee60 2024-09-13T17:31:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faaa37fe640 1 -- 172.21.15.44:0/1184378355 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faaa4004ea0 con 0x7faab40fee60 2024-09-13T17:31:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 Processor -- start 2024-09-13T17:31:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 -- start start 2024-09-13T17:31:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c27340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c2742ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c269ef20 con 0x5599c2734610 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c269c6b0 con 0x5599c2733cf0 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.840+0000 7faabbc47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2697fa0 con 0x5599c27433e0 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c2742ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c2742ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46040/0 (socket says 172.21.15.44:46040) 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 -- 172.21.15.44:0/2902221848 learned_addr learned my addr 172.21.15.44:0/2902221848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab1d9b640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab2d9d640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c27340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faaa37fe640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faa9c0a1200 0x7faa9c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 -- 172.21.15.44:0/2902221848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 msgr2=0x5599c27457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 -- 172.21.15.44:0/2902221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 msgr2=0x5599c27340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c27340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 -- 172.21.15.44:0/2902221848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c2652840 con 0x5599c2734610 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab1d9b640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab2d9d640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c27340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab2d9d640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faa9c0a1200 0x7faa9c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faaa37fe640 1 -- 172.21.15.44:0/1184378355 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7faaa40cca60 con 0x7faab40fee60 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faab259c640 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c2742ea0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faaa40087f0 tx=0x7faaa4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d51336c66ec0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.841+0000 7faaa0ff9640 1 -- 172.21.15.44:0/2902221848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40995c0 con 0x5599c2734610 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faaa0ff9640 1 -- 172.21.15.44:0/2902221848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa4099330 con 0x5599c2734610 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faab2d9d640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faa9c0a1200 0x7faa9c0a36c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faaac006010 tx=0x7faaac009260 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faaa0ff9640 1 -- 172.21.15.44:0/2902221848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40997f0 con 0x5599c2734610 2024-09-13T17:31:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 -- 172.21.15.44:0/2902221848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 msgr2=0x5599c2742ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c2742ea0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faaa40087f0 tx=0x7faaa4022000 comp rx=0 tx=0).stop 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 -- 172.21.15.44:0/2902221848 shutdown_connections 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c2742ea0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 --2- 172.21.15.44:0/2902221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c27340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 -- 172.21.15.44:0/2902221848 >> 172.21.15.44:0/2902221848 conn(0x5599c2730380 msgr2=0x5599c2730760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 -- 172.21.15.44:0/2902221848 shutdown_connections 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.842+0000 7faabbc47740 1 -- 172.21.15.44:0/2902221848 wait complete. 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 -- start start 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 Processor -- start 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c273ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c26392a0 con 0x5599c2733cf0 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2370430 con 0x5599c2734610 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c261b680 con 0x5599c27433e0 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faabbc47740 10 client.0 _ll_register_callbacks cb 0x5599c2691610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46050/0 (socket says 172.21.15.44:46050) 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 -- 172.21.15.44:0/537445352 learned_addr learned my addr 172.21.15.44:0/537445352 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab1d9b640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c273ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab2d9d640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 -- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 msgr2=0x5599c27404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 -- 172.21.15.44:0/537445352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 msgr2=0x5599c273ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c273ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.843+0000 7faab259c640 1 -- 172.21.15.44:0/537445352 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab41a31d0 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faab259c640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742d30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7faaa40994a0 tx=0x7faaa40991c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4067ebb3cba8450c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faa99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faa99ffb640 1 -- 172.21.15.44:0/537445352 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40087f0 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faa99ffb640 1 -- 172.21.15.44:0/537445352 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7faaa409d620 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c238b4e0 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faa99ffb640 1 -- 172.21.15.44:0/537445352 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa40a1c70 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 10 client.16989 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599c2696620 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faa99ffb640 1 -- 172.21.15.44:0/537445352 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599c238b4e0 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faa99ffb640 1 -- 172.21.15.44:0/537445352 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faaa40955e0 con 0x5599c2733cf0 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 10 client.16989 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 20 client.16989 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c2752e50 0x5599c2755270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 4 client.16989 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.844+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c239c7e0 con 0x5599c2752e50 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.845+0000 7faab1d9b640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c2752e50 0x5599c2755270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.845+0000 7faab1d9b640 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c2752e50 0x5599c2755270 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0e5552efd1ea57 server_cookie=5a8d56ea0056411f in_seq=0 out_seq=0 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.845+0000 7faa99ffb640 10 client.16989 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faa99ffb640 1 -- 172.21.15.44:0/537445352 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5599c239c7e0 con 0x5599c2752e50 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faa99ffb640 10 client.16989 handle_command_reply: tid=0 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 client.16989 shutdown 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 msgr2=0x5599c2742d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742d30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7faaa40994a0 tx=0x7faaa40991c0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 shutdown_connections 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c2752e50 0x5599c2755270 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c27404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c273ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 --2- 172.21.15.44:0/537445352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742d30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faa99ffb640 0 client.16989 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 >> 172.21.15.44:0/537445352 conn(0x5599c2730380 msgr2=0x5599c27360e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 shutdown_connections 2024-09-13T17:31:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 1 -- 172.21.15.44:0/537445352 wait complete. 2024-09-13T17:31:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.846+0000 7faabbc47740 20 client.16989 trim_cache size 0 max 16384 2024-09-13T17:31:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 Processor -- start 2024-09-13T17:31:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 -- start start 2024-09-13T17:31:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c2730760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c2730ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c274f0b0 con 0x5599c2733cf0 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c275a4b0 con 0x5599c2734610 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faabbc47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c274f7f0 con 0x5599c27433e0 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faab259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46052/0 (socket says 172.21.15.44:46052) 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.860+0000 7faab259c640 1 -- 172.21.15.44:0/3970638006 learned_addr learned my addr 172.21.15.44:0/3970638006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab1d9b640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c2730760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab259c640 1 -- 172.21.15.44:0/3970638006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 msgr2=0x5599c2730ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab2d9d640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c2730ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab259c640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c2730ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab259c640 1 -- 172.21.15.44:0/3970638006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 msgr2=0x5599c2730760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab259c640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c2730760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab259c640 1 -- 172.21.15.44:0/3970638006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c2638790 con 0x5599c2733cf0 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab1d9b640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c2730760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab2d9d640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c2730ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faab259c640 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742840 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faaa40ccde0 tx=0x7faaa4009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c79d0e1711c2152 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faaa17fa640 1 -- 172.21.15.44:0/3970638006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4095f00 con 0x5599c2733cf0 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faaa17fa640 1 -- 172.21.15.44:0/3970638006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa409d620 con 0x5599c2733cf0 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.861+0000 7faaa17fa640 1 -- 172.21.15.44:0/3970638006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4009d90 con 0x5599c2733cf0 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- 172.21.15.44:0/3970638006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 msgr2=0x5599c2742840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742840 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faaa40ccde0 tx=0x7faaa4009740 comp rx=0 tx=0).stop 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- 172.21.15.44:0/3970638006 shutdown_connections 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c2730ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2734610 0x5599c2730760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- 172.21.15.44:0/3970638006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2733cf0 0x5599c2742840 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- 172.21.15.44:0/3970638006 >> 172.21.15.44:0/3970638006 conn(0x5599c2730380 msgr2=0x5599c273b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- 172.21.15.44:0/3970638006 shutdown_connections 2024-09-13T17:31:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- 172.21.15.44:0/3970638006 wait complete. 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- start start 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 Processor -- start 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c274a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c273d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c273da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c273c1b0 con 0x5599c2734610 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c273c320 con 0x5599c2733cf0 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c273c4e0 con 0x5599c27433e0 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faabbc47740 10 client.0 _ll_register_callbacks cb 0x7faaa4095bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faab1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c273d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faab1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c273d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46056/0 (socket says 172.21.15.44:46056) 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.862+0000 7faab1d9b640 1 -- 172.21.15.44:0/1142065593 learned_addr learned my addr 172.21.15.44:0/1142065593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab2d9d640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c273da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab259c640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c274a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab1d9b640 1 -- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 msgr2=0x5599c273da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab1d9b640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c273da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab1d9b640 1 -- 172.21.15.44:0/1142065593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 msgr2=0x5599c274a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab1d9b640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c274a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab1d9b640 1 -- 172.21.15.44:0/1142065593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599c2696620 con 0x5599c2734610 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faab1d9b640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c273d2a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faaa8003a80 tx=0x7faaa80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6acadee8aef1131 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faa99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faa99ffb640 1 -- 172.21.15.44:0/1142065593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8017040 con 0x5599c2734610 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faa99ffb640 1 -- 172.21.15.44:0/1142065593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7faaa8026420 con 0x5599c2734610 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faaa4099330 con 0x5599c2734610 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faa99ffb640 1 -- 172.21.15.44:0/1142065593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa80265e0 con 0x5599c2734610 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faabbc47740 10 client.16995 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599c265dea0 con 0x5599c2734610 2024-09-13T17:31:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.863+0000 7faa99ffb640 1 -- 172.21.15.44:0/1142065593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa4099330 con 0x5599c2734610 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faa99ffb640 1 -- 172.21.15.44:0/1142065593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faaa8011710 con 0x5599c2734610 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 10 client.16995 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 20 client.16995 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c27a9d50 0x5599c27ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 4 client.16995 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599c23efe30 con 0x5599c27a9d50 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faab259c640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c27a9d50 0x5599c27ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faa99ffb640 10 client.16995 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faab259c640 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c27a9d50 0x5599c27ac170 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8973de9f866c5124 server_cookie=485319f06e4ce1ce in_seq=0 out_seq=0 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faa99ffb640 1 -- 172.21.15.44:0/1142065593 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5599c23efe30 con 0x5599c27a9d50 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faa99ffb640 10 client.16995 handle_command_reply: tid=0 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.864+0000 7faabbc47740 1 client.16995 shutdown 2024-09-13T17:31:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 msgr2=0x5599c273d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c273d2a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faaa8003a80 tx=0x7faaa80128e0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 shutdown_connections 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599c27a9d50 0x5599c27ac170 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599c27433e0 0x5599c273da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faa99ffb640 0 client.16995 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599c2734610 0x5599c273d2a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 --2- 172.21.15.44:0/1142065593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599c2733cf0 0x5599c274a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 >> 172.21.15.44:0/1142065593 conn(0x5599c2730380 msgr2=0x5599c273b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 shutdown_connections 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 1 -- 172.21.15.44:0/1142065593 wait complete. 2024-09-13T17:31:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faabbc47740 20 client.16995 trim_cache size 0 max 16384 2024-09-13T17:31:08.873 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:31:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:31:08.873 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faa9c0a1200 msgr2=0x7faa9c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faa9c0a1200 0x7faa9c0a36c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faaac006010 tx=0x7faaac009260 comp rx=0 tx=0).stop 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 msgr2=0x7faab41962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 0x7faab41962e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7faaa400d770 tx=0x7faaa400dc30 comp rx=0 tx=0).stop 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 shutdown_connections 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faa9c0a1200 0x7faa9c0a36c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faab419ab00 0x7faab419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4103fb0 0x7faab4196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 --2- 172.21.15.44:0/1184378355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faab40fee60 0x7faab41962e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 >> 172.21.15.44:0/1184378355 conn(0x7faab4075c60 msgr2=0x7faab4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 shutdown_connections 2024-09-13T17:31:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.865+0000 7faab8e41640 1 -- 172.21.15.44:0/1184378355 wait complete. 2024-09-13T17:31:08.877 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:31:08.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 -- 172.21.15.44:0/1285167311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16940feeb0 msgr2=0x7f16940ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 --2- 172.21.15.44:0/1285167311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16940feeb0 0x7f16940ff290 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1684009930 tx=0x7f168402a420 comp rx=0 tx=0).stop 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 -- 172.21.15.44:0/1285167311 shutdown_connections 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 --2- 172.21.15.44:0/1285167311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16940ff860 0x7f16941038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 --2- 172.21.15.44:0/1285167311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1694104000 0x7f169410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 --2- 172.21.15.44:0/1285167311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16940feeb0 0x7f16940ff290 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 -- 172.21.15.44:0/1285167311 >> 172.21.15.44:0/1285167311 conn(0x7f16940fa900 msgr2=0x7f16940fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 -- 172.21.15.44:0/1285167311 shutdown_connections 2024-09-13T17:31:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.978+0000 7f1698f25640 1 -- 172.21.15.44:0/1285167311 wait complete. 2024-09-13T17:31:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 Processor -- start 2024-09-13T17:31:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 -- start start 2024-09-13T17:31:08.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16940feeb0 0x7f1694196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 0x7f1694197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1694197a60 0x7f16941a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941a2700 con 0x7f1694104000 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941a2870 con 0x7f16940feeb0 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1698f25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941a2a30 con 0x7f1694197a60 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1691d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 0x7f1694197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1691d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 0x7f1694197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46064/0 (socket says 172.21.15.44:46064) 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1691d9b640 1 -- 172.21.15.44:0/1173870943 learned_addr learned my addr 172.21.15.44:0/1173870943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.979+0000 7f1692d9d640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1694197a60 0x7f16941a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f169259c640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16940feeb0 0x7f1694196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1691d9b640 1 -- 172.21.15.44:0/1173870943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1694197a60 msgr2=0x7f16941a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1691d9b640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1694197a60 0x7f16941a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1691d9b640 1 -- 172.21.15.44:0/1173870943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16940feeb0 msgr2=0x7f1694196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1691d9b640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16940feeb0 0x7f1694196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1691d9b640 1 -- 172.21.15.44:0/1173870943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009560 con 0x7f1694104000 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f169259c640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16940feeb0 0x7f1694196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1691d9b640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 0x7f1694197520 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f168800d770 tx=0x7f168800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688009c30 con 0x7f1694104000 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f168800de80 con 0x7f1694104000 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16941a2d10 con 0x7f1694104000 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800f620 con 0x7f1694104000 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.980+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16941a31d0 con 0x7f1694104000 2024-09-13T17:31:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.981+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1694100380 con 0x7f1694104000 2024-09-13T17:31:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.983+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1688010460 con 0x7f1694104000 2024-09-13T17:31:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.983+0000 7f16837fe640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f167c0a1200 0x7f167c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.983+0000 7f169259c640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f167c0a1200 0x7f167c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.983+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f16880ccdd0 con 0x7f1694104000 2024-09-13T17:31:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.984+0000 7f169259c640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f167c0a1200 0x7f167c0a36c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1684006010 tx=0x7f1684005a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:08.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:08.986+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1688095f00 con 0x7f1694104000 2024-09-13T17:31:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.191+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1694107250 con 0x7f1694104000 2024-09-13T17:31:09.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.192+0000 7f16837fe640 1 -- 172.21.15.44:0/1173870943 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f1688099520 con 0x7f1694104000 2024-09-13T17:31:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:31:09.193 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:31:09.194 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:31:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f167c0a1200 msgr2=0x7f167c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f167c0a1200 0x7f167c0a36c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1684006010 tx=0x7f1684005a40 comp rx=0 tx=0).stop 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 msgr2=0x7f1694197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 0x7f1694197520 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f168800d770 tx=0x7f168800dc30 comp rx=0 tx=0).stop 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 shutdown_connections 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f167c0a1200 0x7f167c0a36c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1694197a60 0x7f16941a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1694104000 0x7f1694197520 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 --2- 172.21.15.44:0/1173870943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16940feeb0 0x7f1694196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 >> 172.21.15.44:0/1173870943 conn(0x7f16940fa900 msgr2=0x7f16940fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 shutdown_connections 2024-09-13T17:31:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.194+0000 7f1698f25640 1 -- 172.21.15.44:0/1173870943 wait complete. 2024-09-13T17:31:09.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:31:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 -- 172.21.15.44:0/595310336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c101860 msgr2=0x7f063c109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 --2- 172.21.15.44:0/595310336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c101860 0x7f063c109d40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0624009900 tx=0x7f062402a410 comp rx=0 tx=0).stop 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 -- 172.21.15.44:0/595310336 shutdown_connections 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 --2- 172.21.15.44:0/595310336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f063c10a310 0x7f063c10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 --2- 172.21.15.44:0/595310336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c101860 0x7f063c109d40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 --2- 172.21.15.44:0/595310336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063c100f40 0x7f063c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 -- 172.21.15.44:0/595310336 >> 172.21.15.44:0/595310336 conn(0x7f063c0fa930 msgr2=0x7f063c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 -- 172.21.15.44:0/595310336 shutdown_connections 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.310+0000 7f0642783640 1 -- 172.21.15.44:0/595310336 wait complete. 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 Processor -- start 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 -- start start 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 0x7f063c196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063c101860 0x7f063c1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f063c10a310 0x7f063c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c19b0b0 con 0x7f063c100f40 2024-09-13T17:31:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c19b220 con 0x7f063c10a310 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0642783640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c19b3e0 con 0x7f063c101860 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f063bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 0x7f063c196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f063bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 0x7f063c196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46092/0 (socket says 172.21.15.44:46092) 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f063bfff640 1 -- 172.21.15.44:0/1419140911 learned_addr learned my addr 172.21.15.44:0/1419140911 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0633fff640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063c101860 0x7f063c1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.311+0000 7f0640d20640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f063c10a310 0x7f063c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f063bfff640 1 -- 172.21.15.44:0/1419140911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063c101860 msgr2=0x7f063c1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f063bfff640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063c101860 0x7f063c1966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f063bfff640 1 -- 172.21.15.44:0/1419140911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f063c10a310 msgr2=0x7f063c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f063bfff640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f063c10a310 0x7f063c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f063bfff640 1 -- 172.21.15.44:0/1419140911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624009560 con 0x7f063c100f40 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f063bfff640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 0x7f063c196180 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c920 tx=0x7f062c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f0639ffb640 1 -- 172.21.15.44:0/1419140911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c007c70 con 0x7f063c100f40 2024-09-13T17:31:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c19b6c0 con 0x7f063c100f40 2024-09-13T17:31:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f0639ffb640 1 -- 172.21.15.44:0/1419140911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f062c007dd0 con 0x7f063c100f40 2024-09-13T17:31:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f0639ffb640 1 -- 172.21.15.44:0/1419140911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c018730 con 0x7f063c100f40 2024-09-13T17:31:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.312+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c1a2f40 con 0x7f063c100f40 2024-09-13T17:31:09.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.314+0000 7f0639ffb640 1 -- 172.21.15.44:0/1419140911 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f062c00f450 con 0x7f063c100f40 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.314+0000 7f06454aa740 1 Processor -- start 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 -- start start 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e50ecc0 con 0x55a00e5a4790 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e50c450 con 0x55a00e5a50b0 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f06454aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e507d40 con 0x55a00e5af950 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46102/0 (socket says 172.21.15.44:46102) 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 -- 172.21.15.44:0/3002866842 learned_addr learned my addr 172.21.15.44:0/3002866842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0633fff640 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f063bfff640 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0639ffb640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06280a1280 0x7f06280a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 -- 172.21.15.44:0/3002866842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 msgr2=0x55a00e5b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 -- 172.21.15.44:0/3002866842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 msgr2=0x55a00e5af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 -- 172.21.15.44:0/3002866842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a00e4cf620 con 0x55a00e5a4790 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0639ffb640 1 -- 172.21.15.44:0/1419140911 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f062c0cd210 con 0x7f063c100f40 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.315+0000 7f0640d20640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06280a1280 0x7f06280a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.316+0000 7f0640d20640 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5a4b70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0634007dd0 tx=0x7f063400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5ee9524ed56b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.316+0000 7f06327fc640 1 -- 172.21.15.44:0/3002866842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634010b30 con 0x55a00e5a4790 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.316+0000 7f06327fc640 1 -- 172.21.15.44:0/3002866842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f063401ae30 con 0x55a00e5a4790 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.316+0000 7f06327fc640 1 -- 172.21.15.44:0/3002866842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400fe60 con 0x55a00e5a4790 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f0640d20640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06280a1280 0x7f06280a3740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f063401d000 tx=0x7f0634020000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.316+0000 7f06454aa740 1 -- 172.21.15.44:0/3002866842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 msgr2=0x55a00e5a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5a4b70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0634007dd0 tx=0x7f063400e720 comp rx=0 tx=0).stop 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 -- 172.21.15.44:0/3002866842 shutdown_connections 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- 172.21.15.44:0/3002866842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5a4b70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 -- 172.21.15.44:0/3002866842 >> 172.21.15.44:0/3002866842 conn(0x55a00e5a0d00 msgr2=0x55a00e5a10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 -- 172.21.15.44:0/3002866842 shutdown_connections 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 -- 172.21.15.44:0/3002866842 wait complete. 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 -- start start 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 Processor -- start 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.317+0000 7f06454aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e4c6180 con 0x55a00e5a4790 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e1e0430 con 0x55a00e5a50b0 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e48b5e0 con 0x55a00e5af950 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 10 client.0 _ll_register_callbacks cb 0x55a00e4f7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46104/0 (socket says 172.21.15.44:46104) 2024-09-13T17:31:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 -- 172.21.15.44:0/2414974220 learned_addr learned my addr 172.21.15.44:0/2414974220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f0633fff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f0640d20640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 -- 172.21.15.44:0/2414974220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 msgr2=0x55a00e5ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5ac400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 -- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 msgr2=0x55a00e5af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 -- 172.21.15.44:0/2414974220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634007a00 con 0x55a00e5a4790 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f0640d20640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f0633fff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f063bfff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5aee90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f062c007590 tx=0x7f062c099e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b6dfb5a359463d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f061b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f061b7fe640 1 -- 172.21.15.44:0/2414974220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0caa60 con 0x55a00e5a4790 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f061b7fe640 1 -- 172.21.15.44:0/2414974220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f062c00de20 con 0x55a00e5a4790 2024-09-13T17:31:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a00e531b80 con 0x55a00e5a4790 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f061b7fe640 1 -- 172.21.15.44:0/2414974220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c0d08d0 con 0x55a00e5a4790 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 10 client.17013 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.318+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a00e5063c0 con 0x55a00e5a4790 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f061b7fe640 1 -- 172.21.15.44:0/2414974220 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a00e531b80 con 0x55a00e5a4790 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f061b7fe640 1 -- 172.21.15.44:0/2414974220 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f062c016020 con 0x55a00e5a4790 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f06454aa740 10 client.17013 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f06454aa740 10 client.17013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f06454aa740 20 client.17013 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f06454aa740 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e5bf210 0x55a00e5c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f06454aa740 4 client.17013 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a00e4fa1f0 con 0x55a00e5bf210 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f0633fff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e5bf210 0x55a00e5c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f0633fff640 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e5bf210 0x55a00e5c1600 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c415a61642429dfb server_cookie=7cd9841de57d52fb in_seq=0 out_seq=0 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.319+0000 7f061b7fe640 10 client.17013 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f061b7fe640 1 -- 172.21.15.44:0/2414974220 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a00e4fa1f0 con 0x55a00e5bf210 2024-09-13T17:31:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f061b7fe640 10 client.17013 handle_command_reply: tid=0 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 client.17013 shutdown 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 msgr2=0x55a00e5aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5aee90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f062c007590 tx=0x7f062c099e30 comp rx=0 tx=0).stop 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 shutdown_connections 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e5bf210 0x55a00e5c1600 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e5ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f061b7fe640 0 client.17013 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 --2- 172.21.15.44:0/2414974220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5aee90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 >> 172.21.15.44:0/2414974220 conn(0x55a00e5a0d00 msgr2=0x55a00e5a6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 shutdown_connections 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 1 -- 172.21.15.44:0/2414974220 wait complete. 2024-09-13T17:31:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.320+0000 7f06454aa740 20 client.17013 trim_cache size 0 max 16384 2024-09-13T17:31:09.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.331+0000 7f06454aa740 1 Processor -- start 2024-09-13T17:31:09.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.331+0000 7f06454aa740 1 -- start start 2024-09-13T17:31:09.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f06454aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5a4790 0x55a00e5b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f06454aa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f06454aa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 0x55a00e5a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f06454aa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e5a4630 con 0x55a00e5af950 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f06454aa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e5ad1c0 con 0x55a00e5a50b0 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f06454aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e4fa1f0 con 0x55a00e5a4790 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 0x55a00e5a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 0x55a00e5a7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46120/0 (socket says 172.21.15.44:46120) 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 -- 172.21.15.44:0/133502881 learned_addr learned my addr 172.21.15.44:0/133502881 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0633fff640 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f063bfff640 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5a4790 0x55a00e5b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 -- 172.21.15.44:0/133502881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5a4790 msgr2=0x55a00e5b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5a4790 0x55a00e5b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 -- 172.21.15.44:0/133502881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 msgr2=0x55a00e5b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 -- 172.21.15.44:0/133502881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a00e4a8770 con 0x55a00e5af950 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f063bfff640 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5a4790 0x55a00e5b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0640d20640 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 0x55a00e5a7250 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0634007d00 tx=0x7f0634023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48c5cad04031dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0632ffd640 1 -- 172.21.15.44:0/133502881 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063402b070 con 0x55a00e5af950 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0632ffd640 1 -- 172.21.15.44:0/133502881 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0634003e80 con 0x55a00e5af950 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.332+0000 7f0632ffd640 1 -- 172.21.15.44:0/133502881 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340042a0 con 0x55a00e5af950 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 -- 172.21.15.44:0/133502881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 msgr2=0x55a00e5a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 0x55a00e5a7250 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0634007d00 tx=0x7f0634023000 comp rx=0 tx=0).stop 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 -- 172.21.15.44:0/133502881 shutdown_connections 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5af950 0x55a00e5a7250 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 --2- 172.21.15.44:0/133502881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5a4790 0x55a00e5b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 -- 172.21.15.44:0/133502881 >> 172.21.15.44:0/133502881 conn(0x55a00e5a0d00 msgr2=0x55a00e59f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 -- 172.21.15.44:0/133502881 shutdown_connections 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 -- 172.21.15.44:0/133502881 wait complete. 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 -- start start 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.333+0000 7f06454aa740 1 Processor -- start 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e615700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e5b63f0 con 0x55a00e5a4790 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e615c40 con 0x55a00e5a50b0 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e615d80 con 0x55a00e5af950 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 10 client.0 _ll_register_callbacks cb 0x7f0634007560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46136/0 (socket says 172.21.15.44:46136) 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 -- 172.21.15.44:0/4081850026 learned_addr learned my addr 172.21.15.44:0/4081850026 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f0640d20640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e615700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f0633fff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 -- 172.21.15.44:0/4081850026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 msgr2=0x55a00e615700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e615700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 -- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 msgr2=0x55a00e5b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 -- 172.21.15.44:0/4081850026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a00e4cf620 con 0x55a00e5a4790 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f0640d20640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e615700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f0633fff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f063bfff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5b5560 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f062c096a10 tx=0x7f062c0cab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee4d77e486ed711 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f061b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f061b7fe640 1 -- 172.21.15.44:0/4081850026 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0d2ec0 con 0x55a00e5a4790 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f061b7fe640 1 -- 172.21.15.44:0/4081850026 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f062c00de20 con 0x55a00e5a4790 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a00e1fb4e0 con 0x55a00e5a4790 2024-09-13T17:31:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.334+0000 7f061b7fe640 1 -- 172.21.15.44:0/4081850026 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c099900 con 0x55a00e5a4790 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 10 client.17019 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a00e4cde80 con 0x55a00e5a4790 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f061b7fe640 1 -- 172.21.15.44:0/4081850026 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a00e1fb4e0 con 0x55a00e5a4790 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f061b7fe640 1 -- 172.21.15.44:0/4081850026 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f062c016070 con 0x55a00e5a4790 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 10 client.17019 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 10 client.17019 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 20 client.17019 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e616f40 0x55a00e619330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 4 client.17019 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a00e20c7e0 con 0x55a00e616f40 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f0633fff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e616f40 0x55a00e619330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f0633fff640 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e616f40 0x55a00e619330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b36a8d2fe30f251a server_cookie=bab8a032f35b3476 in_seq=0 out_seq=0 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.335+0000 7f061b7fe640 10 client.17019 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f061b7fe640 1 -- 172.21.15.44:0/4081850026 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a00e20c7e0 con 0x55a00e616f40 2024-09-13T17:31:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f061b7fe640 10 client.17019 handle_command_reply: tid=0 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 client.17019 shutdown 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 msgr2=0x55a00e5b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5b5560 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f062c096a10 tx=0x7f062c0cab10 comp rx=0 tx=0).stop 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 shutdown_connections 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55a00e616f40 0x55a00e619330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a00e5af950 0x55a00e615700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a00e5a50b0 0x55a00e5b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f061b7fe640 0 client.17019 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 --2- 172.21.15.44:0/4081850026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00e5a4790 0x55a00e5b5560 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 >> 172.21.15.44:0/4081850026 conn(0x55a00e5a0d00 msgr2=0x55a00e59feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 shutdown_connections 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 1 -- 172.21.15.44:0/4081850026 wait complete. 2024-09-13T17:31:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f06454aa740 20 client.17019 trim_cache size 0 max 16384 2024-09-13T17:31:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:09.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06280a1280 msgr2=0x7f06280a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f0642783640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06280a1280 0x7f06280a3740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f063401d000 tx=0x7f0634020000 comp rx=0 tx=0).stop 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 msgr2=0x7f063c196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.336+0000 7f0642783640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 0x7f063c196180 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c920 tx=0x7f062c00cde0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 shutdown_connections 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f06280a1280 0x7f06280a3740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063c101860 0x7f063c1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f063c10a310 0x7f063c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 --2- 172.21.15.44:0/1419140911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c100f40 0x7f063c196180 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 >> 172.21.15.44:0/1419140911 conn(0x7f063c0fa930 msgr2=0x7f063c0fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 shutdown_connections 2024-09-13T17:31:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.337+0000 7f0642783640 1 -- 172.21.15.44:0/1419140911 wait complete. 2024-09-13T17:31:09.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:31:09.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 -- 172.21.15.44:0/504931220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 msgr2=0x7f28ec109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 --2- 172.21.15.44:0/504931220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec109ce0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009900 tx=0x7f28e002a480 comp rx=0 tx=0).stop 2024-09-13T17:31:09.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 -- 172.21.15.44:0/504931220 shutdown_connections 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 --2- 172.21.15.44:0/504931220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 0x7f28ec10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 --2- 172.21.15.44:0/504931220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec109ce0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 --2- 172.21.15.44:0/504931220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec100ee0 0x7f28ec1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 -- 172.21.15.44:0/504931220 >> 172.21.15.44:0/504931220 conn(0x7f28ec0fa8d0 msgr2=0x7f28ec0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 -- 172.21.15.44:0/504931220 shutdown_connections 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.453+0000 7f28f2308640 1 -- 172.21.15.44:0/504931220 wait complete. 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 Processor -- start 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 -- start start 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec100ee0 0x7f28ec1961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 0x7f28ec19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec19b0d0 con 0x7f28ec101800 2024-09-13T17:31:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec19b240 con 0x7f28ec10a330 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28f2308640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec19b400 con 0x7f28ec100ee0 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec1966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55754/0 (socket says 172.21.15.44:55754) 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28eb7fe640 1 -- 172.21.15.44:0/2950606686 learned_addr learned my addr 172.21.15.44:0/2950606686 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.454+0000 7f28ebfff640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec100ee0 0x7f28ec1961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 0x7f28ec19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28eb7fe640 1 -- 172.21.15.44:0/2950606686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec100ee0 msgr2=0x7f28ec1961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec100ee0 0x7f28ec1961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28eb7fe640 1 -- 172.21.15.44:0/2950606686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 msgr2=0x7f28ec19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 0x7f28ec19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28eb7fe640 1 -- 172.21.15.44:0/2950606686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009560 con 0x7f28ec101800 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 0x7f28ec19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec1966e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f28e00057c0 tx=0x7f28e0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28e97fa640 1 -- 172.21.15.44:0/2950606686 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00044d0 con 0x7f28ec101800 2024-09-13T17:31:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28e97fa640 1 -- 172.21.15.44:0/2950606686 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28e0032e10 con 0x7f28ec101800 2024-09-13T17:31:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec19b680 con 0x7f28ec101800 2024-09-13T17:31:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28e97fa640 1 -- 172.21.15.44:0/2950606686 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0003850 con 0x7f28ec101800 2024-09-13T17:31:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.455+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec19b950 con 0x7f28ec101800 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28e97fa640 1 -- 172.21.15.44:0/2950606686 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f28e0004850 con 0x7f28ec101800 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 Processor -- start 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 -- start start 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248d790 0x55eac248db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248e0b0 0x55eac2498410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac249ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac23f7cc0 con 0x55eac2498950 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac23f5450 con 0x55eac248e0b0 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.458+0000 7f28f502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac23f0d40 con 0x55eac248d790 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac249ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac249ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55770/0 (socket says 172.21.15.44:55770) 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 -- 172.21.15.44:0/1238081911 learned_addr learned my addr 172.21.15.44:0/1238081911 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28f08a5640 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248d790 0x55eac248db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28ebfff640 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248e0b0 0x55eac2498410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28e97fa640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28d40a1200 0x7f28d40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 -- 172.21.15.44:0/1238081911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248d790 msgr2=0x55eac248db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248d790 0x55eac248db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 -- 172.21.15.44:0/1238081911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248e0b0 msgr2=0x55eac2498410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248e0b0 0x55eac2498410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 -- 172.21.15.44:0/1238081911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eac23b8620 con 0x55eac2498950 2024-09-13T17:31:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28f08a5640 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248d790 0x55eac248db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28d40a1200 0x7f28d40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28e97fa640 1 -- 172.21.15.44:0/2950606686 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f28e00edee0 con 0x7f28ec101800 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac249ad40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f28e00eb280 tx=0x7f28e0086210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2fa6c43e1f2d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28d27fc640 1 -- 172.21.15.44:0/1238081911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00046c0 con 0x55eac2498950 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28d27fc640 1 -- 172.21.15.44:0/1238081911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28e0004e90 con 0x55eac2498950 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28d40a1200 0x7f28d40a36c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28e4005fd0 tx=0x7f28e4004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.459+0000 7f28d27fc640 1 -- 172.21.15.44:0/1238081911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e008d060 con 0x55eac2498950 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 -- 172.21.15.44:0/1238081911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 msgr2=0x55eac249ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac249ad40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f28e00eb280 tx=0x7f28e0086210 comp rx=0 tx=0).stop 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 -- 172.21.15.44:0/1238081911 shutdown_connections 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac249ad40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248e0b0 0x55eac2498410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 --2- 172.21.15.44:0/1238081911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248d790 0x55eac248db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 -- 172.21.15.44:0/1238081911 >> 172.21.15.44:0/1238081911 conn(0x55eac2489d00 msgr2=0x55eac248a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 -- 172.21.15.44:0/1238081911 shutdown_connections 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 -- 172.21.15.44:0/1238081911 wait complete. 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.460+0000 7f28f502a740 1 -- start start 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 Processor -- start 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac249c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac249c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac23af180 con 0x55eac2498950 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac20c9430 con 0x55eac248d790 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac23745e0 con 0x55eac248e0b0 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f502a740 10 client.0 _ll_register_callbacks cb 0x55eac23e0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55772/0 (socket says 172.21.15.44:55772) 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 -- 172.21.15.44:0/2613156332 learned_addr learned my addr 172.21.15.44:0/2613156332 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac249c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28ebfff640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac249c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 -- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 msgr2=0x55eac249c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac249c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 -- 172.21.15.44:0/2613156332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 msgr2=0x55eac249c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac249c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.461+0000 7f28f08a5640 1 -- 172.21.15.44:0/2613156332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec19b950 con 0x55eac2498950 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f08a5640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a08f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f28e4012970 tx=0x7f28e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b908934ce269a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28cb7fe640 1 -- 172.21.15.44:0/2613156332 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4010de0 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28cb7fe640 1 -- 172.21.15.44:0/2613156332 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f28e401e420 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eac241ab80 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28cb7fe640 1 -- 172.21.15.44:0/2613156332 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e401e5e0 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 10 client.17031 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55eac23ef3c0 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28cb7fe640 1 -- 172.21.15.44:0/2613156332 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eac241ab80 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28cb7fe640 1 -- 172.21.15.44:0/2613156332 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28e401d700 con 0x55eac2498950 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 10 client.17031 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 10 client.17031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 20 client.17031 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac24ac3d0 0x55eac24ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 4 client.17031 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:09.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.462+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eac23e31f0 con 0x55eac24ac3d0 2024-09-13T17:31:09.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.463+0000 7f28ebfff640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac24ac3d0 0x55eac24ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.463+0000 7f28ebfff640 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac24ac3d0 0x55eac24ae7f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f565305637000a5b server_cookie=9f075c80bc792ef5 in_seq=0 out_seq=0 2024-09-13T17:31:09.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.463+0000 7f28cb7fe640 10 client.17031 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28cb7fe640 1 -- 172.21.15.44:0/2613156332 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55eac23e31f0 con 0x55eac24ac3d0 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28cb7fe640 10 client.17031 handle_command_reply: tid=0 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 client.17031 shutdown 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 msgr2=0x55eac24a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a08f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f28e4012970 tx=0x7f28e4004ab0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 shutdown_connections 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac24ac3d0 0x55eac24ae7f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac249c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a08f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28cb7fe640 0 client.17031 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 --2- 172.21.15.44:0/2613156332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac249c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 >> 172.21.15.44:0/2613156332 conn(0x55eac2489d00 msgr2=0x55eac248fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 shutdown_connections 2024-09-13T17:31:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 1 -- 172.21.15.44:0/2613156332 wait complete. 2024-09-13T17:31:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.464+0000 7f28f502a740 20 client.17031 trim_cache size 0 max 16384 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 Processor -- start 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 -- start start 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 0x55eac24a0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac2498950 0x55eac24a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac24ac240 con 0x55eac248d790 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac24a1730 con 0x55eac2498950 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28f502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac248a9f0 con 0x55eac248e0b0 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 0x55eac24a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 0x55eac24a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55774/0 (socket says 172.21.15.44:55774) 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.475+0000 7f28ebfff640 1 -- 172.21.15.44:0/4075094958 learned_addr learned my addr 172.21.15.44:0/4075094958 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28f08a5640 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac2498950 0x55eac24a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28ebfff640 1 -- 172.21.15.44:0/4075094958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 msgr2=0x55eac24a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28ebfff640 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28ebfff640 1 -- 172.21.15.44:0/4075094958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac2498950 msgr2=0x55eac24a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28ebfff640 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac2498950 0x55eac24a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28ebfff640 1 -- 172.21.15.44:0/4075094958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eac2391770 con 0x55eac248d790 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28f08a5640 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac2498950 0x55eac24a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28ebfff640 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 0x55eac24a0870 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55eac24af0c0 tx=0x7f28dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f8087b3a0c809 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28d2ffd640 1 -- 172.21.15.44:0/4075094958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc017040 con 0x55eac248d790 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28d2ffd640 1 -- 172.21.15.44:0/4075094958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28dc01c900 con 0x55eac248d790 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.476+0000 7f28d2ffd640 1 -- 172.21.15.44:0/4075094958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc003bf0 con 0x55eac248d790 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- 172.21.15.44:0/4075094958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 msgr2=0x55eac24a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 0x55eac24a0870 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55eac24af0c0 tx=0x7f28dc0046c0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- 172.21.15.44:0/4075094958 shutdown_connections 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac2498950 0x55eac24a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- 172.21.15.44:0/4075094958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac248d790 0x55eac24a0870 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- 172.21.15.44:0/4075094958 >> 172.21.15.44:0/4075094958 conn(0x55eac2489d00 msgr2=0x55eac2497e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- 172.21.15.44:0/4075094958 shutdown_connections 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- 172.21.15.44:0/4075094958 wait complete. 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- start start 2024-09-13T17:31:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 Processor -- start 2024-09-13T17:31:09.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac24a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac24a48c0 con 0x55eac2498950 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac24a4a30 con 0x55eac248d790 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac24a4bf0 con 0x55eac248e0b0 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.477+0000 7f28f502a740 10 client.0 _ll_register_callbacks cb 0x7f28e00f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55790/0 (socket says 172.21.15.44:55790) 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac24a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 -- 172.21.15.44:0/3586518508 learned_addr learned my addr 172.21.15.44:0/3586518508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 -- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 msgr2=0x55eac24a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 -- 172.21.15.44:0/3586518508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 msgr2=0x55eac24a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac24a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 -- 172.21.15.44:0/3586518508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc00be30 con 0x55eac2498950 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28eb7fe640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f08a5640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a4380 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f28e4000f80 tx=0x7f28e401d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7a981141414973 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28cb7fe640 1 -- 172.21.15.44:0/3586518508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4010de0 con 0x55eac2498950 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e0004e90 con 0x55eac2498950 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28cb7fe640 1 -- 172.21.15.44:0/3586518508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f28e4027440 con 0x55eac2498950 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28cb7fe640 1 -- 172.21.15.44:0/3586518508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e4010820 con 0x55eac2498950 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f502a740 10 client.17037 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.478+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55eac23b6e80 con 0x55eac2498950 2024-09-13T17:31:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28cb7fe640 1 -- 172.21.15.44:0/3586518508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28e0004e90 con 0x55eac2498950 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28cb7fe640 1 -- 172.21.15.44:0/3586518508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28e401e420 con 0x55eac2498950 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28f502a740 10 client.17037 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28f502a740 10 client.17037 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28f502a740 20 client.17037 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28f502a740 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac2503b50 0x55eac2505f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28f502a740 4 client.17037 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eac20e44e0 con 0x55eac2503b50 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28ebfff640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac2503b50 0x55eac2505f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28cb7fe640 10 client.17037 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.479+0000 7f28ebfff640 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac2503b50 0x55eac2505f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59732e0e4bde1e0b server_cookie=4a47aaf82535a67a in_seq=0 out_seq=0 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28cb7fe640 1 -- 172.21.15.44:0/3586518508 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eac20e44e0 con 0x55eac2503b50 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28cb7fe640 10 client.17037 handle_command_reply: tid=0 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 client.17037 shutdown 2024-09-13T17:31:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 msgr2=0x55eac24a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a4380 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f28e4000f80 tx=0x7f28e401d660 comp rx=0 tx=0).stop 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 shutdown_connections 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55eac2503b50 0x55eac2505f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eac248e0b0 0x55eac24a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eac2498950 0x55eac24a4380 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28cb7fe640 0 client.17037 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 --2- 172.21.15.44:0/3586518508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55eac248d790 0x55eac24a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 >> 172.21.15.44:0/3586518508 conn(0x55eac2489d00 msgr2=0x55eac24947c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 shutdown_connections 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 1 -- 172.21.15.44:0/3586518508 wait complete. 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.480+0000 7f28f502a740 20 client.17037 trim_cache size 0 max 16384 2024-09-13T17:31:09.487 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28d40a1200 msgr2=0x7f28d40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28d40a1200 0x7f28d40a36c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28e4005fd0 tx=0x7f28e4004a40 comp rx=0 tx=0).stop 2024-09-13T17:31:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 msgr2=0x7f28ec1966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec1966e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f28e00057c0 tx=0x7f28e0032c20 comp rx=0 tx=0).stop 2024-09-13T17:31:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 shutdown_connections 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28d40a1200 0x7f28d40a36c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28ec10a330 0x7f28ec19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ec101800 0x7f28ec1966e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 --2- 172.21.15.44:0/2950606686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec100ee0 0x7f28ec1961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 >> 172.21.15.44:0/2950606686 conn(0x7f28ec0fa8d0 msgr2=0x7f28ec0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 shutdown_connections 2024-09-13T17:31:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.481+0000 7f28f2308640 1 -- 172.21.15.44:0/2950606686 wait complete. 2024-09-13T17:31:09.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:31:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 -- 172.21.15.44:0/3307950460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d410c670 msgr2=0x7f51d410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 --2- 172.21.15.44:0/3307950460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d410c670 0x7f51d410ea60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ca0 tx=0x7f51cc02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 -- 172.21.15.44:0/3307950460 shutdown_connections 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 --2- 172.21.15.44:0/3307950460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d410c670 0x7f51d410ea60 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 --2- 172.21.15.44:0/3307950460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51d40ff470 0x7f51d410c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 --2- 172.21.15.44:0/3307950460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d40feac0 0x7f51d40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 -- 172.21.15.44:0/3307950460 >> 172.21.15.44:0/3307950460 conn(0x7f51d4075c60 msgr2=0x7f51d40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 -- 172.21.15.44:0/3307950460 shutdown_connections 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.597+0000 7f51db641640 1 -- 172.21.15.44:0/3307950460 wait complete. 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 Processor -- start 2024-09-13T17:31:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 -- start start 2024-09-13T17:31:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 0x7f51d4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d40ff470 0x7f51d41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51d410c670 0x7f51d419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51d93dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 0x7f51d4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d419d2c0 con 0x7f51d40feac0 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d419d430 con 0x7f51d410c670 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51db641640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d419d5f0 con 0x7f51d40ff470 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51d93dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 0x7f51d4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55808/0 (socket says 172.21.15.44:55808) 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51d93dd640 1 -- 172.21.15.44:0/3339409801 learned_addr learned my addr 172.21.15.44:0/3339409801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d40ff470 0x7f51d41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.598+0000 7f51d9bde640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51d410c670 0x7f51d419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d93dd640 1 -- 172.21.15.44:0/3339409801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d40ff470 msgr2=0x7f51d41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d93dd640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d40ff470 0x7f51d41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d93dd640 1 -- 172.21.15.44:0/3339409801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51d410c670 msgr2=0x7f51d419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d93dd640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51d410c670 0x7f51d419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d93dd640 1 -- 172.21.15.44:0/3339409801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc009900 con 0x7f51d40feac0 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d93dd640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 0x7f51d4198390 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f51c400e9f0 tx=0x7f51c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d27fc640 1 -- 172.21.15.44:0/3339409801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4009700 con 0x7f51d40feac0 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d27fc640 1 -- 172.21.15.44:0/3339409801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c400cd80 con 0x7f51d40feac0 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d419d8d0 con 0x7f51d40feac0 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51d27fc640 1 -- 172.21.15.44:0/3339409801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c40187e0 con 0x7f51d40feac0 2024-09-13T17:31:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.599+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d41a5150 con 0x7f51d40feac0 2024-09-13T17:31:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.601+0000 7f51d27fc640 1 -- 172.21.15.44:0/3339409801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f51c4010430 con 0x7f51d40feac0 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 Processor -- start 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 -- start start 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 0x55b850190b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b8501910b0 0x55b85019b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b85019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8500facc0 con 0x55b850190790 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8500f8450 con 0x55b8501910b0 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51de444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8500f3d40 con 0x55b85019b950 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d27fc640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51bc0a1280 0x7f51bc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 0x55b850190b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 0x55b850190b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55814/0 (socket says 172.21.15.44:55814) 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 -- 172.21.15.44:0/4010844240 learned_addr learned my addr 172.21.15.44:0/4010844240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51bc0a1280 0x7f51bc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d27fc640 1 -- 172.21.15.44:0/3339409801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f51c4014070 con 0x7f51d40feac0 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d93dd640 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b8501910b0 0x55b85019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b85019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 -- 172.21.15.44:0/4010844240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 msgr2=0x55b85019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b85019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 -- 172.21.15.44:0/4010844240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b8501910b0 msgr2=0x55b85019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.602+0000 7f51d9bde640 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b8501910b0 0x55b85019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.603+0000 7f51d9bde640 1 -- 172.21.15.44:0/4010844240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8500bb620 con 0x55b850190790 2024-09-13T17:31:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.603+0000 7f51d9bde640 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 0x55b850190b70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0098d0 tx=0x7f51cc031c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c403a4f0867a677 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.603+0000 7f51bb7fe640 1 -- 172.21.15.44:0/4010844240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc003b70 con 0x55b850190790 2024-09-13T17:31:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.603+0000 7f51bb7fe640 1 -- 172.21.15.44:0/4010844240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51cc003cd0 con 0x55b850190790 2024-09-13T17:31:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.603+0000 7f51bb7fe640 1 -- 172.21.15.44:0/4010844240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0042a0 con 0x55b850190790 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.603+0000 7f51d9bde640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51bc0a1280 0x7f51bc0a3740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f51cc04a000 tx=0x7f51cc049000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 -- 172.21.15.44:0/4010844240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 msgr2=0x55b850190b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 0x55b850190b70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0098d0 tx=0x7f51cc031c80 comp rx=0 tx=0).stop 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 -- 172.21.15.44:0/4010844240 shutdown_connections 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b85019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b8501910b0 0x55b85019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 --2- 172.21.15.44:0/4010844240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b850190790 0x55b850190b70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 -- 172.21.15.44:0/4010844240 >> 172.21.15.44:0/4010844240 conn(0x55b85018cd00 msgr2=0x55b85018d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 -- 172.21.15.44:0/4010844240 shutdown_connections 2024-09-13T17:31:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 -- 172.21.15.44:0/4010844240 wait complete. 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 -- start start 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.604+0000 7f51de444740 1 Processor -- start 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b85019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b850198270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b8501987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8500b2180 con 0x55b85019b950 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84fdcc430 con 0x55b850190790 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8500775e0 con 0x55b8501910b0 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 10 client.0 _ll_register_callbacks cb 0x55b8500e3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b8501987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b8501987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55818/0 (socket says 172.21.15.44:55818) 2024-09-13T17:31:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 -- 172.21.15.44:0/1992080208 learned_addr learned my addr 172.21.15.44:0/1992080208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b850198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d93dd640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b85019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 -- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 msgr2=0x55b850198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b850198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 -- 172.21.15.44:0/1992080208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 msgr2=0x55b85019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b85019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 -- 172.21.15.44:0/1992080208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d410f640 con 0x55b85019b950 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51d9bde640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b8501987b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f51cc004270 tx=0x7f51cc0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e1d127094ecd11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51b8ff9640 1 -- 172.21.15.44:0/1992080208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004730 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51b8ff9640 1 -- 172.21.15.44:0/1992080208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f51cc003e40 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b85011db80 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51b8ff9640 1 -- 172.21.15.44:0/1992080208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51cc004890 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 10 client.17049 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.605+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b8500f23c0 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51b8ff9640 1 -- 172.21.15.44:0/1992080208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b85011db80 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51b8ff9640 1 -- 172.21.15.44:0/1992080208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51cc02f820 con 0x55b85019b950 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51de444740 10 client.17049 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51de444740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51de444740 20 client.17049 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51de444740 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b8501ab040 0x55b8501ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51de444740 4 client.17049 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8500e61f0 con 0x55b8501ab040 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.606+0000 7f51d93dd640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b8501ab040 0x55b8501ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51d93dd640 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b8501ab040 0x55b8501ad460 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98212a1372208bba server_cookie=16df887d7f7605e4 in_seq=0 out_seq=0 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51b8ff9640 10 client.17049 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51b8ff9640 1 -- 172.21.15.44:0/1992080208 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b8500e61f0 con 0x55b8501ab040 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51b8ff9640 10 client.17049 handle_command_reply: tid=0 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 client.17049 shutdown 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 msgr2=0x55b8501987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b8501987b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f51cc004270 tx=0x7f51cc0042a0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 shutdown_connections 2024-09-13T17:31:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b8501ab040 0x55b8501ad460 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b850198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b8501987b0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 --2- 172.21.15.44:0/1992080208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b85019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51b8ff9640 0 client.17049 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 >> 172.21.15.44:0/1992080208 conn(0x55b85018cd00 msgr2=0x55b850192a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 shutdown_connections 2024-09-13T17:31:09.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.607+0000 7f51de444740 1 -- 172.21.15.44:0/1992080208 wait complete. 2024-09-13T17:31:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.608+0000 7f51de444740 20 client.17049 trim_cache size 0 max 16384 2024-09-13T17:31:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 Processor -- start 2024-09-13T17:31:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 -- start start 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 0x55b8501a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b850196b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8501873b0 con 0x55b8501910b0 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85019b4c0 con 0x55b850190790 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51de444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b850191620 con 0x55b85019b950 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 0x55b8501a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 0x55b8501a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55820/0 (socket says 172.21.15.44:55820) 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 -- 172.21.15.44:0/1982161815 learned_addr learned my addr 172.21.15.44:0/1982161815 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d93dd640 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d9bde640 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b850196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 -- 172.21.15.44:0/1982161815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 msgr2=0x55b850196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b850196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 -- 172.21.15.44:0/1982161815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 msgr2=0x55b8501a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 -- 172.21.15.44:0/1982161815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b850094770 con 0x55b8501910b0 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.619+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 0x55b8501a2b50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f51d4199930 tx=0x7f51c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804642d288835778 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51bbfff640 1 -- 172.21.15.44:0/1982161815 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8017070 con 0x55b8501910b0 2024-09-13T17:31:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51bbfff640 1 -- 172.21.15.44:0/1982161815 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c8018040 con 0x55b8501910b0 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51bbfff640 1 -- 172.21.15.44:0/1982161815 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8003970 con 0x55b8501910b0 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 -- 172.21.15.44:0/1982161815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 msgr2=0x55b8501a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 0x55b8501a2b50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f51d4199930 tx=0x7f51c8005f90 comp rx=0 tx=0).stop 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 -- 172.21.15.44:0/1982161815 shutdown_connections 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b85019b950 0x55b850196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8501910b0 0x55b8501a2b50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 --2- 172.21.15.44:0/1982161815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 -- 172.21.15.44:0/1982161815 >> 172.21.15.44:0/1982161815 conn(0x55b85018cd00 msgr2=0x55b8501b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 -- 172.21.15.44:0/1982161815 shutdown_connections 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.620+0000 7f51de444740 1 -- 172.21.15.44:0/1982161815 wait complete. 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 -- start start 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 Processor -- start 2024-09-13T17:31:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b85019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b85019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85019fae0 con 0x55b85019b950 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85019fc50 con 0x55b850190790 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b850201540 con 0x55b8501910b0 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51de444740 10 client.0 _ll_register_callbacks cb 0x7f51d4069b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51d9bde640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b85019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51d9bde640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b85019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55834/0 (socket says 172.21.15.44:55834) 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51d9bde640 1 -- 172.21.15.44:0/665104957 learned_addr learned my addr 172.21.15.44:0/665104957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51d8bdc640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b85019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.621+0000 7f51d93dd640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51d9bde640 1 -- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 msgr2=0x55b85019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51d9bde640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b85019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51d9bde640 1 -- 172.21.15.44:0/665104957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 msgr2=0x55b8501a29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51d9bde640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51d9bde640 1 -- 172.21.15.44:0/665104957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8009900 con 0x55b85019b950 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51d9bde640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b85019f3e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f51cc02fca0 tx=0x7f51cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e742d8598e0c15b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51b8ff9640 1 -- 172.21.15.44:0/665104957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc02f3d0 con 0x55b85019b950 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8501a7fa0 con 0x55b85019b950 2024-09-13T17:31:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51b8ff9640 1 -- 172.21.15.44:0/665104957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f51cc003e40 con 0x55b85019b950 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51b8ff9640 1 -- 172.21.15.44:0/665104957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51cc031b80 con 0x55b85019b950 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 10 client.17055 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b8500b9e80 con 0x55b85019b950 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51b8ff9640 1 -- 172.21.15.44:0/665104957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8501a7fa0 con 0x55b85019b950 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51b8ff9640 1 -- 172.21.15.44:0/665104957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51cc049440 con 0x55b85019b950 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 10 client.17055 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 20 client.17055 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b850202800 0x55b850204bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 4 client.17055 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.622+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f51cc003cd0 con 0x55b850202800 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51d93dd640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b850202800 0x55b850204bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51b8ff9640 10 client.17055 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51d93dd640 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b850202800 0x55b850204bf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc948c6986cd619f server_cookie=f689d39d4b24930d in_seq=0 out_seq=0 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51b8ff9640 1 -- 172.21.15.44:0/665104957 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f51cc003cd0 con 0x55b850202800 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51b8ff9640 10 client.17055 handle_command_reply: tid=0 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 client.17055 shutdown 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 msgr2=0x55b85019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b85019f3e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f51cc02fca0 tx=0x7f51cc004ab0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 shutdown_connections 2024-09-13T17:31:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b850202800 0x55b850204bf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8501910b0 0x55b85019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85019b950 0x55b85019f3e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51b8ff9640 0 client.17055 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 --2- 172.21.15.44:0/665104957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b850190790 0x55b8501a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.623+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 >> 172.21.15.44:0/665104957 conn(0x55b85018cd00 msgr2=0x55b850193330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 shutdown_connections 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51de444740 1 -- 172.21.15.44:0/665104957 wait complete. 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51de444740 20 client.17055 trim_cache size 0 max 16384 2024-09-13T17:31:09.638 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:09.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51bc0a1280 msgr2=0x7f51bc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51bc0a1280 0x7f51bc0a3740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f51cc04a000 tx=0x7f51cc049000 comp rx=0 tx=0).stop 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 msgr2=0x7f51d4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 0x7f51d4198390 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f51c400e9f0 tx=0x7f51c400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 shutdown_connections 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f51bc0a1280 0x7f51bc0a3740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d40ff470 0x7f51d41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51d410c670 0x7f51d419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 --2- 172.21.15.44:0/3339409801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d40feac0 0x7f51d4198390 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 >> 172.21.15.44:0/3339409801 conn(0x7f51d4075c60 msgr2=0x7f51d40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 shutdown_connections 2024-09-13T17:31:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:09.624+0000 7f51db641640 1 -- 172.21.15.44:0/3339409801 wait complete. 2024-09-13T17:31:10.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: pgmap v136: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 69 KiB/s wr, 33 op/s 2024-09-13T17:31:10.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:31:10.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:10.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:31:10.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:10.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:31:10.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:09 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1173870943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: pgmap v136: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 69 KiB/s wr, 33 op/s 2024-09-13T17:31:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:31:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:31:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:31:10.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:09 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1173870943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:10.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: pgmap v136: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 69 KiB/s wr, 33 op/s 2024-09-13T17:31:10.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:31:10.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:10.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:31:10.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:10.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:31:10.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:09 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1173870943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:10.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:31:10.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 -- 172.21.15.44:0/3813929187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 msgr2=0x7f3974101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 --2- 172.21.15.44:0/3813929187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974101230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3964009900 tx=0x7f396402a410 comp rx=0 tx=0).stop 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 -- 172.21.15.44:0/3813929187 shutdown_connections 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 --2- 172.21.15.44:0/3813929187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f397410a350 0x7f397410c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 --2- 172.21.15.44:0/3813929187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3974101800 0x7f3974109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 --2- 172.21.15.44:0/3813929187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974101230 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 -- 172.21.15.44:0/3813929187 >> 172.21.15.44:0/3813929187 conn(0x7f39740fa930 msgr2=0x7f39740fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 -- 172.21.15.44:0/3813929187 shutdown_connections 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.764+0000 7f397a4d7640 1 -- 172.21.15.44:0/3813929187 wait complete. 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 Processor -- start 2024-09-13T17:31:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 -- start start 2024-09-13T17:31:10.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3974101800 0x7f3974196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397410a350 0x7f397419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397419b060 con 0x7f3974100e50 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397419b1d0 con 0x7f397410a350 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.765+0000 7f397a4d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397419b390 con 0x7f3974101800 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55858/0 (socket says 172.21.15.44:55858) 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 -- 172.21.15.44:0/1274964524 learned_addr learned my addr 172.21.15.44:0/1274964524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3978a74640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397410a350 0x7f397419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f39737fe640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3974101800 0x7f3974196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 -- 172.21.15.44:0/1274964524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3974101800 msgr2=0x7f3974196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3974101800 0x7f3974196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 -- 172.21.15.44:0/1274964524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397410a350 msgr2=0x7f397419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397410a350 0x7f397419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 -- 172.21.15.44:0/1274964524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964009560 con 0x7f3974100e50 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f39737fe640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3974101800 0x7f3974196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f3973fff640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974196130 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f39640057c0 tx=0x7f3964032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f39717fa640 1 -- 172.21.15.44:0/1274964524 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964032e10 con 0x7f3974100e50 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f39717fa640 1 -- 172.21.15.44:0/1274964524 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39640046c0 con 0x7f3974100e50 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397419b610 con 0x7f3974100e50 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f39717fa640 1 -- 172.21.15.44:0/1274964524 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964003850 con 0x7f3974100e50 2024-09-13T17:31:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.766+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39741a2ef0 con 0x7f3974100e50 2024-09-13T17:31:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f39717fa640 1 -- 172.21.15.44:0/1274964524 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3964005210 con 0x7f3974100e50 2024-09-13T17:31:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 Processor -- start 2024-09-13T17:31:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 -- start start 2024-09-13T17:31:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f40440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29ea0c60 con 0x559c29f360e0 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29e9e3f0 con 0x559c29f357c0 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f397d1fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29e99ce0 con 0x559c29f40980 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f39717fa640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f395c0a1280 0x7f395c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f3973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f40440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f3973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f40440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55870/0 (socket says 172.21.15.44:55870) 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f3973fff640 1 -- 172.21.15.44:0/1645690002 learned_addr learned my addr 172.21.15.44:0/1645690002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.769+0000 7f3978a74640 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f395c0a1280 0x7f395c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f39737fe640 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f42d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f39717fa640 1 -- 172.21.15.44:0/1274964524 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f39640edfb0 con 0x7f3974100e50 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 -- 172.21.15.44:0/1645690002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 msgr2=0x559c29f42d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f42d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 -- 172.21.15.44:0/1645690002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 msgr2=0x559c29f40440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f40440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 -- 172.21.15.44:0/1645690002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c29e55970 con 0x559c29f357c0 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3973fff640 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f40440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f35ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f396c005fd0 tx=0x7f396c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b010068297f96b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f3978a74640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f395c0a1280 0x7f395c0a3740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f396c010c40 tx=0x7f396c017000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f395a7fc640 1 -- 172.21.15.44:0/1645690002 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0126c0 con 0x559c29f357c0 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f395a7fc640 1 -- 172.21.15.44:0/1645690002 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f396c009b90 con 0x559c29f357c0 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.770+0000 7f395a7fc640 1 -- 172.21.15.44:0/1645690002 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c005020 con 0x559c29f357c0 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 -- 172.21.15.44:0/1645690002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 msgr2=0x559c29f35ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f35ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f396c005fd0 tx=0x7f396c010740 comp rx=0 tx=0).stop 2024-09-13T17:31:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 -- 172.21.15.44:0/1645690002 shutdown_connections 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f40440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1645690002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f35ba0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 -- 172.21.15.44:0/1645690002 >> 172.21.15.44:0/1645690002 conn(0x559c29f31d50 msgr2=0x559c29f32130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 -- 172.21.15.44:0/1645690002 shutdown_connections 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 -- 172.21.15.44:0/1645690002 wait complete. 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.771+0000 7f397d1fc740 1 -- start start 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 Processor -- start 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f3dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29e57f80 con 0x559c29f360e0 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29e57c20 con 0x559c29f357c0 2024-09-13T17:31:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29e24ab0 con 0x559c29f40980 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f397d1fc740 10 client.0 _ll_register_callbacks cb 0x559c29e93350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55872/0 (socket says 172.21.15.44:55872) 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 -- 172.21.15.44:0/2170802487 learned_addr learned my addr 172.21.15.44:0/2170802487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f3973fff640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f3978a74640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f3dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 -- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 msgr2=0x559c29f3dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f3dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 -- 172.21.15.44:0/2170802487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 msgr2=0x559c29f40330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f40330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f39737fe640 1 -- 172.21.15.44:0/2170802487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c01a040 con 0x559c29f360e0 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.772+0000 7f3978a74640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f3dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39737fe640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3d650 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3968009ef0 tx=0x7f39680117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381d721549e4b793 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39537fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39537fe640 1 -- 172.21.15.44:0/2170802487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968004220 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39537fe640 1 -- 172.21.15.44:0/2170802487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3968004380 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c29b8d590 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39537fe640 1 -- 172.21.15.44:0/2170802487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3968012660 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 10 client.17067 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c29e98360 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39537fe640 1 -- 172.21.15.44:0/2170802487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c29b8d590 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f39537fe640 1 -- 172.21.15.44:0/2170802487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f396800c070 con 0x559c29f360e0 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 10 client.17067 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 20 client.17067 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f50800 0x559c29f52c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 4 client.17067 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.773+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c29b9e860 con 0x559c29f50800 2024-09-13T17:31:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.774+0000 7f3973fff640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f50800 0x559c29f52c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.774+0000 7f3973fff640 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f50800 0x559c29f52c20 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a74576aee5dc530 server_cookie=38d40ddcf064d679 in_seq=0 out_seq=0 2024-09-13T17:31:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.774+0000 7f39537fe640 10 client.17067 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f39537fe640 1 -- 172.21.15.44:0/2170802487 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559c29b9e860 con 0x559c29f50800 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f39537fe640 10 client.17067 handle_command_reply: tid=0 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 client.17067 shutdown 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 msgr2=0x559c29f3d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3d650 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3968009ef0 tx=0x7f39680117b0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 shutdown_connections 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f50800 0x559c29f52c20 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f40980 0x559c29f3dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3d650 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 --2- 172.21.15.44:0/2170802487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f40330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f39537fe640 0 client.17067 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 >> 172.21.15.44:0/2170802487 conn(0x559c29f31d50 msgr2=0x559c29f371a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 shutdown_connections 2024-09-13T17:31:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 1 -- 172.21.15.44:0/2170802487 wait complete. 2024-09-13T17:31:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.775+0000 7f397d1fc740 20 client.17067 trim_cache size 0 max 16384 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 Processor -- start 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 -- start start 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f31d50 0x559c29f47680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f47bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29f3e730 con 0x559c29f360e0 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29f43430 con 0x559c29f357c0 2024-09-13T17:31:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f397d1fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29f23f20 con 0x559c29f31d50 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55882/0 (socket says 172.21.15.44:55882) 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 -- 172.21.15.44:0/1653246665 learned_addr learned my addr 172.21.15.44:0/1653246665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3973fff640 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f31d50 0x559c29f47680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f39737fe640 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f47bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 -- 172.21.15.44:0/1653246665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f31d50 msgr2=0x559c29f47680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f31d50 0x559c29f47680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 -- 172.21.15.44:0/1653246665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 msgr2=0x559c29f47bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f47bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 -- 172.21.15.44:0/1653246665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c29e3a1c0 con 0x559c29f360e0 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f39737fe640 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f47bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.790+0000 7f3978a74640 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3bf00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f396c009da0 tx=0x7f396c012a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf782bec7744a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f395affd640 1 -- 172.21.15.44:0/1653246665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0127c0 con 0x559c29f360e0 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f395affd640 1 -- 172.21.15.44:0/1653246665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f396c012d30 con 0x559c29f360e0 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f395affd640 1 -- 172.21.15.44:0/1653246665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c005290 con 0x559c29f360e0 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 -- 172.21.15.44:0/1653246665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 msgr2=0x559c29f3bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3bf00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f396c009da0 tx=0x7f396c012a60 comp rx=0 tx=0).stop 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 -- 172.21.15.44:0/1653246665 shutdown_connections 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f360e0 0x559c29f3bf00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29f47bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 --2- 172.21.15.44:0/1653246665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f31d50 0x559c29f47680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 -- 172.21.15.44:0/1653246665 >> 172.21.15.44:0/1653246665 conn(0x559c29f40490 msgr2=0x559c29f38940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 -- 172.21.15.44:0/1653246665 shutdown_connections 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.791+0000 7f397d1fc740 1 -- 172.21.15.44:0/1653246665 wait complete. 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 -- start start 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 Processor -- start 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 0x559c29fa1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29fa16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f360e0 0x559c29f46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29f47130 con 0x559c29f31d50 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29f472a0 con 0x559c29f357c0 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c29f47460 con 0x559c29f360e0 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f397d1fc740 10 client.0 _ll_register_callbacks cb 0x7f39680097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 0x559c29fa1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 0x559c29fa1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55890/0 (socket says 172.21.15.44:55890) 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 -- 172.21.15.44:0/3524255464 learned_addr learned my addr 172.21.15.44:0/3524255464 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3978a74640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f360e0 0x559c29f46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f39737fe640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29fa16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 -- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f360e0 msgr2=0x559c29f46a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f360e0 0x559c29f46a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 -- 172.21.15.44:0/3524255464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 msgr2=0x559c29fa16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.792+0000 7f3973fff640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29fa16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f3973fff640 1 -- 172.21.15.44:0/3524255464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c29e55970 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39737fe640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29fa16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f3973fff640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 0x559c29fa1170 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f39640eb9e0 tx=0x7f3964048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaee343bcda9351 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39537fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39537fe640 1 -- 172.21.15.44:0/3524255464 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640c3340 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39537fe640 1 -- 172.21.15.44:0/3524255464 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f39640d9170 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f396c009b90 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39537fe640 1 -- 172.21.15.44:0/3524255464 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39640eb660 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 10 client.17073 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c29e5f990 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39537fe640 1 -- 172.21.15.44:0/3524255464 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f396c009b90 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f39537fe640 1 -- 172.21.15.44:0/3524255464 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f396404d070 con 0x559c29f31d50 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 10 client.17073 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 20 client.17073 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f40980 0x559c29faad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 4 client.17073 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.793+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559c29bf1eb0 con 0x559c29f40980 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.794+0000 7f39737fe640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f40980 0x559c29faad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.794+0000 7f39537fe640 10 client.17073 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.794+0000 7f39737fe640 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f40980 0x559c29faad90 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4218ee815d423eaf server_cookie=39b37f7c88204df9 in_seq=0 out_seq=0 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.795+0000 7f39537fe640 1 -- 172.21.15.44:0/3524255464 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559c29bf1eb0 con 0x559c29f40980 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.795+0000 7f39537fe640 10 client.17073 handle_command_reply: tid=0 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.795+0000 7f397d1fc740 1 client.17073 shutdown 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.795+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 msgr2=0x559c29fa1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.795+0000 7f397d1fc740 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 0x559c29fa1170 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f39640eb9e0 tx=0x7f3964048000 comp rx=0 tx=0).stop 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.795+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 shutdown_connections 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c29f40980 0x559c29faad90 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c29f360e0 0x559c29f46a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c29f357c0 0x559c29fa16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f39537fe640 0 client.17073 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 --2- 172.21.15.44:0/3524255464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c29f31d50 0x559c29fa1170 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 >> 172.21.15.44:0/3524255464 conn(0x559c29f40490 msgr2=0x559c29f3bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 shutdown_connections 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 1 -- 172.21.15.44:0/3524255464 wait complete. 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397d1fc740 20 client.17073 trim_cache size 0 max 16384 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "3392a5f7-1390-4a78-84ab-a124fa1e16ab", 2024-09-13T17:31:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:31:10.803 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f395c0a1280 msgr2=0x7f395c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f395c0a1280 0x7f395c0a3740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f396c010c40 tx=0x7f396c017000 comp rx=0 tx=0).stop 2024-09-13T17:31:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 msgr2=0x7f3974196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974196130 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f39640057c0 tx=0x7f3964032c20 comp rx=0 tx=0).stop 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 shutdown_connections 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f395c0a1280 0x7f395c0a3740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3974101800 0x7f3974196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397410a350 0x7f397419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 --2- 172.21.15.44:0/1274964524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3974100e50 0x7f3974196130 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 >> 172.21.15.44:0/1274964524 conn(0x7f39740fa930 msgr2=0x7f39740fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 shutdown_connections 2024-09-13T17:31:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.796+0000 7f397a4d7640 1 -- 172.21.15.44:0/1274964524 wait complete. 2024-09-13T17:31:10.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:31:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.912+0000 7f60e6f9d640 1 -- 172.21.15.44:0/3740642127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e010c550 msgr2=0x7f60e010e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.912+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/3740642127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e010c550 0x7f60e010e940 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009ca0 tx=0x7f60d802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- 172.21.15.44:0/3740642127 shutdown_connections 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/3740642127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e00ff2c0 0x7f60e010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/3740642127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e010c550 0x7f60e010e940 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/3740642127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60e00fe9a0 0x7f60e00fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- 172.21.15.44:0/3740642127 >> 172.21.15.44:0/3740642127 conn(0x7f60e00fa870 msgr2=0x7f60e00fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- 172.21.15.44:0/3740642127 shutdown_connections 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- 172.21.15.44:0/3740642127 wait complete. 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 Processor -- start 2024-09-13T17:31:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- start start 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60e00fe9a0 0x7f60e0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 0x7f60e0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e010c550 0x7f60e019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019d140 con 0x7f60e00ff2c0 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019d2b0 con 0x7f60e00fe9a0 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.913+0000 7f60e6f9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019d470 con 0x7f60e010c550 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 0x7f60e0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 0x7f60e0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55918/0 (socket says 172.21.15.44:55918) 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 -- 172.21.15.44:0/1342380975 learned_addr learned my addr 172.21.15.44:0/1342380975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60e00fe9a0 0x7f60e0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e679c640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e010c550 0x7f60e019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 -- 172.21.15.44:0/1342380975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e010c550 msgr2=0x7f60e019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e010c550 0x7f60e019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 -- 172.21.15.44:0/1342380975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60e00fe9a0 msgr2=0x7f60e0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60e00fe9a0 0x7f60e0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 -- 172.21.15.44:0/1342380975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8009900 con 0x7f60e00ff2c0 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e679c640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e010c550 0x7f60e019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e579a640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 0x7f60e0198750 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d770 tx=0x7f60d000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60d6ffd640 1 -- 172.21.15.44:0/1342380975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0005220 con 0x7f60e00ff2c0 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60d6ffd640 1 -- 172.21.15.44:0/1342380975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60d0009cb0 con 0x7f60e00ff2c0 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e019d750 con 0x7f60e00ff2c0 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60d6ffd640 1 -- 172.21.15.44:0/1342380975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004c30 con 0x7f60e00ff2c0 2024-09-13T17:31:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.914+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e01a4fd0 con 0x7f60e00ff2c0 2024-09-13T17:31:10.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.917+0000 7f60d6ffd640 1 -- 172.21.15.44:0/1342380975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f60d0005380 con 0x7f60e00ff2c0 2024-09-13T17:31:10.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.917+0000 7f60eaff7740 1 Processor -- start 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.917+0000 7f60eaff7740 1 -- start start 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c5426463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542646900 0x55c542650c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5426511a0 0x55c542653590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60eaff7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5425b0f20 con 0x55c542645fe0 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60eaff7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5425ae6b0 con 0x55c542646900 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60eaff7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5425a9fa0 con 0x55c5426511a0 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60d6ffd640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f60cc0a1200 0x7f60cc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e679c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c5426463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e579a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5426511a0 0x55c542653590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e679c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c5426463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55930/0 (socket says 172.21.15.44:55930) 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e679c640 1 -- 172.21.15.44:0/500711227 learned_addr learned my addr 172.21.15.44:0/500711227 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542646900 0x55c542650c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e679c640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f60cc0a1200 0x7f60cc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60d6ffd640 1 -- 172.21.15.44:0/1342380975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f60d00cca70 con 0x7f60e00ff2c0 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 -- 172.21.15.44:0/500711227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5426511a0 msgr2=0x55c542653590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5426511a0 0x55c542653590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 -- 172.21.15.44:0/500711227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 msgr2=0x55c5426463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c5426463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 -- 172.21.15.44:0/500711227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c542564840 con 0x55c542646900 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542646900 0x55c542650c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00abf0 tx=0x7f60dc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee0a10f2f599058c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.918+0000 7f60e679c640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f60cc0a1200 0x7f60cc0a36c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60d8031040 tx=0x7f60d80095c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60cbfff640 1 -- 172.21.15.44:0/500711227 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc005220 con 0x55c542646900 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60cbfff640 1 -- 172.21.15.44:0/500711227 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60dc004c30 con 0x55c542646900 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60cbfff640 1 -- 172.21.15.44:0/500711227 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0186d0 con 0x55c542646900 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 -- 172.21.15.44:0/500711227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542646900 msgr2=0x55c542650c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542646900 0x55c542650c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00abf0 tx=0x7f60dc00f790 comp rx=0 tx=0).stop 2024-09-13T17:31:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 -- 172.21.15.44:0/500711227 shutdown_connections 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5426511a0 0x55c542653590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542646900 0x55c542650c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 --2- 172.21.15.44:0/500711227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c5426463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 -- 172.21.15.44:0/500711227 >> 172.21.15.44:0/500711227 conn(0x55c542642570 msgr2=0x55c542642950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.919+0000 7f60eaff7740 1 -- 172.21.15.44:0/500711227 shutdown_connections 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 -- 172.21.15.44:0/500711227 wait complete. 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 -- start start 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 Processor -- start 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542650aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c54264ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c54264e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54254b2a0 con 0x55c542645fe0 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c542282430 con 0x55c5426511a0 2024-09-13T17:31:10.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54252d680 con 0x55c542646900 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60eaff7740 10 client.0 _ll_register_callbacks cb 0x55c5425a3610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60e5f9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542650aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60e5f9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542650aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55944/0 (socket says 172.21.15.44:55944) 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.920+0000 7f60e5f9b640 1 -- 172.21.15.44:0/3109453675 learned_addr learned my addr 172.21.15.44:0/3109453675 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e579a640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c54264ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e679c640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c54264e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e5f9b640 1 -- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 msgr2=0x55c54264ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c54264ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e5f9b640 1 -- 172.21.15.44:0/3109453675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 msgr2=0x55c54264e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c54264e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e5f9b640 1 -- 172.21.15.44:0/3109453675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc00a820 con 0x55c542645fe0 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e679c640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c54264e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542650aa0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01a040 tx=0x7f60dc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9378c709eec609b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60c97fa640 1 -- 172.21.15.44:0/3109453675 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0184b0 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60c97fa640 1 -- 172.21.15.44:0/3109453675 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f60dc018610 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c54229d4e0 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60c97fa640 1 -- 172.21.15.44:0/3109453675 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60dc0187d0 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60eaff7740 10 client.17085 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.921+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c5425a8620 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60c97fa640 1 -- 172.21.15.44:0/3109453675 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c54229d4e0 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60c97fa640 1 -- 172.21.15.44:0/3109453675 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60dc0235f0 con 0x55c542645fe0 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60eaff7740 10 client.17085 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60eaff7740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60eaff7740 20 client.17085 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c542660e10 0x55c542663200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60eaff7740 4 client.17085 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5422ae7e0 con 0x55c542660e10 2024-09-13T17:31:10.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60e579a640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c542660e10 0x55c542663200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60c97fa640 10 client.17085 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.922+0000 7f60e579a640 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c542660e10 0x55c542663200 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7a8694ee4b921f server_cookie=e11f4f8418c14136 in_seq=0 out_seq=0 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60c97fa640 1 -- 172.21.15.44:0/3109453675 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c5422ae7e0 con 0x55c542660e10 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60c97fa640 10 client.17085 handle_command_reply: tid=0 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 client.17085 shutdown 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 msgr2=0x55c542650aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542650aa0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01a040 tx=0x7f60dc005050 comp rx=0 tx=0).stop 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 shutdown_connections 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c542660e10 0x55c542663200 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c54264ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60c97fa640 0 client.17085 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c54264e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3109453675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542650aa0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 >> 172.21.15.44:0/3109453675 conn(0x55c542642570 msgr2=0x55c542641760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 shutdown_connections 2024-09-13T17:31:10.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 1 -- 172.21.15.44:0/3109453675 wait complete. 2024-09-13T17:31:10.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.923+0000 7f60eaff7740 20 client.17085 trim_cache size 0 max 16384 2024-09-13T17:31:10.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 Processor -- start 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 -- start start 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542645fe0 0x55c542641f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542646e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 0x55c542647360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54263dee0 con 0x55c5426511a0 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54265ce20 con 0x55c542645fe0 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60eaff7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54264f090 con 0x55c542646900 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 0x55c542647360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 0x55c542647360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55956/0 (socket says 172.21.15.44:55956) 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 -- 172.21.15.44:0/3564109643 learned_addr learned my addr 172.21.15.44:0/3564109643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542645fe0 0x55c542641f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 -- 172.21.15.44:0/3564109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 msgr2=0x55c542646e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542646e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 -- 172.21.15.44:0/3564109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542645fe0 msgr2=0x55c542641f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542645fe0 0x55c542641f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 -- 172.21.15.44:0/3564109643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c54254a790 con 0x55c5426511a0 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.938+0000 7f60e679c640 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 0x55c542647360 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f60d8032ea0 tx=0x7f60d8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c92f1b3b42ac20f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60d4ff9640 1 -- 172.21.15.44:0/3564109643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8003a30 con 0x55c5426511a0 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60d4ff9640 1 -- 172.21.15.44:0/3564109643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60d8030a50 con 0x55c5426511a0 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60d4ff9640 1 -- 172.21.15.44:0/3564109643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8004400 con 0x55c5426511a0 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 -- 172.21.15.44:0/3564109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 msgr2=0x55c542647360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 0x55c542647360 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f60d8032ea0 tx=0x7f60d8043000 comp rx=0 tx=0).stop 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 -- 172.21.15.44:0/3564109643 shutdown_connections 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542646e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5426511a0 0x55c542647360 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 --2- 172.21.15.44:0/3564109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c542645fe0 0x55c542641f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 -- 172.21.15.44:0/3564109643 >> 172.21.15.44:0/3564109643 conn(0x55c542642570 msgr2=0x55c542667b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 -- 172.21.15.44:0/3564109643 shutdown_connections 2024-09-13T17:31:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.939+0000 7f60eaff7740 1 -- 172.21.15.44:0/3564109643 wait complete. 2024-09-13T17:31:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 -- start start 2024-09-13T17:31:10.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 Processor -- start 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542655cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542656210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c5426b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c542656b60 con 0x55c542645fe0 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5426b6820 con 0x55c5426511a0 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5426b6960 con 0x55c542646900 2024-09-13T17:31:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60eaff7740 10 client.0 _ll_register_callbacks cb 0x7f60d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60e5f9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542655cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60e5f9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542655cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55968/0 (socket says 172.21.15.44:55968) 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60e5f9b640 1 -- 172.21.15.44:0/2298185439 learned_addr learned my addr 172.21.15.44:0/2298185439 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.940+0000 7f60e579a640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542656210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e679c640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c5426b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e5f9b640 1 -- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 msgr2=0x55c542656210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542656210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e5f9b640 1 -- 172.21.15.44:0/2298185439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 msgr2=0x55c5426b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c5426b62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e5f9b640 1 -- 172.21.15.44:0/2298185439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d8046040 con 0x55c542645fe0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60e5f9b640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542655cd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60dc000c00 tx=0x7f60dc005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0580339434cc2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60c97fa640 1 -- 172.21.15.44:0/2298185439 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0050c0 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60dc004c30 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60c97fa640 1 -- 172.21.15.44:0/2298185439 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f60dc000e40 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60c97fa640 1 -- 172.21.15.44:0/2298185439 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60dc004aa0 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60eaff7740 10 client.17091 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c54256fea0 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60c97fa640 1 -- 172.21.15.44:0/2298185439 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60dc004c30 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60c97fa640 1 -- 172.21.15.44:0/2298185439 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60dc02cb60 con 0x55c542645fe0 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60eaff7740 10 client.17091 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60eaff7740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.941+0000 7f60eaff7740 20 client.17091 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c5426b7cb0 0x55c5426ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 4 client.17091 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:31:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c542301e30 con 0x55c5426b7cb0 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60e579a640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c5426b7cb0 0x55c5426ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60e579a640 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c5426b7cb0 0x55c5426ba0a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38a7cab83d7358d server_cookie=eb034149995d3ef2 in_seq=0 out_seq=0 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60c97fa640 10 client.17091 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60c97fa640 1 -- 172.21.15.44:0/2298185439 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c542301e30 con 0x55c5426b7cb0 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60c97fa640 10 client.17091 handle_command_reply: tid=0 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 client.17091 shutdown 2024-09-13T17:31:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 msgr2=0x55c542655cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542655cd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60dc000c00 tx=0x7f60dc005680 comp rx=0 tx=0).stop 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 shutdown_connections 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.942+0000 7f60eaff7740 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c5426b7cb0 0x55c5426ba0a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c542646900 0x55c542656210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60c97fa640 0 client.17091 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5426511a0 0x55c5426b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 1 --2- 172.21.15.44:0/2298185439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c542645fe0 0x55c542655cd0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 >> 172.21.15.44:0/2298185439 conn(0x55c542642570 msgr2=0x55c542646e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 shutdown_connections 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 1 -- 172.21.15.44:0/2298185439 wait complete. 2024-09-13T17:31:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60eaff7740 20 client.17091 trim_cache size 0 max 16384 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: "3392a5f7-1390-4a78-84ab-a124fa1e16ab": { 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "3392a5f7-1390-4a78-84ab-a124fa1e16ab", 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:31:10.951 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f60cc0a1200 msgr2=0x7f60cc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f60cc0a1200 0x7f60cc0a36c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60d8031040 tx=0x7f60d80095c0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 msgr2=0x7f60e0198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 0x7f60e0198750 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d770 tx=0x7f60d000db70 comp rx=0 tx=0).stop 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 shutdown_connections 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f60cc0a1200 0x7f60cc0a36c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e010c550 0x7f60e019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e00ff2c0 0x7f60e0198750 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 --2- 172.21.15.44:0/1342380975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60e00fe9a0 0x7f60e0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 >> 172.21.15.44:0/1342380975 conn(0x7f60e00fa870 msgr2=0x7f60e00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 shutdown_connections 2024-09-13T17:31:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:10.943+0000 7f60e6f9d640 1 -- 172.21.15.44:0/1342380975 wait complete. 2024-09-13T17:31:10.955 INFO:tasks.cephfs.filesystem:scrub status for tag:3392a5f7-1390-4a78-84ab-a124fa1e16ab - {'path': '/', 'tag': '3392a5f7-1390-4a78-84ab-a124fa1e16ab', 'options': 'recursive,force'} 2024-09-13T17:31:12.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:11 smithi044 ceph-mon[31175]: pgmap v137: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-13T17:31:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:11 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:31:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:11 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:11 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:31:12.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:11 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:11 smithi070 ceph-mon[34502]: pgmap v137: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-13T17:31:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:11 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:31:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:11 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:11 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:31:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:11 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:11 smithi100 ceph-mon[35055]: pgmap v137: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-13T17:31:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:11 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:31:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:11 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:11 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:31:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:11 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:14.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:13 smithi044 ceph-mon[31175]: pgmap v138: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-13T17:31:14.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:13 smithi070 ceph-mon[34502]: pgmap v138: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-13T17:31:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:13 smithi100 ceph-mon[35055]: pgmap v138: 129 pgs: 129 active+clean; 2.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-13T17:31:15.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:14 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:31:15.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:14 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:31:15.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:14 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 -- 172.21.15.44:0/1944984185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 msgr2=0x7efc281002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 --2- 172.21.15.44:0/1944984185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281002d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efc18009930 tx=0x7efc1802a420 comp rx=0 tx=0).stop 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 -- 172.21.15.44:0/1944984185 shutdown_connections 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 --2- 172.21.15.44:0/1944984185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc281008a0 0x7efc281048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 --2- 172.21.15.44:0/1944984185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc28105040 0x7efc2810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 --2- 172.21.15.44:0/1944984185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281002d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 -- 172.21.15.44:0/1944984185 >> 172.21.15.44:0/1944984185 conn(0x7efc280fb960 msgr2=0x7efc280fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 -- 172.21.15.44:0/1944984185 shutdown_connections 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.701+0000 7efc2de5a640 1 -- 172.21.15.44:0/1944984185 wait complete. 2024-09-13T17:31:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 Processor -- start 2024-09-13T17:31:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 -- start start 2024-09-13T17:31:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc28105040 0x7efc28196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2819ab10 0x7efc2819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819b500 con 0x7efc280ffef0 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819b670 con 0x7efc28105040 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.702+0000 7efc2de5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819b7e0 con 0x7efc2819ab10 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55986/0 (socket says 172.21.15.44:55986) 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 -- 172.21.15.44:0/461957871 learned_addr learned my addr 172.21.15.44:0/461957871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc26ffd640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc28105040 0x7efc28196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc27fff640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2819ab10 0x7efc2819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 -- 172.21.15.44:0/461957871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2819ab10 msgr2=0x7efc2819afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2819ab10 0x7efc2819afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 -- 172.21.15.44:0/461957871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc28105040 msgr2=0x7efc28196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc28105040 0x7efc28196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 -- 172.21.15.44:0/461957871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009560 con 0x7efc280ffef0 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc277fe640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281961e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efc18005ab0 tx=0x7efc180329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc24ff9640 1 -- 172.21.15.44:0/461957871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18032df0 con 0x7efc280ffef0 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc24ff9640 1 -- 172.21.15.44:0/461957871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc18005490 con 0x7efc280ffef0 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc281a2dc0 con 0x7efc280ffef0 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.703+0000 7efc24ff9640 1 -- 172.21.15.44:0/461957871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180035f0 con 0x7efc280ffef0 2024-09-13T17:31:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.704+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc281a31e0 con 0x7efc280ffef0 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.706+0000 7efc24ff9640 1 -- 172.21.15.44:0/461957871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc180046c0 con 0x7efc280ffef0 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.706+0000 7efc30c56740 1 Processor -- start 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.706+0000 7efc30c56740 1 -- start start 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc30c56740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd06f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc30c56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd079fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc30c56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc30c56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bcfdac30 con 0x5650bd07a4f0 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc30c56740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bcfd83c0 con 0x5650bd06f330 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc30c56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bcfd3cb0 con 0x5650bd06fc50 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc24ff9640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc100a1280 0x7efc100a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56000/0 (socket says 172.21.15.44:56000) 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 -- 172.21.15.44:0/3969110385 learned_addr learned my addr 172.21.15.44:0/3969110385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc277fe640 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd079fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc27fff640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc100a1280 0x7efc100a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc24ff9640 1 -- 172.21.15.44:0/461957871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efc180eda80 con 0x7efc280ffef0 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 -- 172.21.15.44:0/3969110385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 msgr2=0x5650bd079fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd079fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 -- 172.21.15.44:0/3969110385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 msgr2=0x5650bd06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 -- 172.21.15.44:0/3969110385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bcf7a910 con 0x5650bd07a4f0 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.707+0000 7efc26ffd640 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07c8e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009c60 tx=0x7efc1c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fbff3ef6756184 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc27fff640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc100a1280 0x7efc100a3740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc20009fd0 tx=0x7efc20004ab0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc0dffb640 1 -- 172.21.15.44:0/3969110385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c004230 con 0x5650bd07a4f0 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc0dffb640 1 -- 172.21.15.44:0/3969110385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc1c004390 con 0x5650bd07a4f0 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc0dffb640 1 -- 172.21.15.44:0/3969110385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c019640 con 0x5650bd07a4f0 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 -- 172.21.15.44:0/3969110385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 msgr2=0x5650bd07c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07c8e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009c60 tx=0x7efc1c011810 comp rx=0 tx=0).stop 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 -- 172.21.15.44:0/3969110385 shutdown_connections 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd079fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07c8e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 --2- 172.21.15.44:0/3969110385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 -- 172.21.15.44:0/3969110385 >> 172.21.15.44:0/3969110385 conn(0x5650bd06b8c0 msgr2=0x5650bd06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 -- 172.21.15.44:0/3969110385 shutdown_connections 2024-09-13T17:31:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.708+0000 7efc30c56740 1 -- 172.21.15.44:0/3969110385 wait complete. 2024-09-13T17:31:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 -- start start 2024-09-13T17:31:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 Processor -- start 2024-09-13T17:31:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06f330 0x5650bd079e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06fc50 0x5650bd077090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd0775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bcf91e80 con 0x5650bd07a4f0 2024-09-13T17:31:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bcf91b20 con 0x5650bd06fc50 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bcf57260 con 0x5650bd06f330 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc30c56740 10 client.0 _ll_register_callbacks cb 0x5650bcf8cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.709+0000 7efc27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd0775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd0775f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56014/0 (socket says 172.21.15.44:56014) 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 -- 172.21.15.44:0/152537312 learned_addr learned my addr 172.21.15.44:0/152537312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc277fe640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06f330 0x5650bd079e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc26ffd640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06fc50 0x5650bd077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 -- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06f330 msgr2=0x5650bd079e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06f330 0x5650bd079e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 -- 172.21.15.44:0/152537312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06fc50 msgr2=0x5650bd077090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06fc50 0x5650bd077090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 -- 172.21.15.44:0/152537312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c009890 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc26ffd640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06fc50 0x5650bd077090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc27fff640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd0775f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efc20012040 tx=0x7efc20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f2cc94b6e2de6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc06ffd640 1 -- 172.21.15.44:0/152537312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20010dc0 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc06ffd640 1 -- 172.21.15.44:0/152537312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efc2002a460 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.710+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650bcffc680 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc06ffd640 1 -- 172.21.15.44:0/152537312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc200133d0 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 10 client.17103 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5650bcfd2330 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc06ffd640 1 -- 172.21.15.44:0/152537312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5650bcffc680 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc06ffd640 1 -- 172.21.15.44:0/152537312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc2001f460 con 0x5650bd07a4f0 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 10 client.17103 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 20 client.17103 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd089eb0 0x5650bd08c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 4 client.17103 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650bcfc75d0 con 0x5650bd089eb0 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.711+0000 7efc277fe640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd089eb0 0x5650bd08c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.712+0000 7efc06ffd640 10 client.17103 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.712+0000 7efc277fe640 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd089eb0 0x5650bd08c2d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6724390ad9aa8e93 server_cookie=a6b01cf0cdba2730 in_seq=0 out_seq=0 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.712+0000 7efc06ffd640 1 -- 172.21.15.44:0/152537312 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5650bcfc75d0 con 0x5650bd089eb0 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.712+0000 7efc06ffd640 10 client.17103 handle_command_reply: tid=0 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.712+0000 7efc30c56740 1 client.17103 shutdown 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 msgr2=0x5650bd0775f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd0775f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efc20012040 tx=0x7efc20004a40 comp rx=0 tx=0).stop 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 shutdown_connections 2024-09-13T17:31:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd089eb0 0x5650bd08c2d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd0775f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06fc50 0x5650bd077090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 --2- 172.21.15.44:0/152537312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06f330 0x5650bd079e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 >> 172.21.15.44:0/152537312 conn(0x5650bd06b8c0 msgr2=0x5650bd06ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc06ffd640 0 client.17103 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 shutdown_connections 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 1 -- 172.21.15.44:0/152537312 wait complete. 2024-09-13T17:31:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.713+0000 7efc30c56740 20 client.17103 trim_cache size 0 max 16384 2024-09-13T17:31:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.725+0000 7efc30c56740 1 Processor -- start 2024-09-13T17:31:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.725+0000 7efc30c56740 1 -- start start 2024-09-13T17:31:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc30c56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 0x5650bd072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc30c56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc30c56740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd07a4f0 0x5650bd075940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc30c56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bd079a00 con 0x5650bd06f330 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc30c56740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bd06c4c0 con 0x5650bd07a4f0 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc30c56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bd077bf0 con 0x5650bd06fc50 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 0x5650bd072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 0x5650bd072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56020/0 (socket says 172.21.15.44:56020) 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 -- 172.21.15.44:0/2104698074 learned_addr learned my addr 172.21.15.44:0/2104698074 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc27fff640 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd07a4f0 0x5650bd075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc26ffd640 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 -- 172.21.15.44:0/2104698074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 msgr2=0x5650bd0728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd0728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 -- 172.21.15.44:0/2104698074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd07a4f0 msgr2=0x5650bd075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd07a4f0 0x5650bd075940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 -- 172.21.15.44:0/2104698074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bcf74300 con 0x5650bd06f330 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc26ffd640 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd0728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc277fe640 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 0x5650bd072370 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efc180b6d70 tx=0x7efc18032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c5169c058cb1da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc0e7fc640 1 -- 172.21.15.44:0/2104698074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180b8a00 con 0x5650bd06f330 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc0e7fc640 1 -- 172.21.15.44:0/2104698074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc180b8c60 con 0x5650bd06f330 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.726+0000 7efc0e7fc640 1 -- 172.21.15.44:0/2104698074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180da0b0 con 0x5650bd06f330 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 -- 172.21.15.44:0/2104698074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 msgr2=0x5650bd072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 0x5650bd072370 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efc180b6d70 tx=0x7efc18032f50 comp rx=0 tx=0).stop 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 -- 172.21.15.44:0/2104698074 shutdown_connections 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd0728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd07a4f0 0x5650bd075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 --2- 172.21.15.44:0/2104698074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd06f330 0x5650bd072370 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 -- 172.21.15.44:0/2104698074 >> 172.21.15.44:0/2104698074 conn(0x5650bd06b8c0 msgr2=0x5650bd090c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 -- 172.21.15.44:0/2104698074 shutdown_connections 2024-09-13T17:31:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 -- 172.21.15.44:0/2104698074 wait complete. 2024-09-13T17:31:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.727+0000 7efc30c56740 1 -- start start 2024-09-13T17:31:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 Processor -- start 2024-09-13T17:31:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd081430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bd07fa10 con 0x5650bd07a4f0 2024-09-13T17:31:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bd07fb80 con 0x5650bd06f330 2024-09-13T17:31:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bd07fd40 con 0x5650bd06fc50 2024-09-13T17:31:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc30c56740 10 client.0 _ll_register_callbacks cb 0x7efc1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56028/0 (socket says 172.21.15.44:56028) 2024-09-13T17:31:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 -- 172.21.15.44:0/1609670802 learned_addr learned my addr 172.21.15.44:0/1609670802 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc26ffd640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd081430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc277fe640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 -- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 msgr2=0x5650bd081430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd081430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 -- 172.21.15.44:0/1609670802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 msgr2=0x5650bd080ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd080ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.728+0000 7efc27fff640 1 -- 172.21.15.44:0/1609670802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc281a31e0 con 0x5650bd07a4f0 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc27fff640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07f340 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x5650bd078600 tx=0x7efc20020680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde06976b8d9315b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc06ffd640 1 -- 172.21.15.44:0/1609670802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20010800 con 0x5650bd07a4f0 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc06ffd640 1 -- 172.21.15.44:0/1609670802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efc2001f5e0 con 0x5650bd07a4f0 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc1c004390 con 0x5650bd07a4f0 2024-09-13T17:31:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc06ffd640 1 -- 172.21.15.44:0/1609670802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc20009be0 con 0x5650bd07a4f0 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 10 client.17109 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5650bcf998e0 con 0x5650bd07a4f0 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc06ffd640 1 -- 172.21.15.44:0/1609670802 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc1c004390 con 0x5650bd07a4f0 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc06ffd640 1 -- 172.21.15.44:0/1609670802 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc20003a90 con 0x5650bd07a4f0 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 10 client.17109 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 20 client.17109 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd0e1430 0x5650bd0e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 4 client.17109 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:31:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.729+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650bccc7300 con 0x5650bd0e1430 2024-09-13T17:31:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.730+0000 7efc277fe640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd0e1430 0x5650bd0e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.730+0000 7efc06ffd640 10 client.17109 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.730+0000 7efc277fe640 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd0e1430 0x5650bd0e3850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e44c66f916d1d7d server_cookie=bf7797d38b4630df in_seq=0 out_seq=0 2024-09-13T17:31:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc06ffd640 1 -- 172.21.15.44:0/1609670802 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650bccc7300 con 0x5650bd0e1430 2024-09-13T17:31:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc06ffd640 10 client.17109 handle_command_reply: tid=0 2024-09-13T17:31:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 client.17109 shutdown 2024-09-13T17:31:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 msgr2=0x5650bd07f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07f340 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x5650bd078600 tx=0x7efc20020680 comp rx=0 tx=0).stop 2024-09-13T17:31:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 shutdown_connections 2024-09-13T17:31:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5650bd0e1430 0x5650bd0e3850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650bd06fc50 0x5650bd081430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650bd07a4f0 0x5650bd07f340 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 --2- 172.21.15.44:0/1609670802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5650bd06f330 0x5650bd080ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 >> 172.21.15.44:0/1609670802 conn(0x5650bd06b8c0 msgr2=0x5650bd072370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc06ffd640 0 client.17109 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 shutdown_connections 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 1 -- 172.21.15.44:0/1609670802 wait complete. 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.743+0000 7efc30c56740 20 client.17109 trim_cache size 0 max 16384 2024-09-13T17:31:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:31:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:31:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:31:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:31:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc100a1280 msgr2=0x7efc100a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc100a1280 0x7efc100a3740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc20009fd0 tx=0x7efc20004ab0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 msgr2=0x7efc281961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281961e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efc18005ab0 tx=0x7efc180329f0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 shutdown_connections 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc100a1280 0x7efc100a3740 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2819ab10 0x7efc2819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc28105040 0x7efc28196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 --2- 172.21.15.44:0/461957871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc280ffef0 0x7efc281961e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 >> 172.21.15.44:0/461957871 conn(0x7efc280fb960 msgr2=0x7efc280fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 shutdown_connections 2024-09-13T17:31:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:15.744+0000 7efc2de5a640 1 -- 172.21.15.44:0/461957871 wait complete. 2024-09-13T17:31:16.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:15 smithi044 ceph-mon[31175]: pgmap v139: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-09-13T17:31:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:15 smithi070 ceph-mon[34502]: pgmap v139: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-09-13T17:31:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:15 smithi100 ceph-mon[35055]: pgmap v139: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-09-13T17:31:18.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:17 smithi044 ceph-mon[31175]: pgmap v140: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-09-13T17:31:18.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:17 smithi070 ceph-mon[34502]: pgmap v140: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-09-13T17:31:18.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:17 smithi100 ceph-mon[35055]: pgmap v140: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-09-13T17:31:20.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:19 smithi044 ceph-mon[31175]: pgmap v141: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-09-13T17:31:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:19 smithi070 ceph-mon[34502]: pgmap v141: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-09-13T17:31:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:19 smithi100 ceph-mon[35055]: pgmap v141: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-09-13T17:31:22.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:21 smithi044 ceph-mon[31175]: pgmap v142: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-13T17:31:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:21 smithi070 ceph-mon[34502]: pgmap v142: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-13T17:31:22.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:21 smithi100 ceph-mon[35055]: pgmap v142: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-13T17:31:24.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:23 smithi044 ceph-mon[31175]: pgmap v143: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-13T17:31:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:23 smithi070 ceph-mon[34502]: pgmap v143: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-13T17:31:24.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:23 smithi100 ceph-mon[35055]: pgmap v143: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-13T17:31:26.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:25 smithi044 ceph-mon[31175]: pgmap v144: 129 pgs: 129 active+clean; 2.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-09-13T17:31:26.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:25 smithi070 ceph-mon[34502]: pgmap v144: 129 pgs: 129 active+clean; 2.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-09-13T17:31:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:25 smithi100 ceph-mon[35055]: pgmap v144: 129 pgs: 129 active+clean; 2.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-09-13T17:31:28.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:27 smithi044 ceph-mon[31175]: pgmap v145: 129 pgs: 129 active+clean; 2.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-13T17:31:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:27 smithi070 ceph-mon[34502]: pgmap v145: 129 pgs: 129 active+clean; 2.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-13T17:31:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:27 smithi100 ceph-mon[35055]: pgmap v145: 129 pgs: 129 active+clean; 2.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 -- 172.21.15.44:0/3063763138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4104ce0 msgr2=0x7f6ff41050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 --2- 172.21.15.44:0/3063763138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4104ce0 0x7f6ff41050c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009900 tx=0x7f6ff002a410 comp rx=0 tx=0).stop 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 -- 172.21.15.44:0/3063763138 shutdown_connections 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 --2- 172.21.15.44:0/3063763138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff40681d0 0x7f6ff410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 --2- 172.21.15.44:0/3063763138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4067830 0x7f6ff4067c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 --2- 172.21.15.44:0/3063763138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4104ce0 0x7f6ff41050c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 -- 172.21.15.44:0/3063763138 >> 172.21.15.44:0/3063763138 conn(0x7f6ff4074940 msgr2=0x7f6ff4074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 -- 172.21.15.44:0/3063763138 shutdown_connections 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.841+0000 7f6ffb984640 1 -- 172.21.15.44:0/3063763138 wait complete. 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 Processor -- start 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 -- start start 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 0x7f6ff41961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff40681d0 0x7f6ff41966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4104ce0 0x7f6ff419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff419b0e0 con 0x7f6ff4067830 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff419b250 con 0x7f6ff4104ce0 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ffb984640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff419b410 con 0x7f6ff40681d0 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 0x7f6ff41961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 0x7f6ff41961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47796/0 (socket says 172.21.15.44:47796) 2024-09-13T17:31:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 -- 172.21.15.44:0/2089315775 learned_addr learned my addr 172.21.15.44:0/2089315775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4104ce0 0x7f6ff419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff40681d0 0x7f6ff41966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 -- 172.21.15.44:0/2089315775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff40681d0 msgr2=0x7f6ff41966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff40681d0 0x7f6ff41966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 -- 172.21.15.44:0/2089315775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4104ce0 msgr2=0x7f6ff419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4104ce0 0x7f6ff419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.842+0000 7f6ff9720640 1 -- 172.21.15.44:0/2089315775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009560 con 0x7f6ff4067830 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.843+0000 7f6ff9720640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 0x7f6ff41961b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00057c0 tx=0x7f6ff0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.843+0000 7f6fea7fc640 1 -- 172.21.15.44:0/2089315775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00044d0 con 0x7f6ff4067830 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.843+0000 7f6fea7fc640 1 -- 172.21.15.44:0/2089315775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ff0032e10 con 0x7f6ff4067830 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.843+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff419b690 con 0x7f6ff4067830 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.843+0000 7f6fea7fc640 1 -- 172.21.15.44:0/2089315775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0003850 con 0x7f6ff4067830 2024-09-13T17:31:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.843+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff419b960 con 0x7f6ff4067830 2024-09-13T17:31:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.845+0000 7f6fea7fc640 1 -- 172.21.15.44:0/2089315775 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6ff0004850 con 0x7f6ff4067830 2024-09-13T17:31:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 Processor -- start 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 -- start start 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x56220807d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 0x562208087fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622080884f0 0x56220808a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207fe8c30 con 0x56220807dc50 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207fe63c0 con 0x5622080884f0 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207fe1cb0 con 0x56220807d330 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 0x562208087fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6fea7fc640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6fe00a1200 0x7f6fe00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff8f1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622080884f0 0x56220808a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 0x562208087fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47804/0 (socket says 172.21.15.44:47804) 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 -- 172.21.15.44:0/3081113289 learned_addr learned my addr 172.21.15.44:0/3081113289 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x56220807d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6fe00a1200 0x7f6fe00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6fea7fc640 1 -- 172.21.15.44:0/2089315775 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6ff00eddc0 con 0x7f6ff4067830 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 -- 172.21.15.44:0/3081113289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 msgr2=0x56220807d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x56220807d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 -- 172.21.15.44:0/3081113289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622080884f0 msgr2=0x56220808a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.846+0000 7f6ff9720640 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622080884f0 0x56220808a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.847+0000 7f6ff9720640 1 -- 172.21.15.44:0/3081113289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562207f88910 con 0x56220807dc50 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.847+0000 7f6ff9720640 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 0x562208087fb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00bae20 tx=0x7f6ff002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778045e6aedb2c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.847+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6fe00a1200 0x7f6fe00a36c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005c40 tx=0x7f6fec009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.847+0000 7f6fdf7fe640 1 -- 172.21.15.44:0/3081113289 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00046c0 con 0x56220807dc50 2024-09-13T17:31:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.847+0000 7f6fdf7fe640 1 -- 172.21.15.44:0/3081113289 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ff00ba5c0 con 0x56220807dc50 2024-09-13T17:31:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.847+0000 7f6fdf7fe640 1 -- 172.21.15.44:0/3081113289 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00b7370 con 0x56220807dc50 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/3081113289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 msgr2=0x562208087fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 0x562208087fb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00bae20 tx=0x7f6ff002a410 comp rx=0 tx=0).stop 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/3081113289 shutdown_connections 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622080884f0 0x56220808a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807dc50 0x562208087fb0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/3081113289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x56220807d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/3081113289 >> 172.21.15.44:0/3081113289 conn(0x5622080798c0 msgr2=0x562208079ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/3081113289 shutdown_connections 2024-09-13T17:31:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/3081113289 wait complete. 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 -- start start 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.848+0000 7f6ffe6a4740 1 Processor -- start 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807d330 0x562208087f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807dc50 0x5622080851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208085980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207f9fe80 con 0x5622080884f0 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207f9fb20 con 0x56220807d330 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207f65260 con 0x56220807dc50 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ffe6a4740 10 client.0 _ll_register_callbacks cb 0x562207f9adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208085980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208085980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47818/0 (socket says 172.21.15.44:47818) 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 -- 172.21.15.44:0/1958699178 learned_addr learned my addr 172.21.15.44:0/1958699178 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807dc50 0x5622080851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9720640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807d330 0x562208087f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 -- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807dc50 msgr2=0x5622080851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807dc50 0x5622080851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 -- 172.21.15.44:0/1958699178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807d330 msgr2=0x562208087f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807d330 0x562208087f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 -- 172.21.15.44:0/1958699178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff419b960 con 0x5622080884f0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807dc50 0x5622080851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208085980 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6fec011e60 tx=0x7f6fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca478e3f16a0a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6fdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1958699178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0047c0 con 0x5622080884f0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.849+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1958699178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6fec020460 con 0x5622080884f0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56220800a680 con 0x5622080884f0 2024-09-13T17:31:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1958699178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fec020620 con 0x5622080884f0 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 10 client.17121 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562207fe0330 con 0x5622080884f0 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1958699178 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56220800a680 con 0x5622080884f0 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1958699178 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6fec01e760 con 0x5622080884f0 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 10 client.17121 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 20 client.17121 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562208097de0 0x56220809a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 4 client.17121 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562207fd55d0 con 0x562208097de0 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.850+0000 7f6ff9720640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562208097de0 0x56220809a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.851+0000 7f6fdcff9640 10 client.17121 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.851+0000 7f6ff9720640 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562208097de0 0x56220809a200 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deea541958c8d104 server_cookie=8279477f46a6f916 in_seq=0 out_seq=0 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.851+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1958699178 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562207fd55d0 con 0x562208097de0 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.851+0000 7f6fdcff9640 10 client.17121 handle_command_reply: tid=0 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.851+0000 7f6ffe6a4740 1 client.17121 shutdown 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.851+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 msgr2=0x562208085980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208085980 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6fec011e60 tx=0x7f6fec004a40 comp rx=0 tx=0).stop 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 shutdown_connections 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562208097de0 0x56220809a200 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807dc50 0x5622080851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208085980 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1958699178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807d330 0x562208087f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 >> 172.21.15.44:0/1958699178 conn(0x5622080798c0 msgr2=0x562208078b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6fdcff9640 0 client.17121 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 shutdown_connections 2024-09-13T17:31:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1958699178 wait complete. 2024-09-13T17:31:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.852+0000 7f6ffe6a4740 20 client.17121 trim_cache size 0 max 16384 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.864+0000 7f6ffe6a4740 1 Processor -- start 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 -- start start 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x562208074a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x562208074f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208083aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220809f1e0 con 0x5622080884f0 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207fd55d0 con 0x56220807dc50 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220800a680 con 0x56220807d330 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208083aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208083aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47834/0 (socket says 172.21.15.44:47834) 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 -- 172.21.15.44:0/2596352337 learned_addr learned my addr 172.21.15.44:0/2596352337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x562208074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9720640 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x562208074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 -- 172.21.15.44:0/2596352337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 msgr2=0x562208074a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x562208074a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 -- 172.21.15.44:0/2596352337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 msgr2=0x562208074f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x562208074f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 -- 172.21.15.44:0/2596352337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562207f82300 con 0x5622080884f0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9720640 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x562208074a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.865+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208083aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009670 tx=0x7f6fec005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792cf5be08c10776 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6fdffff640 1 -- 172.21.15.44:0/2596352337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec01ed40 con 0x5622080884f0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6fdffff640 1 -- 172.21.15.44:0/2596352337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6fec01e890 con 0x5622080884f0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6fdffff640 1 -- 172.21.15.44:0/2596352337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0206c0 con 0x5622080884f0 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/2596352337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 msgr2=0x562208083aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208083aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009670 tx=0x7f6fec005f70 comp rx=0 tx=0).stop 2024-09-13T17:31:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/2596352337 shutdown_connections 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622080884f0 0x562208083aa0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x562208074f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/2596352337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220807d330 0x562208074a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/2596352337 >> 172.21.15.44:0/2596352337 conn(0x5622080798c0 msgr2=0x5622080804b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/2596352337 shutdown_connections 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.866+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/2596352337 wait complete. 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 -- start start 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 Processor -- start 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 0x56220808f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x56220808b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622080884f0 0x56220808bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220808c290 con 0x56220807d330 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220808c400 con 0x56220807dc50 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622080ee2e0 con 0x5622080884f0 2024-09-13T17:31:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ffe6a4740 10 client.0 _ll_register_callbacks cb 0x7f6ff00baa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ff9720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 0x56220808f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ff9720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 0x56220808f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47850/0 (socket says 172.21.15.44:47850) 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ff9720640 1 -- 172.21.15.44:0/1330370083 learned_addr learned my addr 172.21.15.44:0/1330370083 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x56220808b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.867+0000 7f6ff9f21640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622080884f0 0x56220808bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ff9720640 1 -- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622080884f0 msgr2=0x56220808bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ff9720640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622080884f0 0x56220808bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ff9720640 1 -- 172.21.15.44:0/1330370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 msgr2=0x56220808b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ff9720640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x56220808b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ff9720640 1 -- 172.21.15.44:0/1330370083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562207fe0330 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ff9720640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 0x56220808f200 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00eb7f0 tx=0x7f6ff00ed040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b239e33524029053 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6fdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1330370083 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00be9e0 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1330370083 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6ff00c3030 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ff00ba5c0 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1330370083 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ff00b7370 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 10 client.17127 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562207fa78e0 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1330370083 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ff00ba5c0 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1330370083 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ff004d070 con 0x56220807d330 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 10 client.17127 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 20 client.17127 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5622080ef3d0 0x5622080f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 4 client.17127 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:31:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.868+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562207cd5300 con 0x5622080ef3d0 2024-09-13T17:31:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.869+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5622080ef3d0 0x5622080f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.869+0000 7f6ff8f1f640 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5622080ef3d0 0x5622080f17c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64b11d9dbd92351 server_cookie=45b8fad34fd3c81b in_seq=0 out_seq=0 2024-09-13T17:31:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.869+0000 7f6fdcff9640 10 client.17127 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6fdcff9640 1 -- 172.21.15.44:0/1330370083 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562207cd5300 con 0x5622080ef3d0 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6fdcff9640 10 client.17127 handle_command_reply: tid=0 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 client.17127 shutdown 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 msgr2=0x56220808f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 0x56220808f200 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00eb7f0 tx=0x7f6ff00ed040 comp rx=0 tx=0).stop 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 shutdown_connections 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5622080ef3d0 0x5622080f17c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622080884f0 0x56220808bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6fdcff9640 0 client.17127 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56220807dc50 0x56220808b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 --2- 172.21.15.44:0/1330370083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220807d330 0x56220808f200 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 >> 172.21.15.44:0/1330370083 conn(0x5622080798c0 msgr2=0x5622080804b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 shutdown_connections 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 1 -- 172.21.15.44:0/1330370083 wait complete. 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.884+0000 7f6ffe6a4740 20 client.17127 trim_cache size 0 max 16384 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:31:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:31:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6fe00a1200 msgr2=0x7f6fe00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6fe00a1200 0x7f6fe00a36c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005c40 tx=0x7f6fec009500 comp rx=0 tx=0).stop 2024-09-13T17:31:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 msgr2=0x7f6ff41961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 0x7f6ff41961b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00057c0 tx=0x7f6ff0032c20 comp rx=0 tx=0).stop 2024-09-13T17:31:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 shutdown_connections 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6fe00a1200 0x7f6fe00a36c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff40681d0 0x7f6ff41966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4104ce0 0x7f6ff419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 --2- 172.21.15.44:0/2089315775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ff4067830 0x7f6ff41961b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 >> 172.21.15.44:0/2089315775 conn(0x7f6ff4074940 msgr2=0x7f6ff40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 shutdown_connections 2024-09-13T17:31:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:28.885+0000 7f6ffb984640 1 -- 172.21.15.44:0/2089315775 wait complete. 2024-09-13T17:31:30.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:29 smithi044 ceph-mon[31175]: pgmap v146: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:30.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:29 smithi070 ceph-mon[34502]: pgmap v146: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:30.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:29 smithi100 ceph-mon[35055]: pgmap v146: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:32.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:31 smithi044 ceph-mon[31175]: pgmap v147: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-13T17:31:32.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:31 smithi070 ceph-mon[34502]: pgmap v147: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-13T17:31:32.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:31 smithi100 ceph-mon[35055]: pgmap v147: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-13T17:31:34.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:33 smithi044 ceph-mon[31175]: pgmap v148: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-13T17:31:34.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:33 smithi070 ceph-mon[34502]: pgmap v148: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-13T17:31:34.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:33 smithi100 ceph-mon[35055]: pgmap v148: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-13T17:31:36.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:35 smithi044 ceph-mon[31175]: pgmap v149: 129 pgs: 129 active+clean; 2.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:36.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:35 smithi070 ceph-mon[34502]: pgmap v149: 129 pgs: 129 active+clean; 2.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:36.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:35 smithi100 ceph-mon[35055]: pgmap v149: 129 pgs: 129 active+clean; 2.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:38.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:37 smithi044 ceph-mon[31175]: pgmap v150: 129 pgs: 129 active+clean; 2.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-13T17:31:38.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:37 smithi070 ceph-mon[34502]: pgmap v150: 129 pgs: 129 active+clean; 2.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-13T17:31:38.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:37 smithi100 ceph-mon[35055]: pgmap v150: 129 pgs: 129 active+clean; 2.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-13T17:31:40.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:39 smithi044 ceph-mon[31175]: pgmap v151: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:40.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:39 smithi070 ceph-mon[34502]: pgmap v151: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:40.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:39 smithi100 ceph-mon[35055]: pgmap v151: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:31:40.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:31:41.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.071+0000 7f5befb8a640 1 -- 172.21.15.44:0/2147981614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be810c600 msgr2=0x7f5be810e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.071+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2147981614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be810c600 0x7f5be810e9f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009ca0 tx=0x7f5be002a8c0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 -- 172.21.15.44:0/2147981614 shutdown_connections 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2147981614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 0x7f5be810c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2147981614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be810c600 0x7f5be810e9f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2147981614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80fea50 0x7f5be80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 -- 172.21.15.44:0/2147981614 >> 172.21.15.44:0/2147981614 conn(0x7f5be80fa8a0 msgr2=0x7f5be80fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 -- 172.21.15.44:0/2147981614 shutdown_connections 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 -- 172.21.15.44:0/2147981614 wait complete. 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 Processor -- start 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 -- start start 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 0x7f5be8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.072+0000 7f5befb8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 0x7f5be81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5befb8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be810c600 0x7f5be819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5befb8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819d2a0 con 0x7f5be80fea50 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5befb8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819d410 con 0x7f5be810c600 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5befb8a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819d5d0 con 0x7f5be80ff370 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 0x7f5be8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 0x7f5be8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34270/0 (socket says 172.21.15.44:34270) 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 -- 172.21.15.44:0/2633330871 learned_addr learned my addr 172.21.15.44:0/2633330871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed125640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 0x7f5be81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 -- 172.21.15.44:0/2633330871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 msgr2=0x7f5be81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 0x7f5be81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 -- 172.21.15.44:0/2633330871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be810c600 msgr2=0x7f5be819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bee127640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be810c600 0x7f5be819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be810c600 0x7f5be819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 -- 172.21.15.44:0/2633330871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0009900 con 0x7f5be80fea50 2024-09-13T17:31:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed125640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 0x7f5be81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bee127640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be810c600 0x7f5be819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bed926640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 0x7f5be8198370 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5be400d950 tx=0x7f5be400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bdeffd640 1 -- 172.21.15.44:0/2633330871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4009c30 con 0x7f5be80fea50 2024-09-13T17:31:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be819d8b0 con 0x7f5be80fea50 2024-09-13T17:31:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bdeffd640 1 -- 172.21.15.44:0/2633330871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5be4009d90 con 0x7f5be80fea50 2024-09-13T17:31:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5bdeffd640 1 -- 172.21.15.44:0/2633330871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be400f620 con 0x7f5be80fea50 2024-09-13T17:31:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.073+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be81a5130 con 0x7f5be80fea50 2024-09-13T17:31:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bdeffd640 1 -- 172.21.15.44:0/2633330871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5be40109f0 con 0x7f5be80fea50 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 Processor -- start 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 -- start start 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d32e10 0x557378d331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378c9df20 con 0x557378d33730 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378c9b6b0 con 0x557378d3dfd0 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.076+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378c96fa0 con 0x557378d32e10 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bdeffd640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5bd40a1200 0x7f5bd40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34286/0 (socket says 172.21.15.44:34286) 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 -- 172.21.15.44:0/2691655127 learned_addr learned my addr 172.21.15.44:0/2691655127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bee127640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d32e10 0x557378d331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed125640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bee127640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5bd40a1200 0x7f5bd40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bdeffd640 1 -- 172.21.15.44:0/2633330871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5be40cccb0 con 0x7f5be80fea50 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 -- 172.21.15.44:0/2691655127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d32e10 msgr2=0x557378d331f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d32e10 0x557378d331f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 -- 172.21.15.44:0/2691655127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 msgr2=0x557378d403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 -- 172.21.15.44:0/2691655127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557378c51840 con 0x557378d33730 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bee127640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d32e10 0x557378d331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed125640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bed926640 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3da90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5be40a19f0 tx=0x7f5be400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bfd88381418f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bd3fff640 1 -- 172.21.15.44:0/2691655127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be409aab0 con 0x557378d33730 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bd3fff640 1 -- 172.21.15.44:0/2691655127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5be4099aa0 con 0x557378d33730 2024-09-13T17:31:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bd3fff640 1 -- 172.21.15.44:0/2691655127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40ca4c0 con 0x557378d33730 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.077+0000 7f5bee127640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5bd40a1200 0x7f5bd40a36c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5be0031040 tx=0x7f5be00095c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2691655127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 msgr2=0x557378d3da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3da90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5be40a19f0 tx=0x7f5be400bf60 comp rx=0 tx=0).stop 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2691655127 shutdown_connections 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3da90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2691655127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d32e10 0x557378d331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2691655127 >> 172.21.15.44:0/2691655127 conn(0x557378d2f380 msgr2=0x557378d2f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2691655127 shutdown_connections 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.078+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2691655127 wait complete. 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 -- start start 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 Processor -- start 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d3d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d33730 0x557378d3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d3b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378c382a0 con 0x557378d32e10 2024-09-13T17:31:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737896f430 con 0x557378d3dfd0 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378c1a680 con 0x557378d33730 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bf28b0740 10 client.0 _ll_register_callbacks cb 0x557378c90610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d3d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d3d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34300/0 (socket says 172.21.15.44:34300) 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.079+0000 7f5bed926640 1 -- 172.21.15.44:0/2069180900 learned_addr learned my addr 172.21.15.44:0/2069180900 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bee127640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d3b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed125640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d33730 0x557378d3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed926640 1 -- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d33730 msgr2=0x557378d3ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed926640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d33730 0x557378d3ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed926640 1 -- 172.21.15.44:0/2069180900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 msgr2=0x557378d3b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed926640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d3b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed926640 1 -- 172.21.15.44:0/2069180900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be81a5130 con 0x557378d32e10 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed125640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d33730 0x557378d3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bed926640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d3d9e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5be409a100 tx=0x7f5be409a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3e9389799661e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2069180900 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40b2200 con 0x557378d32e10 2024-09-13T17:31:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2069180900 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5be40b2360 con 0x557378d32e10 2024-09-13T17:31:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55737898a4e0 con 0x557378d32e10 2024-09-13T17:31:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2069180900 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be40a1a90 con 0x557378d32e10 2024-09-13T17:31:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bf28b0740 10 client.17139 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557378c95620 con 0x557378d32e10 2024-09-13T17:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.080+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2069180900 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5be4063020 con 0x557378d32e10 2024-09-13T17:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2069180900 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5be4098be0 con 0x557378d32e10 2024-09-13T17:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bf28b0740 10 client.17139 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bf28b0740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bf28b0740 20 client.17139 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378d4d950 0x557378d4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bf28b0740 4 client.17139 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55737899b7e0 con 0x557378d4d950 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bed125640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378d4d950 0x557378d4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bd17fa640 10 client.17139 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.081+0000 7f5bed125640 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378d4d950 0x557378d4fd40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ca97a6222b174 server_cookie=4545834818471877 in_seq=0 out_seq=0 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2069180900 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55737899b7e0 con 0x557378d4d950 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bd17fa640 10 client.17139 handle_command_reply: tid=0 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 client.17139 shutdown 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 msgr2=0x557378d3d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d3d9e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5be409a100 tx=0x7f5be409a2e0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 shutdown_connections 2024-09-13T17:31:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378d4d950 0x557378d4fd40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d33730 0x557378d3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d3dfd0 0x557378d3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2069180900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d3d9e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bd17fa640 0 client.17139 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 >> 172.21.15.44:0/2069180900 conn(0x557378d2f380 msgr2=0x557378d2e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 shutdown_connections 2024-09-13T17:31:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2069180900 wait complete. 2024-09-13T17:31:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.082+0000 7f5bf28b0740 20 client.17139 trim_cache size 0 max 16384 2024-09-13T17:31:41.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 Processor -- start 2024-09-13T17:31:41.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 -- start start 2024-09-13T17:31:41.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d32e10 0x557378d3d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d3ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378d33ca0 con 0x557378d33730 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378d4ab60 con 0x557378d32e10 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378d2d9c0 con 0x557378d3dfd0 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34314/0 (socket says 172.21.15.44:34314) 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 -- 172.21.15.44:0/2245912102 learned_addr learned my addr 172.21.15.44:0/2245912102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed926640 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d32e10 0x557378d3d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 -- 172.21.15.44:0/2245912102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 msgr2=0x557378d3ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d3ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 -- 172.21.15.44:0/2245912102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d32e10 msgr2=0x557378d3d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d32e10 0x557378d3d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 -- 172.21.15.44:0/2245912102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557378c37790 con 0x557378d33730 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed926640 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d32e10 0x557378d3d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.097+0000 7f5bed125640 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3a620 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x557378d359b0 tx=0x7f5bd8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0dba67ec5e8d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bdce05640 1 -- 172.21.15.44:0/2245912102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8017040 con 0x557378d33730 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bdce05640 1 -- 172.21.15.44:0/2245912102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bd80044b0 con 0x557378d33730 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bdce05640 1 -- 172.21.15.44:0/2245912102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8005020 con 0x557378d33730 2024-09-13T17:31:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2245912102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 msgr2=0x557378d3a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3a620 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x557378d359b0 tx=0x7f5bd8009620 comp rx=0 tx=0).stop 2024-09-13T17:31:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2245912102 shutdown_connections 2024-09-13T17:31:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d3ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d33730 0x557378d3a620 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2245912102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d32e10 0x557378d3d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2245912102 >> 172.21.15.44:0/2245912102 conn(0x557378d2f380 msgr2=0x557378d39630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2245912102 shutdown_connections 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2245912102 wait complete. 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.098+0000 7f5bf28b0740 1 -- start start 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 Processor -- start 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d44fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d33730 0x557378d41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378d420c0 con 0x557378d32e10 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378d42230 con 0x557378d33730 2024-09-13T17:31:41.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378da3dc0 con 0x557378d3dfd0 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bf28b0740 10 client.0 _ll_register_callbacks cb 0x7f5be4010ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d44fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d44fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34318/0 (socket says 172.21.15.44:34318) 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 -- 172.21.15.44:0/2500094189 learned_addr learned my addr 172.21.15.44:0/2500094189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed125640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d33730 0x557378d41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bee127640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 -- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 msgr2=0x557378d419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 -- 172.21.15.44:0/2500094189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d33730 msgr2=0x557378d41480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d33730 0x557378d41480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed926640 1 -- 172.21.15.44:0/2500094189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd80093b0 con 0x557378d32e10 2024-09-13T17:31:41.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.099+0000 7f5bed125640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d33730 0x557378d41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bed926640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d44fe0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5be4096510 tx=0x7f5be40ccfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362a08ffaac254dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2500094189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40a1c90 con 0x557378d32e10 2024-09-13T17:31:41.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2500094189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f5be4064080 con 0x557378d32e10 2024-09-13T17:31:41.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5be4099aa0 con 0x557378d32e10 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2500094189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be4099400 con 0x557378d32e10 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 10 client.17145 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557378c5cea0 con 0x557378d32e10 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2500094189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5be4099aa0 con 0x557378d32e10 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2500094189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5be4078b80 con 0x557378d32e10 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 10 client.17145 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 20 client.17145 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378da4f10 0x557378da7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 4 client.17145 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:31:41.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.100+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573789eee30 con 0x557378da4f10 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bed125640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378da4f10 0x557378da7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bed125640 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378da4f10 0x557378da7300 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69debd8d1563a699 server_cookie=ba5f8b18c3cecc97 in_seq=0 out_seq=0 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bd17fa640 10 client.17145 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bd17fa640 1 -- 172.21.15.44:0/2500094189 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5573789eee30 con 0x557378da4f10 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bd17fa640 10 client.17145 handle_command_reply: tid=0 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 client.17145 shutdown 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 msgr2=0x557378d44fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d44fe0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5be4096510 tx=0x7f5be40ccfe0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 shutdown_connections 2024-09-13T17:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557378da4f10 0x557378da7300 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557378d3dfd0 0x557378d419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557378d33730 0x557378d41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bd17fa640 0 client.17145 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 --2- 172.21.15.44:0/2500094189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557378d32e10 0x557378d44fe0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 >> 172.21.15.44:0/2500094189 conn(0x557378d2f380 msgr2=0x557378d3a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.101+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 shutdown_connections 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5bf28b0740 1 -- 172.21.15.44:0/2500094189 wait complete. 2024-09-13T17:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5bf28b0740 20 client.17145 trim_cache size 0 max 16384 2024-09-13T17:31:41.123 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:41.123 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:31:41.123 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:31:41.123 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:41.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5bd40a1200 msgr2=0x7f5bd40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5bd40a1200 0x7f5bd40a36c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5be0031040 tx=0x7f5be00095c0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 msgr2=0x7f5be8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 0x7f5be8198370 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5be400d950 tx=0x7f5be400de10 comp rx=0 tx=0).stop 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 shutdown_connections 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5bd40a1200 0x7f5bd40a36c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80ff370 0x7f5be81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be810c600 0x7f5be819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 --2- 172.21.15.44:0/2633330871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5be80fea50 0x7f5be8198370 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 >> 172.21.15.44:0/2633330871 conn(0x7f5be80fa8a0 msgr2=0x7f5be80fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 shutdown_connections 2024-09-13T17:31:41.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.102+0000 7f5befb8a640 1 -- 172.21.15.44:0/2633330871 wait complete. 2024-09-13T17:31:41.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:31:41.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:31:41.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 -- 172.21.15.44:0/3921843993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a8410a370 msgr2=0x7f2a8410c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 --2- 172.21.15.44:0/3921843993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a8410a370 0x7f2a8410c7c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009ca0 tx=0x7f2a7c02a950 comp rx=0 tx=0).stop 2024-09-13T17:31:41.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 -- 172.21.15.44:0/3921843993 shutdown_connections 2024-09-13T17:31:41.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 --2- 172.21.15.44:0/3921843993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a841018c0 0x7f2a84109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 --2- 172.21.15.44:0/3921843993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a8410a370 0x7f2a8410c7c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 --2- 172.21.15.44:0/3921843993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a84100fa0 0x7f2a84101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 -- 172.21.15.44:0/3921843993 >> 172.21.15.44:0/3921843993 conn(0x7f2a84075c60 msgr2=0x7f2a840fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 -- 172.21.15.44:0/3921843993 shutdown_connections 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 -- 172.21.15.44:0/3921843993 wait complete. 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 Processor -- start 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.227+0000 7f2a89be9640 1 -- start start 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a84100fa0 0x7f2a84196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 0x7f2a841966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8410a370 0x7f2a8419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8419b0c0 con 0x7f2a841018c0 2024-09-13T17:31:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8419b230 con 0x7f2a84100fa0 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8419b3f0 con 0x7f2a8410a370 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 0x7f2a841966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 0x7f2a841966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34330/0 (socket says 172.21.15.44:34330) 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 -- 172.21.15.44:0/4257923348 learned_addr learned my addr 172.21.15.44:0/4257923348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a83fff640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8410a370 0x7f2a8419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a837fe640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a84100fa0 0x7f2a84196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 -- 172.21.15.44:0/4257923348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8410a370 msgr2=0x7f2a8419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8410a370 0x7f2a8419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 -- 172.21.15.44:0/4257923348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a84100fa0 msgr2=0x7f2a84196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a84100fa0 0x7f2a84196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 -- 172.21.15.44:0/4257923348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c009900 con 0x7f2a841018c0 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a83fff640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8410a370 0x7f2a8419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a82ffd640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 0x7f2a841966d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9f0 tx=0x7f2a7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78009720 con 0x7f2a841018c0 2024-09-13T17:31:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a7800cd80 con 0x7f2a841018c0 2024-09-13T17:31:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8419b6d0 con 0x7f2a841018c0 2024-09-13T17:31:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.228+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78010430 con 0x7f2a841018c0 2024-09-13T17:31:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.229+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841a2f50 con 0x7f2a841018c0 2024-09-13T17:31:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.229+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a84067dc0 con 0x7f2a841018c0 2024-09-13T17:31:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.231+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2a78004fb0 con 0x7f2a841018c0 2024-09-13T17:31:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.231+0000 7f2a80ff9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2a6c0a1280 0x7f2a6c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.231+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f2a78014070 con 0x7f2a841018c0 2024-09-13T17:31:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.231+0000 7f2a837fe640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2a6c0a1280 0x7f2a6c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.232+0000 7f2a837fe640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2a6c0a1280 0x7f2a6c0a3740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a74005fd0 tx=0x7f2a7400b590 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.235+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2a78096510 con 0x7f2a841018c0 2024-09-13T17:31:41.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.444+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a84101380 con 0x7f2a841018c0 2024-09-13T17:31:41.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.444+0000 7f2a80ff9640 1 -- 172.21.15.44:0/4257923348 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f2a78099b30 con 0x7f2a841018c0 2024-09-13T17:31:41.446 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:31:41.447 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:31:41.447 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2a6c0a1280 msgr2=0x7f2a6c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2a6c0a1280 0x7f2a6c0a3740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a74005fd0 tx=0x7f2a7400b590 comp rx=0 tx=0).stop 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 msgr2=0x7f2a841966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 0x7f2a841966d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9f0 tx=0x7f2a7800edf0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 shutdown_connections 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2a6c0a1280 0x7f2a6c0a3740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8410a370 0x7f2a8419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.446+0000 7f2a89be9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a841018c0 0x7f2a841966d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.447+0000 7f2a89be9640 1 --2- 172.21.15.44:0/4257923348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a84100fa0 0x7f2a84196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.447+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 >> 172.21.15.44:0/4257923348 conn(0x7f2a84075c60 msgr2=0x7f2a840fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.447+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 shutdown_connections 2024-09-13T17:31:41.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.447+0000 7f2a89be9640 1 -- 172.21.15.44:0/4257923348 wait complete. 2024-09-13T17:31:41.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 -- 172.21.15.44:0/886567722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc641050e0 msgr2=0x7efc6410cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/886567722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc641050e0 0x7efc6410cc30 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009cd0 tx=0x7efc5c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 -- 172.21.15.44:0/886567722 shutdown_connections 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/886567722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc641050e0 0x7efc6410cc30 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/886567722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64100940 0x7efc64104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/886567722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc640fff90 0x7efc64100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 -- 172.21.15.44:0/886567722 >> 172.21.15.44:0/886567722 conn(0x7efc640fba20 msgr2=0x7efc640fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 -- 172.21.15.44:0/886567722 shutdown_connections 2024-09-13T17:31:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.562+0000 7efc6ad3e640 1 -- 172.21.15.44:0/886567722 wait complete. 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 Processor -- start 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 -- start start 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640fff90 0x7efc640727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc641050e0 0x7efc64072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 0x7efc641a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641a4940 con 0x7efc64073230 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641a4ab0 con 0x7efc640fff90 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc6ad3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641a4c70 con 0x7efc641050e0 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 0x7efc641a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 0x7efc641a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34346/0 (socket says 172.21.15.44:34346) 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 -- 172.21.15.44:0/1398123825 learned_addr learned my addr 172.21.15.44:0/1398123825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc68ada640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640fff90 0x7efc640727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc63fff640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc641050e0 0x7efc64072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 -- 172.21.15.44:0/1398123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc641050e0 msgr2=0x7efc64072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc641050e0 0x7efc64072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 -- 172.21.15.44:0/1398123825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640fff90 msgr2=0x7efc640727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640fff90 0x7efc640727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 -- 172.21.15.44:0/1398123825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c009900 con 0x7efc64073230 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc63fff640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc641050e0 0x7efc64072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.563+0000 7efc692db640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 0x7efc641a43d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efc5c004af0 tx=0x7efc5c030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.564+0000 7efc61ffb640 1 -- 172.21.15.44:0/1398123825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c031b80 con 0x7efc64073230 2024-09-13T17:31:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.564+0000 7efc61ffb640 1 -- 172.21.15.44:0/1398123825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc5c003e00 con 0x7efc64073230 2024-09-13T17:31:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.564+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc641a4ef0 con 0x7efc64073230 2024-09-13T17:31:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.564+0000 7efc61ffb640 1 -- 172.21.15.44:0/1398123825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c045410 con 0x7efc64073230 2024-09-13T17:31:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.564+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc641a5420 con 0x7efc64073230 2024-09-13T17:31:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 Processor -- start 2024-09-13T17:31:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc61ffb640 1 -- 172.21.15.44:0/1398123825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc5c00a040 con 0x7efc64073230 2024-09-13T17:31:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 -- start start 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb7790 0x557d03eb7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 0x557d03ec4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03e21cc0 con 0x557d03ec2950 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03e1f450 con 0x557d03eb80b0 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc6da5e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03e1ad40 con 0x557d03eb7790 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc63fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 0x557d03ec4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc63fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 0x557d03ec4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34354/0 (socket says 172.21.15.44:34354) 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.567+0000 7efc63fff640 1 -- 172.21.15.44:0/3227190953 learned_addr learned my addr 172.21.15.44:0/3227190953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc68ada640 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc61ffb640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc540a1280 0x7efc540a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb7790 0x557d03eb7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 -- 172.21.15.44:0/3227190953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb7790 msgr2=0x557d03eb7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb7790 0x557d03eb7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc692db640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc540a1280 0x7efc540a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 -- 172.21.15.44:0/3227190953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 msgr2=0x557d03ec2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 -- 172.21.15.44:0/3227190953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d03de2620 con 0x557d03ec2950 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc68ada640 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc61ffb640 1 -- 172.21.15.44:0/1398123825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efc5c0ee350 con 0x7efc64073230 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc63fff640 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 0x557d03ec4d40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efc50009f90 tx=0x7efc50009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c53c5fb4e88b4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc692db640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc540a1280 0x7efc540a3740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0ebd80 tx=0x7efc5c0377e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc4effd640 1 -- 172.21.15.44:0/3227190953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50010710 con 0x557d03ec2950 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc4effd640 1 -- 172.21.15.44:0/3227190953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc50010d00 con 0x557d03ec2950 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.568+0000 7efc4effd640 1 -- 172.21.15.44:0/3227190953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001db90 con 0x557d03ec2950 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 -- 172.21.15.44:0/3227190953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 msgr2=0x557d03ec4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 0x557d03ec4d40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efc50009f90 tx=0x7efc50009730 comp rx=0 tx=0).stop 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 -- 172.21.15.44:0/3227190953 shutdown_connections 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03ec2950 0x557d03ec4d40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 --2- 172.21.15.44:0/3227190953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb7790 0x557d03eb7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 -- 172.21.15.44:0/3227190953 >> 172.21.15.44:0/3227190953 conn(0x557d03eb3d00 msgr2=0x557d03eb40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 -- 172.21.15.44:0/3227190953 shutdown_connections 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 -- 172.21.15.44:0/3227190953 wait complete. 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 -- start start 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.569+0000 7efc6da5e740 1 Processor -- start 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb7790 0x557d03ec1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 0x557d03ebf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ebf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03dd9180 con 0x557d03eb80b0 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03af3430 con 0x557d03eb7790 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03d9e5e0 con 0x557d03ec2950 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 10 client.0 _ll_register_callbacks cb 0x557d03e0aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 0x557d03ebf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 0x557d03ebf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34364/0 (socket says 172.21.15.44:34364) 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 -- 172.21.15.44:0/532547655 learned_addr learned my addr 172.21.15.44:0/532547655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc63fff640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ebf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 -- 172.21.15.44:0/532547655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 msgr2=0x557d03ebf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc63fff640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb7790 0x557d03ec1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ebf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 -- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb7790 msgr2=0x557d03ec1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb7790 0x557d03ec1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 -- 172.21.15.44:0/532547655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50009cb0 con 0x557d03eb80b0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc63fff640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb7790 0x557d03ec1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc68ada640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 0x557d03ebf250 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efc5800b9a0 tx=0x7efc5800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ae69bc9087f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc43fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc43fff640 1 -- 172.21.15.44:0/532547655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5801b640 con 0x557d03eb80b0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc43fff640 1 -- 172.21.15.44:0/532547655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efc5801b7a0 con 0x557d03eb80b0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.570+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d03e44b80 con 0x557d03eb80b0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc43fff640 1 -- 172.21.15.44:0/532547655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc58003ea0 con 0x557d03eb80b0 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 10 client.17163 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557d03e193c0 con 0x557d03eb80b0 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc43fff640 1 -- 172.21.15.44:0/532547655 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557d03e44b80 con 0x557d03eb80b0 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc43fff640 1 -- 172.21.15.44:0/532547655 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc58011a80 con 0x557d03eb80b0 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 10 client.17163 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 10 client.17163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 20 client.17163 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03ed2300 0x557d03ed4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 4 client.17163 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.571+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d03e0d1f0 con 0x557d03ed2300 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.572+0000 7efc63fff640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03ed2300 0x557d03ed4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.572+0000 7efc43fff640 10 client.17163 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.572+0000 7efc63fff640 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03ed2300 0x557d03ed4720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64ede8786abc4a75 server_cookie=468c2ae09e76955b in_seq=0 out_seq=0 2024-09-13T17:31:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc43fff640 1 -- 172.21.15.44:0/532547655 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557d03e0d1f0 con 0x557d03ed2300 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc43fff640 10 client.17163 handle_command_reply: tid=0 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 client.17163 shutdown 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 msgr2=0x557d03ebf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 0x557d03ebf250 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efc5800b9a0 tx=0x7efc5800be60 comp rx=0 tx=0).stop 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 shutdown_connections 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03ed2300 0x557d03ed4720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ebf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc43fff640 0 client.17163 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb80b0 0x557d03ebf250 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 --2- 172.21.15.44:0/532547655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb7790 0x557d03ec1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 >> 172.21.15.44:0/532547655 conn(0x557d03eb3d00 msgr2=0x557d03eb9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 shutdown_connections 2024-09-13T17:31:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 1 -- 172.21.15.44:0/532547655 wait complete. 2024-09-13T17:31:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.573+0000 7efc6da5e740 20 client.17163 trim_cache size 0 max 16384 2024-09-13T17:31:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 Processor -- start 2024-09-13T17:31:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 -- start start 2024-09-13T17:31:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03eb36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ec9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03ed0bd0 con 0x557d03eb7790 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03ec5400 con 0x557d03eb80b0 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc6da5e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03ebfb00 con 0x557d03ec2950 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc63fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03eb36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc63fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03eb36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34372/0 (socket says 172.21.15.44:34372) 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.584+0000 7efc63fff640 1 -- 172.21.15.44:0/2424135064 learned_addr learned my addr 172.21.15.44:0/2424135064 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc68ada640 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc63fff640 1 -- 172.21.15.44:0/2424135064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 msgr2=0x557d03ec9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc63fff640 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ec9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc63fff640 1 -- 172.21.15.44:0/2424135064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 msgr2=0x557d03ec9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc63fff640 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc63fff640 1 -- 172.21.15.44:0/2424135064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d03dbb770 con 0x557d03eb7790 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc68ada640 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc63fff640 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03eb36a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x557d03eba330 tx=0x7efc50009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6251e9bdd64b365f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc4f7fe640 1 -- 172.21.15.44:0/2424135064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001db10 con 0x557d03eb7790 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc4f7fe640 1 -- 172.21.15.44:0/2424135064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc5001d6f0 con 0x557d03eb7790 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc4f7fe640 1 -- 172.21.15.44:0/2424135064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50010770 con 0x557d03eb7790 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc6da5e740 1 -- 172.21.15.44:0/2424135064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 msgr2=0x557d03eb36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.585+0000 7efc6da5e740 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03eb36a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x557d03eba330 tx=0x7efc50009730 comp rx=0 tx=0).stop 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- 172.21.15.44:0/2424135064 shutdown_connections 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03ec2950 0x557d03ec9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03eb80b0 0x557d03ec9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 --2- 172.21.15.44:0/2424135064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03eb36a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- 172.21.15.44:0/2424135064 >> 172.21.15.44:0/2424135064 conn(0x557d03eb3d00 msgr2=0x557d03ed90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- 172.21.15.44:0/2424135064 shutdown_connections 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- 172.21.15.44:0/2424135064 wait complete. 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- start start 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 Processor -- start 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03ec97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb80b0 0x557d03ec5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03ec2950 0x557d03ec6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03ec6690 con 0x557d03eb7790 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03ec6800 con 0x557d03ec2950 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d03ec69c0 con 0x557d03eb80b0 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc6da5e740 10 client.0 _ll_register_callbacks cb 0x7efc50009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.586+0000 7efc63fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03ec97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03ec97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34380/0 (socket says 172.21.15.44:34380) 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 -- 172.21.15.44:0/1741999069 learned_addr learned my addr 172.21.15.44:0/1741999069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc68ada640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb80b0 0x557d03ec5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 -- 172.21.15.44:0/1741999069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb80b0 msgr2=0x557d03ec5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb80b0 0x557d03ec5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 -- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03ec2950 msgr2=0x557d03ec6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03ec2950 0x557d03ec6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 -- 172.21.15.44:0/1741999069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50022040 con 0x557d03eb7790 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc68ada640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb80b0 0x557d03ec5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc63fff640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03ec97b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efc5001d4d0 tx=0x7efc5001df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1134cff6ffe77b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc43fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc43fff640 1 -- 172.21.15.44:0/1741999069 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001dbe0 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc43fff640 1 -- 172.21.15.44:0/1741999069 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7efc50000e40 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc50010d00 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc43fff640 1 -- 172.21.15.44:0/1741999069 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc50005030 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc6da5e740 10 client.17169 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557d03de0e80 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.587+0000 7efc43fff640 1 -- 172.21.15.44:0/1741999069 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc50010d00 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc43fff640 1 -- 172.21.15.44:0/1741999069 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc5002f220 con 0x557d03eb7790 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc6da5e740 10 client.17169 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc6da5e740 10 client.17169 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc6da5e740 20 client.17169 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc6da5e740 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03f297f0 0x557d03f2bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc6da5e740 4 client.17169 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d03b0e4e0 con 0x557d03f297f0 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.588+0000 7efc68ada640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03f297f0 0x557d03f2bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc43fff640 10 client.17169 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc68ada640 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03f297f0 0x557d03f2bbe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11cec778f5d8e7e5 server_cookie=cac60e1376776c9f in_seq=0 out_seq=0 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc43fff640 1 -- 172.21.15.44:0/1741999069 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d03b0e4e0 con 0x557d03f297f0 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc43fff640 10 client.17169 handle_command_reply: tid=0 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 client.17169 shutdown 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 msgr2=0x557d03ec97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03ec97b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efc5001d4d0 tx=0x7efc5001df10 comp rx=0 tx=0).stop 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 shutdown_connections 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x557d03f297f0 0x557d03f2bbe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d03eb80b0 0x557d03ec5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d03ec2950 0x557d03ec6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc43fff640 0 client.17169 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 --2- 172.21.15.44:0/1741999069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d03eb7790 0x557d03ec97b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 >> 172.21.15.44:0/1741999069 conn(0x557d03eb3d00 msgr2=0x557d03ebdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 shutdown_connections 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 1 -- 172.21.15.44:0/1741999069 wait complete. 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.589+0000 7efc6da5e740 20 client.17169 trim_cache size 0 max 16384 2024-09-13T17:31:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:41.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc540a1280 msgr2=0x7efc540a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc540a1280 0x7efc540a3740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0ebd80 tx=0x7efc5c0377e0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 msgr2=0x7efc641a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 0x7efc641a43d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efc5c004af0 tx=0x7efc5c030e50 comp rx=0 tx=0).stop 2024-09-13T17:31:41.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 shutdown_connections 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7efc540a1280 0x7efc540a3740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc641050e0 0x7efc64072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc64073230 0x7efc641a43d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 --2- 172.21.15.44:0/1398123825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640fff90 0x7efc640727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 >> 172.21.15.44:0/1398123825 conn(0x7efc640fba20 msgr2=0x7efc640fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 shutdown_connections 2024-09-13T17:31:41.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.590+0000 7efc6ad3e640 1 -- 172.21.15.44:0/1398123825 wait complete. 2024-09-13T17:31:41.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:31:41.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 -- 172.21.15.44:0/1143138089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32480fee80 msgr2=0x7f32480ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 --2- 172.21.15.44:0/1143138089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32480fee80 0x7f32480ff260 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3238009930 tx=0x7f323802a420 comp rx=0 tx=0).stop 2024-09-13T17:31:41.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 -- 172.21.15.44:0/1143138089 shutdown_connections 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 --2- 172.21.15.44:0/1143138089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3248103fd0 0x7f324810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 --2- 172.21.15.44:0/1143138089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32480ff830 0x7f3248103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 --2- 172.21.15.44:0/1143138089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32480fee80 0x7f32480ff260 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 -- 172.21.15.44:0/1143138089 >> 172.21.15.44:0/1143138089 conn(0x7f32480fa8d0 msgr2=0x7f32480fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 -- 172.21.15.44:0/1143138089 shutdown_connections 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 -- 172.21.15.44:0/1143138089 wait complete. 2024-09-13T17:31:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.705+0000 7f324d93e640 1 Processor -- start 2024-09-13T17:31:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 -- start start 2024-09-13T17:31:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32480fee80 0x7f3248100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 0x7f3248100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3248101390 0x7f32481a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32481a4960 con 0x7f3248103fd0 2024-09-13T17:31:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32481a4ad0 con 0x7f3248101390 2024-09-13T17:31:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f324d93e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32481a4c90 con 0x7f32480fee80 2024-09-13T17:31:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 0x7f3248100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 0x7f3248100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34400/0 (socket says 172.21.15.44:34400) 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 -- 172.21.15.44:0/3309377375 learned_addr learned my addr 172.21.15.44:0/3309377375 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f3246ffd640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32480fee80 0x7f3248100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32477fe640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3248101390 0x7f32481a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 -- 172.21.15.44:0/3309377375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32480fee80 msgr2=0x7f3248100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32480fee80 0x7f3248100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 -- 172.21.15.44:0/3309377375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3248101390 msgr2=0x7f32481a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3248101390 0x7f32481a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.706+0000 7f32467fc640 1 -- 172.21.15.44:0/3309377375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238009560 con 0x7f3248103fd0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.707+0000 7f32467fc640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 0x7f3248100e50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c970 tx=0x7f323c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.707+0000 7f324c93c640 1 -- 172.21.15.44:0/3309377375 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c004440 con 0x7f3248103fd0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.707+0000 7f324c93c640 1 -- 172.21.15.44:0/3309377375 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f323c007cf0 con 0x7f3248103fd0 2024-09-13T17:31:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.707+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32481a4f70 con 0x7f3248103fd0 2024-09-13T17:31:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.707+0000 7f324c93c640 1 -- 172.21.15.44:0/3309377375 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00f450 con 0x7f3248103fd0 2024-09-13T17:31:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.707+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32481a53f0 con 0x7f3248103fd0 2024-09-13T17:31:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.709+0000 7f324c93c640 1 -- 172.21.15.44:0/3309377375 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f323c007860 con 0x7f3248103fd0 2024-09-13T17:31:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.709+0000 7f325065e740 1 Processor -- start 2024-09-13T17:31:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.709+0000 7f325065e740 1 -- start start 2024-09-13T17:31:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f325065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f325065e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f325065e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f325065e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc302dcc0 con 0x55dfc30ce950 2024-09-13T17:31:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f325065e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc302b450 con 0x55dfc30c40b0 2024-09-13T17:31:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f325065e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc3026d40 con 0x55dfc30c3790 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34416/0 (socket says 172.21.15.44:34416) 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 -- 172.21.15.44:0/3467622021 learned_addr learned my addr 172.21.15.44:0/3467622021 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f324c93c640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f32300a1200 0x7f32300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f3246ffd640 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 -- 172.21.15.44:0/3467622021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 msgr2=0x55dfc30c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 -- 172.21.15.44:0/3467622021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 msgr2=0x55dfc30ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 -- 172.21.15.44:0/3467622021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc2fee620 con 0x55dfc30ce950 2024-09-13T17:31:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32477fe640 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32477fe640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f32300a1200 0x7f32300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f324c93c640 1 -- 172.21.15.44:0/3309377375 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f323c0cd0a0 con 0x7f3248103fd0 2024-09-13T17:31:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.710+0000 7f32467fc640 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d0d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f323c096900 tx=0x7f323c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3050abf540a971 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f322dffb640 1 -- 172.21.15.44:0/3467622021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c008930 con 0x55dfc30ce950 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f322dffb640 1 -- 172.21.15.44:0/3467622021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f323c008230 con 0x55dfc30ce950 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f322dffb640 1 -- 172.21.15.44:0/3467622021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c007b10 con 0x55dfc30ce950 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 -- 172.21.15.44:0/3467622021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 msgr2=0x55dfc30d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f32477fe640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f32300a1200 0x7f32300a36c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3240009fd0 tx=0x7f3240004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d0d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f323c096900 tx=0x7f323c020000 comp rx=0 tx=0).stop 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 -- 172.21.15.44:0/3467622021 shutdown_connections 2024-09-13T17:31:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d0d40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 --2- 172.21.15.44:0/3467622021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 -- 172.21.15.44:0/3467622021 >> 172.21.15.44:0/3467622021 conn(0x55dfc30bfd00 msgr2=0x55dfc30c00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 -- 172.21.15.44:0/3467622021 shutdown_connections 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.711+0000 7f325065e740 1 -- 172.21.15.44:0/3467622021 wait complete. 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 -- start start 2024-09-13T17:31:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 Processor -- start 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2fe5180 con 0x55dfc30ce950 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2cff430 con 0x55dfc30c40b0 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2faa5e0 con 0x55dfc30c3790 2024-09-13T17:31:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f325065e740 10 client.0 _ll_register_callbacks cb 0x55dfc3016aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f32477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f32467fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f32477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34422/0 (socket says 172.21.15.44:34422) 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.712+0000 7f32477fe640 1 -- 172.21.15.44:0/346778479 learned_addr learned my addr 172.21.15.44:0/346778479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f3246ffd640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32477fe640 1 -- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 msgr2=0x55dfc30cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32477fe640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32477fe640 1 -- 172.21.15.44:0/346778479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 msgr2=0x55dfc30ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32477fe640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32477fe640 1 -- 172.21.15.44:0/346778479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32481a53f0 con 0x55dfc30ce950 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32467fc640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f32477fe640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30cb3c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f324000fc30 tx=0x7f3240004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdfb1a74694ded2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f3226ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f3226ffd640 1 -- 172.21.15.44:0/346778479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240022070 con 0x55dfc30ce950 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f3226ffd640 1 -- 172.21.15.44:0/346778479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f324001d0d0 con 0x55dfc30ce950 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfc3050b80 con 0x55dfc30ce950 2024-09-13T17:31:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f3226ffd640 1 -- 172.21.15.44:0/346778479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f324001d2d0 con 0x55dfc30ce950 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f325065e740 10 client.17181 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dfc30253c0 con 0x55dfc30ce950 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.713+0000 7f3226ffd640 1 -- 172.21.15.44:0/346778479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dfc3050b80 con 0x55dfc30ce950 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f3226ffd640 1 -- 172.21.15.44:0/346778479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32400050b0 con 0x55dfc30ce950 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f325065e740 10 client.17181 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f325065e740 10 client.17181 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f325065e740 20 client.17181 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f325065e740 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc30de240 0x55dfc30e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f325065e740 4 client.17181 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfc30191f0 con 0x55dfc30de240 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f3246ffd640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc30de240 0x55dfc30e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f3226ffd640 10 client.17181 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.714+0000 7f3246ffd640 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc30de240 0x55dfc30e0660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbc06267f7fb1bf6 server_cookie=e763cea2f59aa20 in_seq=0 out_seq=0 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f3226ffd640 1 -- 172.21.15.44:0/346778479 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dfc30191f0 con 0x55dfc30de240 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f3226ffd640 10 client.17181 handle_command_reply: tid=0 2024-09-13T17:31:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 client.17181 shutdown 2024-09-13T17:31:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 msgr2=0x55dfc30cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30cb3c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f324000fc30 tx=0x7f3240004ab0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 shutdown_connections 2024-09-13T17:31:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc30de240 0x55dfc30e0660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30cb3c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f3226ffd640 0 client.17181 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 --2- 172.21.15.44:0/346778479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 >> 172.21.15.44:0/346778479 conn(0x55dfc30bfd00 msgr2=0x55dfc30beeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 shutdown_connections 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.715+0000 7f325065e740 1 -- 172.21.15.44:0/346778479 wait complete. 2024-09-13T17:31:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.716+0000 7f325065e740 20 client.17181 trim_cache size 0 max 16384 2024-09-13T17:31:41.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.726+0000 7f325065e740 1 Processor -- start 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.726+0000 7f325065e740 1 -- start start 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f325065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f325065e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f325065e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f325065e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc30c0c60 con 0x55dfc30ce950 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f325065e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc30cb790 con 0x55dfc30c40b0 2024-09-13T17:31:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f325065e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc30daf60 con 0x55dfc30c3790 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34434/0 (socket says 172.21.15.44:34434) 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 -- 172.21.15.44:0/3338586885 learned_addr learned my addr 172.21.15.44:0/3338586885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f3246ffd640 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32467fc640 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 -- 172.21.15.44:0/3338586885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 msgr2=0x55dfc30e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 -- 172.21.15.44:0/3338586885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 msgr2=0x55dfc30d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30d5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 -- 172.21.15.44:0/3338586885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc2fc7770 con 0x55dfc30ce950 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f32477fe640 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d5b60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3240010630 tx=0x7f3240009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1936d773da1aff0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f322e7fc640 1 -- 172.21.15.44:0/3338586885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240039070 con 0x55dfc30ce950 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f322e7fc640 1 -- 172.21.15.44:0/3338586885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3240023040 con 0x55dfc30ce950 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.727+0000 7f322e7fc640 1 -- 172.21.15.44:0/3338586885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240022070 con 0x55dfc30ce950 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 -- 172.21.15.44:0/3338586885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 msgr2=0x55dfc30d5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d5b60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3240010630 tx=0x7f3240009500 comp rx=0 tx=0).stop 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 -- 172.21.15.44:0/3338586885 shutdown_connections 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30ce950 0x55dfc30d5b60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30c40b0 0x55dfc30d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 --2- 172.21.15.44:0/3338586885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c3790 0x55dfc30e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 -- 172.21.15.44:0/3338586885 >> 172.21.15.44:0/3338586885 conn(0x55dfc30bfd00 msgr2=0x55dfc30da0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 -- 172.21.15.44:0/3338586885 shutdown_connections 2024-09-13T17:31:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 -- 172.21.15.44:0/3338586885 wait complete. 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 -- start start 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.728+0000 7f325065e740 1 Processor -- start 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 0x55dfc30d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c40b0 0x55dfc30d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30ce950 0x55dfc30d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc30d4570 con 0x55dfc30c3790 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc30d46e0 con 0x55dfc30ce950 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc30d48a0 con 0x55dfc30c40b0 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 0x55dfc30d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f325065e740 10 client.0 _ll_register_callbacks cb 0x7f323c0d0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 0x55dfc30d56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34436/0 (socket says 172.21.15.44:34436) 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 -- 172.21.15.44:0/3978382766 learned_addr learned my addr 172.21.15.44:0/3978382766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f32467fc640 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c40b0 0x55dfc30d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 -- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c40b0 msgr2=0x55dfc30d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c40b0 0x55dfc30d3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 -- 172.21.15.44:0/3978382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30ce950 msgr2=0x55dfc30d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30ce950 0x55dfc30d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 -- 172.21.15.44:0/3978382766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfc30253c0 con 0x55dfc30c3790 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3246ffd640 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 0x55dfc30d56f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3238009900 tx=0x7f3238005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc9914a979fdd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3226ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.729+0000 7f3226ffd640 1 -- 172.21.15.44:0/3978382766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238042050 con 0x55dfc30c3790 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f3226ffd640 1 -- 172.21.15.44:0/3978382766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3238044d70 con 0x55dfc30c3790 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f323c008230 con 0x55dfc30c3790 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f3226ffd640 1 -- 172.21.15.44:0/3978382766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3238004510 con 0x55dfc30c3790 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dfc2fece80 con 0x55dfc30c3790 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f3226ffd640 1 -- 172.21.15.44:0/3978382766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f323c008230 con 0x55dfc30c3790 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f3226ffd640 1 -- 172.21.15.44:0/3978382766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f323804a020 con 0x55dfc30c3790 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 10 client.17187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 20 client.17187 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc3135c20 0x55dfc3138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 4 client.17187 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfc2d1a4e0 con 0x55dfc3135c20 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.730+0000 7f32467fc640 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc3135c20 0x55dfc3138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f32467fc640 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc3135c20 0x55dfc3138040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88b25f374a4b02a8 server_cookie=982d7eebc51bdbdc in_seq=0 out_seq=0 2024-09-13T17:31:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f3226ffd640 10 client.17187 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f3226ffd640 1 -- 172.21.15.44:0/3978382766 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfc2d1a4e0 con 0x55dfc3135c20 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f3226ffd640 10 client.17187 handle_command_reply: tid=0 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 client.17187 shutdown 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 msgr2=0x55dfc30d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 0x55dfc30d56f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3238009900 tx=0x7f3238005fd0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 shutdown_connections 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55dfc3135c20 0x55dfc3138040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfc30c40b0 0x55dfc30d3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dfc30ce950 0x55dfc30d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f3226ffd640 0 client.17187 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:31:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 --2- 172.21.15.44:0/3978382766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfc30c3790 0x55dfc30d56f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 >> 172.21.15.44:0/3978382766 conn(0x55dfc30bfd00 msgr2=0x55dfc30da0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 shutdown_connections 2024-09-13T17:31:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 1 -- 172.21.15.44:0/3978382766 wait complete. 2024-09-13T17:31:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.731+0000 7f325065e740 20 client.17187 trim_cache size 0 max 16384 2024-09-13T17:31:41.747 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:41.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f32300a1200 msgr2=0x7f32300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f32300a1200 0x7f32300a36c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3240009fd0 tx=0x7f3240004a40 comp rx=0 tx=0).stop 2024-09-13T17:31:41.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 msgr2=0x7f3248100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 0x7f3248100e50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c970 tx=0x7f323c00ce30 comp rx=0 tx=0).stop 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 shutdown_connections 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f32300a1200 0x7f32300a36c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3248101390 0x7f32481a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3248103fd0 0x7f3248100e50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 --2- 172.21.15.44:0/3309377375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32480fee80 0x7f3248100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 >> 172.21.15.44:0/3309377375 conn(0x7f32480fa8d0 msgr2=0x7f3248068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 shutdown_connections 2024-09-13T17:31:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.732+0000 7f324d93e640 1 -- 172.21.15.44:0/3309377375 wait complete. 2024-09-13T17:31:41.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:31:41.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 -- 172.21.15.44:0/466365664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef80ff830 msgr2=0x7f6ef8103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/466365664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef80ff830 0x7f6ef8103870 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009930 tx=0x7f6eec02a420 comp rx=0 tx=0).stop 2024-09-13T17:31:41.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 -- 172.21.15.44:0/466365664 shutdown_connections 2024-09-13T17:31:41.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/466365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef8103fd0 0x7f6ef810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/466365664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef80ff830 0x7f6ef8103870 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/466365664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef80fee80 0x7f6ef80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 -- 172.21.15.44:0/466365664 >> 172.21.15.44:0/466365664 conn(0x7f6ef80fa930 msgr2=0x7f6ef80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 -- 172.21.15.44:0/466365664 shutdown_connections 2024-09-13T17:31:41.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.853+0000 7f6efd8b1640 1 -- 172.21.15.44:0/466365664 wait complete. 2024-09-13T17:31:41.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 Processor -- start 2024-09-13T17:31:41.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 -- start start 2024-09-13T17:31:41.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef80fee80 0x7f6ef8100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 0x7f6ef8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef8101390 0x7f6ef81a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81a49d0 con 0x7f6ef8103fd0 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81a4b40 con 0x7f6ef8101390 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.854+0000 7f6efd8b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81a4d00 con 0x7f6ef80fee80 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 0x7f6ef8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef80fee80 0x7f6ef8100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 0x7f6ef8100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34458/0 (socket says 172.21.15.44:34458) 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 -- 172.21.15.44:0/587956704 learned_addr learned my addr 172.21.15.44:0/587956704 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef8101390 0x7f6ef81a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 -- 172.21.15.44:0/587956704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef80fee80 msgr2=0x7f6ef8100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef80fee80 0x7f6ef8100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 -- 172.21.15.44:0/587956704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef8101390 msgr2=0x7f6ef81a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef8101390 0x7f6ef81a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 -- 172.21.15.44:0/587956704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec009560 con 0x7f6ef8103fd0 2024-09-13T17:31:41.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.855+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 0x7f6ef8100e50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02a940 tx=0x7f6eec005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.860+0000 7f6efc8af640 1 -- 172.21.15.44:0/587956704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec004b50 con 0x7f6ef8103fd0 2024-09-13T17:31:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.860+0000 7f6efc8af640 1 -- 172.21.15.44:0/587956704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6eec004cb0 con 0x7f6ef8103fd0 2024-09-13T17:31:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.860+0000 7f6efc8af640 1 -- 172.21.15.44:0/587956704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec03c890 con 0x7f6ef8103fd0 2024-09-13T17:31:41.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.860+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef81a4f80 con 0x7f6ef8103fd0 2024-09-13T17:31:41.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.861+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef81a53a0 con 0x7f6ef8103fd0 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6efc8af640 1 -- 172.21.15.44:0/587956704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6eec03c9f0 con 0x7f6ef8103fd0 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 Processor -- start 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 -- start start 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e5057cc0 con 0x55e4e50ed790 2024-09-13T17:31:41.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e5055450 con 0x55e4e50ee0b0 2024-09-13T17:31:41.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6f005d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e5050d40 con 0x55e4e50f8950 2024-09-13T17:31:41.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34462/0 (socket says 172.21.15.44:34462) 2024-09-13T17:31:41.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6ef77fe640 1 -- 172.21.15.44:0/842344109 learned_addr learned my addr 172.21.15.44:0/842344109 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.863+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6efc8af640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6ee00a12b0 0x7f6ee00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 -- 172.21.15.44:0/842344109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 msgr2=0x55e4e50fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 -- 172.21.15.44:0/842344109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 msgr2=0x55e4e50f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 -- 172.21.15.44:0/842344109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e5018620 con 0x55e4e50ed790 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6efc8af640 1 -- 172.21.15.44:0/587956704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6eec0edd90 con 0x7f6ef8103fd0 2024-09-13T17:31:41.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6ee00a12b0 0x7f6ee00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50edb70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0011590 tx=0x7f6ef0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbab672e3575f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6eddffb640 1 -- 172.21.15.44:0/842344109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef001bb90 con 0x55e4e50ed790 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6ee00a12b0 0x7f6ee00a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0022000 tx=0x7f6ef0023000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6eddffb640 1 -- 172.21.15.44:0/842344109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ef001f070 con 0x55e4e50ed790 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.864+0000 7f6eddffb640 1 -- 172.21.15.44:0/842344109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000ed90 con 0x55e4e50ed790 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 -- 172.21.15.44:0/842344109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 msgr2=0x55e4e50edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50edb70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0011590 tx=0x7f6ef0011a50 comp rx=0 tx=0).stop 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 -- 172.21.15.44:0/842344109 shutdown_connections 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 --2- 172.21.15.44:0/842344109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50edb70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 -- 172.21.15.44:0/842344109 >> 172.21.15.44:0/842344109 conn(0x55e4e50e9d00 msgr2=0x55e4e50ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 -- 172.21.15.44:0/842344109 shutdown_connections 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 -- 172.21.15.44:0/842344109 wait complete. 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.865+0000 7f6f005d1740 1 -- start start 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 Processor -- start 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ed790 0x55e4e50f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 0x55e4e50f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e500f180 con 0x55e4e50f8950 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e4d29430 con 0x55e4e50ed790 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e4fd45e0 con 0x55e4e50ee0b0 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6f005d1740 10 client.0 _ll_register_callbacks cb 0x55e4e5040aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 0x55e4e50f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ed790 0x55e4e50f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ed790 0x55e4e50f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:56306/0 (socket says 172.21.15.44:56306) 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 0x55e4e50f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34476/0 (socket says 172.21.15.44:34476) 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/2414690458 learned_addr learned my addr 172.21.15.44:0/2414690458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 -- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 msgr2=0x55e4e50f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 -- 172.21.15.44:0/2414690458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ed790 msgr2=0x55e4e50f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ed790 0x55e4e50f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.866+0000 7f6ef77fe640 1 -- 172.21.15.44:0/2414690458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef0009c50 con 0x55e4e50f8950 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 0x55e4e50f5730 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000ffd0 tx=0x7f6ef00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34d686570cfe98b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2414690458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0004d80 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2414690458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6ef000e3b0 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4e507ab80 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2414690458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef000e570 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 10 client.17199 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4e504f3c0 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2414690458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e4e507ab80 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2414690458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ef0005210 con 0x55e4e50f8950 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 10 client.17199 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:41.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 20 client.17199 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e5108250 0x55e4e510a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 4 client.17199 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.867+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4e50431f0 con 0x55e4e5108250 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.868+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e5108250 0x55e4e510a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.868+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e5108250 0x55e4e510a640 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0c6cf1f758b8f8 server_cookie=b02803076522c77 in_seq=0 out_seq=0 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.868+0000 7f6ed6ffd640 10 client.17199 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.868+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2414690458 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e4e50431f0 con 0x55e4e5108250 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.868+0000 7f6ed6ffd640 10 client.17199 handle_command_reply: tid=0 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 client.17199 shutdown 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 msgr2=0x55e4e50f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 0x55e4e50f5730 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000ffd0 tx=0x7f6ef00096c0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 shutdown_connections 2024-09-13T17:31:41.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e5108250 0x55e4e510a640 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50f8950 0x55e4e50f5730 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6ed6ffd640 0 client.17199 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2414690458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ed790 0x55e4e50f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 >> 172.21.15.44:0/2414690458 conn(0x55e4e50e9d00 msgr2=0x55e4e50efa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 shutdown_connections 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 1 -- 172.21.15.44:0/2414690458 wait complete. 2024-09-13T17:31:41.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.869+0000 7f6f005d1740 20 client.17199 trim_cache size 0 max 16384 2024-09-13T17:31:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 Processor -- start 2024-09-13T17:31:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 -- start start 2024-09-13T17:31:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50f8950 0x55e4e50f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e50f4520 con 0x55e4e50ed790 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e50f3720 con 0x55e4e50f8950 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6f005d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e50ed580 con 0x55e4e50ee0b0 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50e96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34482/0 (socket says 172.21.15.44:34482) 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/3212508969 learned_addr learned my addr 172.21.15.44:0/3212508969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50f8950 0x55e4e50f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/3212508969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 msgr2=0x55e4e50f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/3212508969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50f8950 msgr2=0x55e4e50f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50f8950 0x55e4e50f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/3212508969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e4ff1770 con 0x55e4e50ed790 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.880+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50f8950 0x55e4e50f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50e96a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x55e4e510af20 tx=0x7f6ee8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e9bcaf3f22227c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6ede7fc640 1 -- 172.21.15.44:0/3212508969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8017070 con 0x55e4e50ed790 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6ede7fc640 1 -- 172.21.15.44:0/3212508969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ee8018040 con 0x55e4e50ed790 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6ede7fc640 1 -- 172.21.15.44:0/3212508969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8010950 con 0x55e4e50ed790 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 -- 172.21.15.44:0/3212508969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 msgr2=0x55e4e50e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50e96a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x55e4e510af20 tx=0x7f6ee8005f90 comp rx=0 tx=0).stop 2024-09-13T17:31:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 -- 172.21.15.44:0/3212508969 shutdown_connections 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50ee0b0 0x55e4e50f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50f8950 0x55e4e50f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 --2- 172.21.15.44:0/3212508969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50e96a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 -- 172.21.15.44:0/3212508969 >> 172.21.15.44:0/3212508969 conn(0x55e4e50e9d00 msgr2=0x55e4e510efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 -- 172.21.15.44:0/3212508969 shutdown_connections 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.881+0000 7f6f005d1740 1 -- 172.21.15.44:0/3212508969 wait complete. 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 -- start start 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 Processor -- start 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e50fe3b0 con 0x55e4e50ed790 2024-09-13T17:31:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e50fe520 con 0x55e4e50ee0b0 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e50fe6e0 con 0x55e4e50f8950 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6f005d1740 10 client.0 _ll_register_callbacks cb 0x7f6ef0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50ff5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34494/0 (socket says 172.21.15.44:34494) 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/2237779071 learned_addr learned my addr 172.21.15.44:0/2237779071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.882+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef77fe640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 msgr2=0x55e4e50fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/2237779071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 msgr2=0x55e4e50ffae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50ffae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef6ffd640 1 -- 172.21.15.44:0/2237779071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee8009bc0 con 0x55e4e50ed790 2024-09-13T17:31:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ef6ffd640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50ff5a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8004650 tx=0x7f6ee8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4b97661055b575 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2237779071 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8017070 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2237779071 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6ee801e830 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ef001f070 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2237779071 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee8003eb0 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6f005d1740 10 client.17205 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4e5016e80 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2237779071 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ef001f070 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.883+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2237779071 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ee8025070 con 0x55e4e50ed790 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 10 client.17205 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 20 client.17205 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e515f960 0x55e4e5161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 4 client.17205 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4e4d444e0 con 0x55e4e515f960 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e515f960 0x55e4e5161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6ef67fc640 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e515f960 0x55e4e5161d50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758b292d304df078 server_cookie=ce984218a1d908fc in_seq=0 out_seq=0 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6ed6ffd640 10 client.17205 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6ed6ffd640 1 -- 172.21.15.44:0/2237779071 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4e4d444e0 con 0x55e4e515f960 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6ed6ffd640 10 client.17205 handle_command_reply: tid=0 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 1 client.17205 shutdown 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 msgr2=0x55e4e50ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.884+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50ff5a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8004650 tx=0x7f6ee8003c80 comp rx=0 tx=0).stop 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 shutdown_connections 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e4e515f960 0x55e4e5161d50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e50f8950 0x55e4e50fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4e50ee0b0 0x55e4e50ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 --2- 172.21.15.44:0/2237779071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4e50ed790 0x55e4e50ff5a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6ed6ffd640 0 client.17205 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 >> 172.21.15.44:0/2237779071 conn(0x55e4e50e9d00 msgr2=0x55e4e51040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 shutdown_connections 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 1 -- 172.21.15.44:0/2237779071 wait complete. 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6f005d1740 20 client.17205 trim_cache size 0 max 16384 2024-09-13T17:31:41.899 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6ee00a12b0 msgr2=0x7f6ee00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6ee00a12b0 0x7f6ee00a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0022000 tx=0x7f6ef0023000 comp rx=0 tx=0).stop 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 msgr2=0x7f6ef8100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 0x7f6ef8100e50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02a940 tx=0x7f6eec005530 comp rx=0 tx=0).stop 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 shutdown_connections 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6ee00a12b0 0x7f6ee00a3770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef8101390 0x7f6ef81a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8103fd0 0x7f6ef8100e50 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 --2- 172.21.15.44:0/587956704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef80fee80 0x7f6ef8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 >> 172.21.15.44:0/587956704 conn(0x7f6ef80fa930 msgr2=0x7f6ef8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 shutdown_connections 2024-09-13T17:31:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:41.885+0000 7f6efd8b1640 1 -- 172.21.15.44:0/587956704 wait complete. 2024-09-13T17:31:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 -- 172.21.15.44:0/1269613582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b4105070 msgr2=0x7ff3b410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 --2- 172.21.15.44:0/1269613582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b4105070 0x7ff3b410cbc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009cd0 tx=0x7ff3ac02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 -- 172.21.15.44:0/1269613582 shutdown_connections 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 --2- 172.21.15.44:0/1269613582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b41008d0 0x7ff3b4104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 --2- 172.21.15.44:0/1269613582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b4105070 0x7ff3b410cbc0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 --2- 172.21.15.44:0/1269613582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b40fff20 0x7ff3b4100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 -- 172.21.15.44:0/1269613582 >> 172.21.15.44:0/1269613582 conn(0x7ff3b40fb9b0 msgr2=0x7ff3b40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 -- 172.21.15.44:0/1269613582 shutdown_connections 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.973+0000 7ff3b948b640 1 -- 172.21.15.44:0/1269613582 wait complete. 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 Processor -- start 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 -- start start 2024-09-13T17:31:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 0x7ff3b4196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b4105070 0x7ff3b41966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b419aaa0 0x7ff3b419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419b490 con 0x7ff3b40fff20 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419b600 con 0x7ff3b4105070 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b948b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419b770 con 0x7ff3b419aaa0 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 0x7ff3b4196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 0x7ff3b4196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34500/0 (socket says 172.21.15.44:34500) 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/4106576347 learned_addr learned my addr 172.21.15.44:0/4106576347 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b4105070 0x7ff3b41966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/4106576347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b419aaa0 msgr2=0x7ff3b419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.974+0000 7ff3b37fe640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b419aaa0 0x7ff3b419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b419aaa0 0x7ff3b419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/4106576347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b4105070 msgr2=0x7ff3b41966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b4105070 0x7ff3b41966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/4106576347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac009900 con 0x7ff3b40fff20 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b37fe640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b419aaa0 0x7ff3b419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b4105070 0x7ff3b41966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 0x7ff3b4196170 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400d950 tx=0x7ff3a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff39bfff640 1 -- 172.21.15.44:0/4106576347 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4009a30 con 0x7ff3b40fff20 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b41a2d50 con 0x7ff3b40fff20 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff39bfff640 1 -- 172.21.15.44:0/4106576347 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3a40044b0 con 0x7ff3b40fff20 2024-09-13T17:31:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff39bfff640 1 -- 172.21.15.44:0/4106576347 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40047a0 con 0x7ff3b40fff20 2024-09-13T17:31:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.975+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b41a31a0 con 0x7ff3b40fff20 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 Processor -- start 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 -- start start 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff39bfff640 1 -- 172.21.15.44:0/4106576347 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff3a4004020 con 0x7ff3b40fff20 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56330 0x55cd99d56710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d60fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 0x55cd99d638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99cc1c30 con 0x55cd99d614f0 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99cbf3c0 con 0x55cd99d56330 2024-09-13T17:31:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99cbacb0 con 0x55cd99d56c50 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 0x55cd99d638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 0x55cd99d638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34506/0 (socket says 172.21.15.44:34506) 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.978+0000 7ff3b27fc640 1 -- 172.21.15.44:0/192951317 learned_addr learned my addr 172.21.15.44:0/192951317 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b37fe640 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56330 0x55cd99d56710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d60fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b27fc640 1 -- 172.21.15.44:0/192951317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 msgr2=0x55cd99d60fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d60fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b27fc640 1 -- 172.21.15.44:0/192951317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56330 msgr2=0x55cd99d56710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56330 0x55cd99d56710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b27fc640 1 -- 172.21.15.44:0/192951317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd99c61910 con 0x55cd99d614f0 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff39bfff640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff39c0a1200 0x7ff39c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 0x55cd99d638e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800bc80 tx=0x7ff3a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33972d2b460f59e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3b37fe640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff39c0a1200 0x7ff39c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff39bfff640 1 -- 172.21.15.44:0/4106576347 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff3a40ccac0 con 0x7ff3b40fff20 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3997fa640 1 -- 172.21.15.44:0/192951317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8005020 con 0x55cd99d614f0 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3997fa640 1 -- 172.21.15.44:0/192951317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3a8005610 con 0x55cd99d614f0 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.979+0000 7ff3997fa640 1 -- 172.21.15.44:0/192951317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8011440 con 0x55cd99d614f0 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 -- 172.21.15.44:0/192951317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 msgr2=0x55cd99d638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 0x55cd99d638e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800bc80 tx=0x7ff3a800f730 comp rx=0 tx=0).stop 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3b37fe640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff39c0a1200 0x7ff39c0a36c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0098d0 tx=0x7ff3ac005fd0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 -- 172.21.15.44:0/192951317 shutdown_connections 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d60fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d614f0 0x55cd99d638e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/192951317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56330 0x55cd99d56710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 -- 172.21.15.44:0/192951317 >> 172.21.15.44:0/192951317 conn(0x55cd99d528c0 msgr2=0x55cd99d52ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 -- 172.21.15.44:0/192951317 shutdown_connections 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 -- 172.21.15.44:0/192951317 wait complete. 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 -- start start 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.980+0000 7ff3bc28a740 1 Processor -- start 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d60e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d5e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d614f0 0x55cd99d5e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99c78e80 con 0x55cd99d56330 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99c78b20 con 0x55cd99d614f0 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99c3e260 con 0x55cd99d56c50 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 10 client.0 _ll_register_callbacks cb 0x55cd99c73df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d60e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d60e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34508/0 (socket says 172.21.15.44:34508) 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/487203498 learned_addr learned my addr 172.21.15.44:0/487203498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d5e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b37fe640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d614f0 0x55cd99d5e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 msgr2=0x55cd99d5e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d5e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/487203498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d614f0 msgr2=0x55cd99d5e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d614f0 0x55cd99d5e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/487203498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a800b8e0 con 0x55cd99d56330 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d60e80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009610 tx=0x7ff3a40997b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a01ab99bf5ef02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff396ff5640 1 -- 172.21.15.44:0/487203498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40b7ea0 con 0x55cd99d56330 2024-09-13T17:31:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff396ff5640 1 -- 172.21.15.44:0/487203498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff3a400e350 con 0x55cd99d56330 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.981+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd99ce3680 con 0x55cd99d56330 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff396ff5640 1 -- 172.21.15.44:0/487203498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a40042d0 con 0x55cd99d56330 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 10 client.17217 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cd99cb9330 con 0x55cd99d56330 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff396ff5640 1 -- 172.21.15.44:0/487203498 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cd99ce3680 con 0x55cd99d56330 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff396ff5640 1 -- 172.21.15.44:0/487203498 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3a4095fc0 con 0x55cd99d56330 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 10 client.17217 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 20 client.17217 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99d70eb0 0x55cd99d732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 4 client.17217 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd99cae5d0 con 0x55cd99d70eb0 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.982+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99d70eb0 0x55cd99d732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.983+0000 7ff396ff5640 10 client.17217 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.983+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99d70eb0 0x55cd99d732a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4502adacd1a6c6bf server_cookie=6c4b4b10d52f1d3e in_seq=0 out_seq=0 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.983+0000 7ff396ff5640 1 -- 172.21.15.44:0/487203498 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cd99cae5d0 con 0x55cd99d70eb0 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.983+0000 7ff396ff5640 10 client.17217 handle_command_reply: tid=0 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.983+0000 7ff3bc28a740 1 client.17217 shutdown 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 msgr2=0x55cd99d60e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d60e80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009610 tx=0x7ff3a40997b0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 shutdown_connections 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99d70eb0 0x55cd99d732a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d56c50 0x55cd99d5e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d614f0 0x55cd99d5e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff396ff5640 0 client.17217 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/487203498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d60e80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 >> 172.21.15.44:0/487203498 conn(0x55cd99d528c0 msgr2=0x55cd99d51b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 shutdown_connections 2024-09-13T17:31:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 1 -- 172.21.15.44:0/487203498 wait complete. 2024-09-13T17:31:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.984+0000 7ff3bc28a740 20 client.17217 trim_cache size 0 max 16384 2024-09-13T17:31:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 Processor -- start 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 -- start start 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d52300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99d685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 0x55cd99d609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99d561d0 con 0x55cd99d56330 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99d56a70 con 0x55cd99d56c50 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99d59cd0 con 0x55cd99d614f0 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d52300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d52300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34510/0 (socket says 172.21.15.44:34510) 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2919145270 learned_addr learned my addr 172.21.15.44:0/2919145270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2919145270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 msgr2=0x55cd99d609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 0x55cd99d609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2919145270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 msgr2=0x55cd99d685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99d685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.996+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2919145270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd99c5b300 con 0x55cd99d56330 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d52300 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40995b0 tx=0x7ff3a4097e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1263d2f9e5d13ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff399ffb640 1 -- 172.21.15.44:0/2919145270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40b7ea0 con 0x55cd99d56330 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff399ffb640 1 -- 172.21.15.44:0/2919145270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3a4099fb0 con 0x55cd99d56330 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff399ffb640 1 -- 172.21.15.44:0/2919145270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400e350 con 0x55cd99d56330 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2919145270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 msgr2=0x55cd99d52300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d52300 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40995b0 tx=0x7ff3a4097e60 comp rx=0 tx=0).stop 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2919145270 shutdown_connections 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 0x55cd99d609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99d685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2919145270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99d52300 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2919145270 >> 172.21.15.44:0/2919145270 conn(0x55cd99d528c0 msgr2=0x55cd99d4da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2919145270 shutdown_connections 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.997+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2919145270 wait complete. 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 -- start start 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 Processor -- start 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99dc3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99dc40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 0x55cd99dbf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99dbf8c0 con 0x55cd99d56330 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99dbfa30 con 0x55cd99d56c50 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd99dbfbf0 con 0x55cd99d614f0 2024-09-13T17:31:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3bc28a740 10 client.0 _ll_register_callbacks cb 0x7ff3a800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99dc3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99dc3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34516/0 (socket says 172.21.15.44:34516) 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2015150196 learned_addr learned my addr 172.21.15.44:0/2015150196 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99dc40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 msgr2=0x55cd99dbf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 0x55cd99dbf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2015150196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 msgr2=0x55cd99dc40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99dc40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.998+0000 7ff3b2ffd640 1 -- 172.21.15.44:0/2015150196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd99cb9330 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3b2ffd640 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99dc3b80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a1800 tx=0x7ff3a409d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af069cccfa28fafb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff396ff5640 1 -- 172.21.15.44:0/2015150196 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40b7ea0 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff396ff5640 1 -- 172.21.15.44:0/2015150196 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff3a40042d0 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3a8005610 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff396ff5640 1 -- 172.21.15.44:0/2015150196 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a40954e0 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 10 client.17223 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cd99c808e0 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff396ff5640 1 -- 172.21.15.44:0/2015150196 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3a8005610 con 0x55cd99d56330 2024-09-13T17:31:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff396ff5640 1 -- 172.21.15.44:0/2015150196 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3a40b90b0 con 0x55cd99d56330 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 10 client.17223 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 20 client.17223 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99dcc6b0 0x55cd99dceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 4 client.17223 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:41.999+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd999ae300 con 0x55cd99dcc6b0 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.000+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99dcc6b0 0x55cd99dceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.000+0000 7ff3b27fc640 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99dcc6b0 0x55cd99dceaa0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f261c7fe1a5986 server_cookie=2f081bb7be879d15 in_seq=0 out_seq=0 2024-09-13T17:31:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.000+0000 7ff396ff5640 10 client.17223 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff396ff5640 1 -- 172.21.15.44:0/2015150196 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd999ae300 con 0x55cd99dcc6b0 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff396ff5640 10 client.17223 handle_command_reply: tid=0 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 client.17223 shutdown 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 msgr2=0x55cd99dc3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99dc3b80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a1800 tx=0x7ff3a409d6e0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 shutdown_connections 2024-09-13T17:31:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cd99dcc6b0 0x55cd99dceaa0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd99d614f0 0x55cd99dbf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff396ff5640 0 client.17223 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cd99d56c50 0x55cd99dc40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 --2- 172.21.15.44:0/2015150196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd99d56330 0x55cd99dc3b80 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 >> 172.21.15.44:0/2015150196 conn(0x55cd99d528c0 msgr2=0x55cd99d5c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 shutdown_connections 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 1 -- 172.21.15.44:0/2015150196 wait complete. 2024-09-13T17:31:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3bc28a740 20 client.17223 trim_cache size 0 max 16384 2024-09-13T17:31:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:31:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:31:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:31:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff39c0a1200 msgr2=0x7ff39c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff39c0a1200 0x7ff39c0a36c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0098d0 tx=0x7ff3ac005fd0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 msgr2=0x7ff3b4196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 0x7ff3b4196170 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400d950 tx=0x7ff3a400de10 comp rx=0 tx=0).stop 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 shutdown_connections 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff39c0a1200 0x7ff39c0a36c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b419aaa0 0x7ff3b419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b4105070 0x7ff3b41966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 --2- 172.21.15.44:0/4106576347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b40fff20 0x7ff3b4196170 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 >> 172.21.15.44:0/4106576347 conn(0x7ff3b40fb9b0 msgr2=0x7ff3b40fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 shutdown_connections 2024-09-13T17:31:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:42.015+0000 7ff3b948b640 1 -- 172.21.15.44:0/4106576347 wait complete. 2024-09-13T17:31:42.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:41 smithi044 ceph-mon[31175]: pgmap v152: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-13T17:31:42.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:41 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4257923348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:41 smithi070 ceph-mon[34502]: pgmap v152: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-13T17:31:42.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:41 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4257923348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:41 smithi100 ceph-mon[35055]: pgmap v152: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-13T17:31:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:41 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4257923348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:31:42.902 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:31:42.902 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 -- 172.21.15.44:0/928878558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 msgr2=0x7fad1010cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 --2- 172.21.15.44:0/928878558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad1010cbf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fad08009cd0 tx=0x7fad0802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 -- 172.21.15.44:0/928878558 shutdown_connections 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 --2- 172.21.15.44:0/928878558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad10100900 0x7fad10104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 --2- 172.21.15.44:0/928878558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad1010cbf0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 --2- 172.21.15.44:0/928878558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad100fff50 0x7fad10100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.020+0000 7fad17448640 1 -- 172.21.15.44:0/928878558 >> 172.21.15.44:0/928878558 conn(0x7fad100fb9e0 msgr2=0x7fad100fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.021+0000 7fad17448640 1 -- 172.21.15.44:0/928878558 shutdown_connections 2024-09-13T17:31:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.021+0000 7fad17448640 1 -- 172.21.15.44:0/928878558 wait complete. 2024-09-13T17:31:43.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.021+0000 7fad17448640 1 Processor -- start 2024-09-13T17:31:43.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.021+0000 7fad17448640 1 -- start start 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.021+0000 7fad17448640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad100fff50 0x7fad10198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.021+0000 7fad17448640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad101988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad17448640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad1019cc90 0x7fad1019d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad17448640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019d680 con 0x7fad101050a0 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad17448640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019d7f0 con 0x7fad100fff50 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad17448640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019d960 con 0x7fad1019cc90 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad101988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad101988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34534/0 (socket says 172.21.15.44:34534) 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 -- 172.21.15.44:0/1163065632 learned_addr learned my addr 172.21.15.44:0/1163065632 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad151e4640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad100fff50 0x7fad10198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad159e5640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad1019cc90 0x7fad1019d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 -- 172.21.15.44:0/1163065632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad1019cc90 msgr2=0x7fad1019d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad1019cc90 0x7fad1019d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 -- 172.21.15.44:0/1163065632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad100fff50 msgr2=0x7fad10198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad100fff50 0x7fad10198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 -- 172.21.15.44:0/1163065632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08009900 con 0x7fad101050a0 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad149e3640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad101988a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fad0000c970 tx=0x7fad0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad067fc640 1 -- 172.21.15.44:0/1163065632 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00007bb0 con 0x7fad101050a0 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad067fc640 1 -- 172.21.15.44:0/1163065632 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad00007d10 con 0x7fad101050a0 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad101a4f40 con 0x7fad101050a0 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.022+0000 7fad067fc640 1 -- 172.21.15.44:0/1163065632 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00004da0 con 0x7fad101050a0 2024-09-13T17:31:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.023+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad101a5360 con 0x7fad101050a0 2024-09-13T17:31:43.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad067fc640 1 -- 172.21.15.44:0/1163065632 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fad00005000 con 0x7fad101050a0 2024-09-13T17:31:43.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 Processor -- start 2024-09-13T17:31:43.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 -- start start 2024-09-13T17:31:43.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573203197c0 0x557320319ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x557320324440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x557320326d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557320284c60 con 0x55732031a0e0 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573202823f0 con 0x5573203197c0 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.025+0000 7fad1a248740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732027dce0 con 0x557320324980 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x557320324440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad067fc640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facfc0a1200 0x7facfc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x557320324440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34548/0 (socket says 172.21.15.44:34548) 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 -- 172.21.15.44:0/1037185783 learned_addr learned my addr 172.21.15.44:0/1037185783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad149e3640 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x557320326d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad159e5640 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573203197c0 0x557320319ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad159e5640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facfc0a1200 0x7facfc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 -- 172.21.15.44:0/1037185783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 msgr2=0x557320326d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x557320326d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 -- 172.21.15.44:0/1037185783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573203197c0 msgr2=0x557320319ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad067fc640 1 -- 172.21.15.44:0/1163065632 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fad000ccb90 con 0x7fad101050a0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573203197c0 0x557320319ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 -- 172.21.15.44:0/1037185783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557320239970 con 0x55732031a0e0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad151e4640 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x557320324440 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00ac20 tx=0x7fad0c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be53daf26d4a3204 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7fad159e5640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facfc0a1200 0x7facfc0a36c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad08006010 tx=0x7fad08005c50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.026+0000 7facfb7fe640 1 -- 172.21.15.44:0/1037185783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00a720 con 0x55732031a0e0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7facfb7fe640 1 -- 172.21.15.44:0/1037185783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad0c018ad0 con 0x55732031a0e0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7facfb7fe640 1 -- 172.21.15.44:0/1037185783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0109e0 con 0x55732031a0e0 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 -- 172.21.15.44:0/1037185783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 msgr2=0x557320324440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x557320324440 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00ac20 tx=0x7fad0c00f790 comp rx=0 tx=0).stop 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 -- 172.21.15.44:0/1037185783 shutdown_connections 2024-09-13T17:31:43.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x557320326d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x557320324440 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 --2- 172.21.15.44:0/1037185783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573203197c0 0x557320319ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 -- 172.21.15.44:0/1037185783 >> 172.21.15.44:0/1037185783 conn(0x557320315d50 msgr2=0x557320316130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 -- 172.21.15.44:0/1037185783 shutdown_connections 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.027+0000 7fad1a248740 1 -- 172.21.15.44:0/1037185783 wait complete. 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 -- start start 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 Processor -- start 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573203197c0 0x557320324350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x5573203215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x557320321c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732023bf80 con 0x55732031a0e0 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732023bc20 con 0x557320324980 2024-09-13T17:31:43.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557320208ab0 con 0x5573203197c0 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad1a248740 10 client.0 _ll_register_callbacks cb 0x557320277350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad149e3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x5573203215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad149e3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x5573203215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34552/0 (socket says 172.21.15.44:34552) 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.028+0000 7fad149e3640 1 -- 172.21.15.44:0/2493976502 learned_addr learned my addr 172.21.15.44:0/2493976502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad151e4640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573203197c0 0x557320324350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad159e5640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x557320321c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad149e3640 1 -- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573203197c0 msgr2=0x557320324350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad149e3640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573203197c0 0x557320324350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad149e3640 1 -- 172.21.15.44:0/2493976502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 msgr2=0x557320321c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad149e3640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x557320321c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad149e3640 1 -- 172.21.15.44:0/2493976502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c00a880 con 0x55732031a0e0 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad159e5640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x557320321c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad151e4640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573203197c0 0x557320324350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:43.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad149e3640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x5573203215d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fad0000afd0 tx=0x7fad000699b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36599bed90df36b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7facf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7facf8ff9640 1 -- 172.21.15.44:0/2493976502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000a1b70 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7facf8ff9640 1 -- 172.21.15.44:0/2493976502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fad000046c0 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55731ff71590 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7facf8ff9640 1 -- 172.21.15.44:0/2493976502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad00099a30 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 10 client.17235 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55732027c360 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7facf8ff9640 1 -- 172.21.15.44:0/2493976502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55731ff71590 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7facf8ff9640 1 -- 172.21.15.44:0/2493976502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad00016030 con 0x55732031a0e0 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 20 client.17235 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573203346f0 0x557320336ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 4 client.17235 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:43.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.029+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55731ff82860 con 0x5573203346f0 2024-09-13T17:31:43.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.030+0000 7fad151e4640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573203346f0 0x557320336ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.030+0000 7facf8ff9640 10 client.17235 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.030+0000 7fad151e4640 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573203346f0 0x557320336ae0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b05ff0b41785a1 server_cookie=74ec8745d599ac6a in_seq=0 out_seq=0 2024-09-13T17:31:43.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7facf8ff9640 1 -- 172.21.15.44:0/2493976502 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55731ff82860 con 0x5573203346f0 2024-09-13T17:31:43.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7facf8ff9640 10 client.17235 handle_command_reply: tid=0 2024-09-13T17:31:43.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 client.17235 shutdown 2024-09-13T17:31:43.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 msgr2=0x5573203215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x5573203215d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fad0000afd0 tx=0x7fad000699b0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 shutdown_connections 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573203346f0 0x557320336ae0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x557320321c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732031a0e0 0x5573203215d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7facf8ff9640 0 client.17235 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 --2- 172.21.15.44:0/2493976502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573203197c0 0x557320324350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 >> 172.21.15.44:0/2493976502 conn(0x557320315d50 msgr2=0x55732031b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 shutdown_connections 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 1 -- 172.21.15.44:0/2493976502 wait complete. 2024-09-13T17:31:43.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.031+0000 7fad1a248740 20 client.17235 trim_cache size 0 max 16384 2024-09-13T17:31:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 Processor -- start 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 -- start start 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55732031a0e0 0x55732032bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x55732031c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732031fb80 con 0x5573203197c0 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557320311880 con 0x55732031a0e0 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad1a248740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557320334480 con 0x557320324980 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34562/0 (socket says 172.21.15.44:34562) 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 -- 172.21.15.44:0/2257681078 learned_addr learned my addr 172.21.15.44:0/2257681078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad159e5640 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x55732031c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 -- 172.21.15.44:0/2257681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 msgr2=0x55732031c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad149e3640 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55732031a0e0 0x55732032bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x55732031c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 -- 172.21.15.44:0/2257681078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55732031a0e0 msgr2=0x55732032bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55732031a0e0 0x55732032bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 -- 172.21.15.44:0/2257681078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55732021e1c0 con 0x5573203197c0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad149e3640 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55732031a0e0 0x55732032bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.046+0000 7fad151e4640 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032b650 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x557320315850 tx=0x7fad0c010620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947911795ef29d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7facfbfff640 1 -- 172.21.15.44:0/2257681078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c027ea0 con 0x5573203197c0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7facfbfff640 1 -- 172.21.15.44:0/2257681078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad0c0044b0 con 0x5573203197c0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7facfbfff640 1 -- 172.21.15.44:0/2257681078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0054d0 con 0x5573203197c0 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 -- 172.21.15.44:0/2257681078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 msgr2=0x55732032b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032b650 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x557320315850 tx=0x7fad0c010620 comp rx=0 tx=0).stop 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 -- 172.21.15.44:0/2257681078 shutdown_connections 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557320324980 0x55732031c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55732031a0e0 0x55732032bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 --2- 172.21.15.44:0/2257681078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032b650 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 -- 172.21.15.44:0/2257681078 >> 172.21.15.44:0/2257681078 conn(0x557320315d50 msgr2=0x55732033b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 -- 172.21.15.44:0/2257681078 shutdown_connections 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.047+0000 7fad1a248740 1 -- 172.21.15.44:0/2257681078 wait complete. 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 -- start start 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 Processor -- start 2024-09-13T17:31:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55732031a0e0 0x55732032a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x55732032ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732038ac10 con 0x5573203197c0 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732038ad50 con 0x557320324980 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732038ae90 con 0x55732031a0e0 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad1a248740 10 client.0 _ll_register_callbacks cb 0x7fad0000c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34578/0 (socket says 172.21.15.44:34578) 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 -- 172.21.15.44:0/2290468882 learned_addr learned my addr 172.21.15.44:0/2290468882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad159e5640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x55732032ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad149e3640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55732031a0e0 0x55732032a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 -- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55732031a0e0 msgr2=0x55732032a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55732031a0e0 0x55732032a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 -- 172.21.15.44:0/2290468882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 msgr2=0x55732032ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x55732032ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad151e4640 1 -- 172.21.15.44:0/2290468882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557320239970 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.048+0000 7fad159e5640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x55732032ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad151e4640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032a0d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005b10 tx=0x7fad0c00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668b04f4c7ba3d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7facf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7facf8ff9640 1 -- 172.21.15.44:0/2290468882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c005350 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7facf8ff9640 1 -- 172.21.15.44:0/2290468882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fad0c0054b0 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad0c018ad0 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7facf8ff9640 1 -- 172.21.15.44:0/2290468882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad0c027150 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 10 client.17241 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557320243990 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7facf8ff9640 1 -- 172.21.15.44:0/2290468882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad0c018ad0 con 0x5573203197c0 2024-09-13T17:31:43.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7facf8ff9640 1 -- 172.21.15.44:0/2290468882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad0c010970 con 0x5573203197c0 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 10 client.17241 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 20 client.17241 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55732038c100 0x55732038e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 4 client.17241 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.049+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55731ffd5eb0 con 0x55732038c100 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.050+0000 7fad149e3640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55732038c100 0x55732038e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.050+0000 7facf8ff9640 10 client.17241 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.050+0000 7fad149e3640 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55732038c100 0x55732038e4f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46921c5b2af7f9b server_cookie=44900ff689d10dae in_seq=0 out_seq=0 2024-09-13T17:31:43.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7facf8ff9640 1 -- 172.21.15.44:0/2290468882 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55731ffd5eb0 con 0x55732038c100 2024-09-13T17:31:43.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7facf8ff9640 10 client.17241 handle_command_reply: tid=0 2024-09-13T17:31:43.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 client.17241 shutdown 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 msgr2=0x55732032a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032a0d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005b10 tx=0x7fad0c00fcc0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 shutdown_connections 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55732038c100 0x55732038e4f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55732031a0e0 0x55732032a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557320324980 0x55732032ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 --2- 172.21.15.44:0/2290468882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573203197c0 0x55732032a0d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 >> 172.21.15.44:0/2290468882 conn(0x557320315d50 msgr2=0x55732031c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.051+0000 7facf8ff9640 0 client.17241 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 shutdown_connections 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad1a248740 1 -- 172.21.15.44:0/2290468882 wait complete. 2024-09-13T17:31:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad1a248740 20 client.17241 trim_cache size 0 max 16384 2024-09-13T17:31:43.061 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:31:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "8c636ddb-7209-446f-871b-3e0b22b58ec2", 2024-09-13T17:31:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:31:43.061 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facfc0a1200 msgr2=0x7facfc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facfc0a1200 0x7facfc0a36c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad08006010 tx=0x7fad08005c50 comp rx=0 tx=0).stop 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 msgr2=0x7fad101988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad101988a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fad0000c970 tx=0x7fad0000ce30 comp rx=0 tx=0).stop 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 shutdown_connections 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facfc0a1200 0x7facfc0a36c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad1019cc90 0x7fad1019d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad101050a0 0x7fad101988a0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 --2- 172.21.15.44:0/1163065632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad100fff50 0x7fad10198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 >> 172.21.15.44:0/1163065632 conn(0x7fad100fb9e0 msgr2=0x7fad100fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 shutdown_connections 2024-09-13T17:31:43.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.052+0000 7fad17448640 1 -- 172.21.15.44:0/1163065632 wait complete. 2024-09-13T17:31:43.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:31:43.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 -- 172.21.15.44:0/803550954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb140fee60 msgr2=0x7fbb140ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 --2- 172.21.15.44:0/803550954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb140fee60 0x7fbb140ff240 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009930 tx=0x7fbafc02a420 comp rx=0 tx=0).stop 2024-09-13T17:31:43.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 -- 172.21.15.44:0/803550954 shutdown_connections 2024-09-13T17:31:43.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 --2- 172.21.15.44:0/803550954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14103fb0 0x7fbb1410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 --2- 172.21.15.44:0/803550954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140ff810 0x7fbb14103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 --2- 172.21.15.44:0/803550954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb140fee60 0x7fbb140ff240 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 -- 172.21.15.44:0/803550954 >> 172.21.15.44:0/803550954 conn(0x7fbb14075c60 msgr2=0x7fbb140fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 -- 172.21.15.44:0/803550954 shutdown_connections 2024-09-13T17:31:43.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 -- 172.21.15.44:0/803550954 wait complete. 2024-09-13T17:31:43.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 Processor -- start 2024-09-13T17:31:43.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.168+0000 7fbb1b232640 1 -- start start 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140fee60 0x7fbb141961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 0x7fbb141966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb1419aae0 0x7fbb1419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419b4d0 con 0x7fbb14103fb0 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419b640 con 0x7fbb140fee60 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419b7b0 con 0x7fbb1419aae0 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 0x7fbb141966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 0x7fbb141966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34604/0 (socket says 172.21.15.44:34604) 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 -- 172.21.15.44:0/1247880347 learned_addr learned my addr 172.21.15.44:0/1247880347 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb1419aae0 0x7fbb1419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb18fce640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140fee60 0x7fbb141961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 -- 172.21.15.44:0/1247880347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb1419aae0 msgr2=0x7fbb1419af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb1419aae0 0x7fbb1419af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 -- 172.21.15.44:0/1247880347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140fee60 msgr2=0x7fbb141961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140fee60 0x7fbb141961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 -- 172.21.15.44:0/1247880347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc009560 con 0x7fbb14103fb0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 0x7fbb141966f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400e9c0 tx=0x7fbb0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb11ffb640 1 -- 172.21.15.44:0/1247880347 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04009700 con 0x7fbb14103fb0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb11ffb640 1 -- 172.21.15.44:0/1247880347 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0400cd80 con 0x7fbb14103fb0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.169+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb141a2d90 con 0x7fbb14103fb0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.170+0000 7fbb11ffb640 1 -- 172.21.15.44:0/1247880347 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04010430 con 0x7fbb14103fb0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.170+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb141a31e0 con 0x7fbb14103fb0 2024-09-13T17:31:43.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.172+0000 7fbb11ffb640 1 -- 172.21.15.44:0/1247880347 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbb04004eb0 con 0x7fbb14103fb0 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.172+0000 7fbb1e02f740 1 Processor -- start 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.172+0000 7fbb1e02f740 1 -- start start 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07eabb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eabb630 0x55c07eac9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 0x55c07eacc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07ea25f20 con 0x55c07eaca3e0 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07ea236b0 con 0x55c07eabb630 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07ea1efa0 con 0x55c07eabad10 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb11ffb640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb000a1200 0x7fbb000a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 0x55c07eacc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 0x55c07eacc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34620/0 (socket says 172.21.15.44:34620) 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 -- 172.21.15.44:0/1221535274 learned_addr learned my addr 172.21.15.44:0/1221535274 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07eabb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb18fce640 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eabb630 0x55c07eac9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb000a1200 0x7fbb000a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb11ffb640 1 -- 172.21.15.44:0/1247880347 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbb04014070 con 0x7fbb14103fb0 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 -- 172.21.15.44:0/1221535274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 msgr2=0x55c07eabb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07eabb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 -- 172.21.15.44:0/1221535274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eabb630 msgr2=0x55c07eac9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eabb630 0x55c07eac9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 -- 172.21.15.44:0/1221535274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07e9d9840 con 0x55c07eaca3e0 2024-09-13T17:31:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07eabb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 0x55c07eacc7d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ca890 tx=0x7fbb040055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b1a7b9f02f0f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.173+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb000a1200 0x7fbb000a36c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009fd0 tx=0x7fbb0c009550 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb0affd640 1 -- 172.21.15.44:0/1221535274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0409a9e0 con 0x55c07eaca3e0 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb0affd640 1 -- 172.21.15.44:0/1221535274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0409a2e0 con 0x55c07eaca3e0 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb0affd640 1 -- 172.21.15.44:0/1221535274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040969f0 con 0x55c07eaca3e0 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1221535274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 msgr2=0x55c07eacc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 0x55c07eacc7d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ca890 tx=0x7fbb040055b0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1221535274 shutdown_connections 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eaca3e0 0x55c07eacc7d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eabb630 0x55c07eac9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1221535274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07eabb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1221535274 >> 172.21.15.44:0/1221535274 conn(0x55c07eab7380 msgr2=0x55c07eab7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1221535274 shutdown_connections 2024-09-13T17:31:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.174+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1221535274 wait complete. 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 -- start start 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 Processor -- start 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 0x55c07eac9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabb630 0x55c07eac6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eac75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07e9c02a0 con 0x55c07eabad10 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07e6f7430 con 0x55c07eaca3e0 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07e9a2680 con 0x55c07eabb630 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb1e02f740 10 client.0 _ll_register_callbacks cb 0x55c07ea18610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb18fce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 0x55c07eac9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb18fce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 0x55c07eac9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34634/0 (socket says 172.21.15.44:34634) 2024-09-13T17:31:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb18fce640 1 -- 172.21.15.44:0/287534663 learned_addr learned my addr 172.21.15.44:0/287534663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb13fff640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabb630 0x55c07eac6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.175+0000 7fbb197cf640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eac75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb18fce640 1 -- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabb630 msgr2=0x55c07eac6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb18fce640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabb630 0x55c07eac6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb18fce640 1 -- 172.21.15.44:0/287534663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 msgr2=0x55c07eac75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb18fce640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eac75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb18fce640 1 -- 172.21.15.44:0/287534663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb141a31e0 con 0x55c07eabad10 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb18fce640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 0x55c07eac9dc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbafc006010 tx=0x7fbafc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51322b1c24c3672 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbaf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbaf3fff640 1 -- 172.21.15.44:0/287534663 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc031b60 con 0x55c07eabad10 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbaf3fff640 1 -- 172.21.15.44:0/287534663 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbafc031cc0 con 0x55c07eabad10 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07e7124e0 con 0x55c07eabad10 2024-09-13T17:31:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbaf3fff640 1 -- 172.21.15.44:0/287534663 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbafc031e80 con 0x55c07eabad10 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb1e02f740 10 client.17253 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c07ea1d620 con 0x55c07eabad10 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbaf3fff640 1 -- 172.21.15.44:0/287534663 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c07e7124e0 con 0x55c07eabad10 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.176+0000 7fbaf3fff640 1 -- 172.21.15.44:0/287534663 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbafc046820 con 0x55c07eabad10 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb1e02f740 10 client.17253 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb1e02f740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb1e02f740 20 client.17253 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eada1b0 0x55c07eadc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb1e02f740 4 client.17253 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c07e7237e0 con 0x55c07eada1b0 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb13fff640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eada1b0 0x55c07eadc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbb13fff640 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eada1b0 0x55c07eadc5a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f979301f20c4fb3 server_cookie=b74a6e2691e453d4 in_seq=0 out_seq=0 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.177+0000 7fbaf3fff640 10 client.17253 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbaf3fff640 1 -- 172.21.15.44:0/287534663 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c07e7237e0 con 0x55c07eada1b0 2024-09-13T17:31:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbaf3fff640 10 client.17253 handle_command_reply: tid=0 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 client.17253 shutdown 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 msgr2=0x55c07eac9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 0x55c07eac9dc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbafc006010 tx=0x7fbafc004270 comp rx=0 tx=0).stop 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 shutdown_connections 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eada1b0 0x55c07eadc5a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabb630 0x55c07eac6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eac75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbaf3fff640 0 client.17253 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/287534663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabad10 0x55c07eac9dc0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 >> 172.21.15.44:0/287534663 conn(0x55c07eab7380 msgr2=0x55c07eabd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 shutdown_connections 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 1 -- 172.21.15.44:0/287534663 wait complete. 2024-09-13T17:31:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.178+0000 7fbb1e02f740 20 client.17253 trim_cache size 0 max 16384 2024-09-13T17:31:43.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.192+0000 7fbb1e02f740 1 Processor -- start 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.192+0000 7fbb1e02f740 1 -- start start 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.192+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07ead16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eab7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07eac8160 con 0x55c07eabb630 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07eabc230 con 0x55c07eaca3e0 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07eac1e40 con 0x55c07eabad10 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07ead16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07ead16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34644/0 (socket says 172.21.15.44:34644) 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 -- 172.21.15.44:0/302311974 learned_addr learned my addr 172.21.15.44:0/302311974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb18fce640 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb197cf640 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eab7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 -- 172.21.15.44:0/302311974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 msgr2=0x55c07ead1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 -- 172.21.15.44:0/302311974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 msgr2=0x55c07eab7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eab7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 -- 172.21.15.44:0/302311974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07e9bf790 con 0x55c07eabb630 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb13fff640 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07ead16b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbb04096d50 tx=0x7fbb04099c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc87b381f0183827 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb0b7fe640 1 -- 172.21.15.44:0/302311974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0409a0b0 con 0x55c07eabb630 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb0b7fe640 1 -- 172.21.15.44:0/302311974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb040b2e90 con 0x55c07eabb630 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.193+0000 7fbb0b7fe640 1 -- 172.21.15.44:0/302311974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040cd160 con 0x55c07eabb630 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 -- 172.21.15.44:0/302311974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 msgr2=0x55c07ead16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07ead16b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbb04096d50 tx=0x7fbb04099c40 comp rx=0 tx=0).stop 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 -- 172.21.15.44:0/302311974 shutdown_connections 2024-09-13T17:31:43.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eab7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07ead16b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/302311974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 -- 172.21.15.44:0/302311974 >> 172.21.15.44:0/302311974 conn(0x55c07eab7380 msgr2=0x55c07eac2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 -- 172.21.15.44:0/302311974 shutdown_connections 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 -- 172.21.15.44:0/302311974 wait complete. 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 -- start start 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.194+0000 7fbb1e02f740 1 Processor -- start 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07eacd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eacdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07eace330 con 0x55c07eabb630 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07eace4a0 con 0x55c07eaca3e0 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07eace660 con 0x55c07eabad10 2024-09-13T17:31:43.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 10 client.0 _ll_register_callbacks cb 0x7fbafc031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07eacd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07eacd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34652/0 (socket says 172.21.15.44:34652) 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 -- 172.21.15.44:0/1636556891 learned_addr learned my addr 172.21.15.44:0/1636556891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb18fce640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eacdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 -- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 msgr2=0x55c07ead1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 -- 172.21.15.44:0/1636556891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 msgr2=0x55c07eacdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eacdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 -- 172.21.15.44:0/1636556891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c07e9d9840 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb197cf640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eacdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb13fff640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07eacd830 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbb040a2230 tx=0x7fbb040a2820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f50248c8fe02020 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbaf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbaf3fff640 1 -- 172.21.15.44:0/1636556891 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040cd160 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbaf3fff640 1 -- 172.21.15.44:0/1636556891 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbb040cd2c0 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb0409a2e0 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbaf3fff640 1 -- 172.21.15.44:0/1636556891 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb040caba0 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 10 client.17259 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.195+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c07e9e4ea0 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbaf3fff640 1 -- 172.21.15.44:0/1636556891 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0409a2e0 con 0x55c07eabb630 2024-09-13T17:31:43.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbaf3fff640 1 -- 172.21.15.44:0/1636556891 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb0401f060 con 0x55c07eabb630 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb1e02f740 10 client.17259 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb1e02f740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb1e02f740 20 client.17259 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eb31750 0x55c07eb33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb1e02f740 4 client.17259 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c07e776e30 con 0x55c07eb31750 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.196+0000 7fbb18fce640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eb31750 0x55c07eb33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbaf3fff640 10 client.17259 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb18fce640 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eb31750 0x55c07eb33b70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7c5dac3de3c1e8 server_cookie=1afc4692e509fcd3 in_seq=0 out_seq=0 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbaf3fff640 1 -- 172.21.15.44:0/1636556891 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c07e776e30 con 0x55c07eb31750 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbaf3fff640 10 client.17259 handle_command_reply: tid=0 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 client.17259 shutdown 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 msgr2=0x55c07eacd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07eacd830 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbb040a2230 tx=0x7fbb040a2820 comp rx=0 tx=0).stop 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 shutdown_connections 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c07eb31750 0x55c07eb33b70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07eaca3e0 0x55c07eacdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07eabb630 0x55c07eacd830 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 --2- 172.21.15.44:0/1636556891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07eabad10 0x55c07ead1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbaf3fff640 0 client.17259 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 >> 172.21.15.44:0/1636556891 conn(0x55c07eab7380 msgr2=0x55c07eab7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 shutdown_connections 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 1 -- 172.21.15.44:0/1636556891 wait complete. 2024-09-13T17:31:43.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.197+0000 7fbb1e02f740 20 client.17259 trim_cache size 0 max 16384 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "8c636ddb-7209-446f-871b-3e0b22b58ec2": { 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "8c636ddb-7209-446f-871b-3e0b22b58ec2", 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:31:43.207 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:31:43.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb000a1200 msgr2=0x7fbb000a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb000a1200 0x7fbb000a36c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009fd0 tx=0x7fbb0c009550 comp rx=0 tx=0).stop 2024-09-13T17:31:43.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 msgr2=0x7fbb141966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:43.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 0x7fbb141966f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400e9c0 tx=0x7fbb0400ee80 comp rx=0 tx=0).stop 2024-09-13T17:31:43.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 shutdown_connections 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb000a1200 0x7fbb000a36c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb1419aae0 0x7fbb1419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14103fb0 0x7fbb141966f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 --2- 172.21.15.44:0/1247880347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140fee60 0x7fbb141961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 >> 172.21.15.44:0/1247880347 conn(0x7fbb14075c60 msgr2=0x7fbb140677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 shutdown_connections 2024-09-13T17:31:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:31:43.198+0000 7fbb1b232640 1 -- 172.21.15.44:0/1247880347 wait complete. 2024-09-13T17:31:43.210 INFO:tasks.cephfs.filesystem:scrub status for tag:8c636ddb-7209-446f-871b-3e0b22b58ec2 - {'path': '/', 'tag': '8c636ddb-7209-446f-871b-3e0b22b58ec2', 'options': 'recursive,force'} 2024-09-13T17:31:44.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:43 smithi070 ceph-mon[34502]: pgmap v153: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-13T17:31:44.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:43 smithi100 ceph-mon[35055]: pgmap v153: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-13T17:31:44.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:43 smithi044 ceph-mon[31175]: pgmap v153: 129 pgs: 129 active+clean; 3.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-13T17:31:45.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:44 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:31:45.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:44 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:45.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:44 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:31:45.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:44 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:45.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:44 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:31:45.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:44 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:31:45.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:44 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:45.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:44 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:31:45.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:44 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:45.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:44 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:31:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:44 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:31:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:44 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:44 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:31:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:44 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:31:45.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:44 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:31:46.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:45 smithi070 ceph-mon[34502]: pgmap v154: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-13T17:31:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:45 smithi100 ceph-mon[35055]: pgmap v154: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-13T17:31:46.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:45 smithi044 ceph-mon[31175]: pgmap v154: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-13T17:31:48.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:47 smithi070 ceph-mon[34502]: pgmap v155: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:48.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:47 smithi100 ceph-mon[35055]: pgmap v155: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:48.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:47 smithi044 ceph-mon[31175]: pgmap v155: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:50.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:49 smithi070 ceph-mon[34502]: pgmap v156: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-09-13T17:31:50.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:49 smithi100 ceph-mon[35055]: pgmap v156: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-09-13T17:31:50.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:49 smithi044 ceph-mon[31175]: pgmap v156: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-09-13T17:31:52.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:51 smithi070 ceph-mon[34502]: pgmap v157: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:52.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:51 smithi100 ceph-mon[35055]: pgmap v157: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:52.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:51 smithi044 ceph-mon[31175]: pgmap v157: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:54.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:53 smithi070 ceph-mon[34502]: pgmap v158: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:54.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:53 smithi100 ceph-mon[35055]: pgmap v158: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:54.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:53 smithi044 ceph-mon[31175]: pgmap v158: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-13T17:31:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 -- 172.21.15.44:0/2080239296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62941008d0 msgr2=0x7f6294104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 --2- 172.21.15.44:0/2080239296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62941008d0 0x7f6294104910 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6284009930 tx=0x7f628402a420 comp rx=0 tx=0).stop 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 -- 172.21.15.44:0/2080239296 shutdown_connections 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 --2- 172.21.15.44:0/2080239296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294105070 0x7f629410cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 --2- 172.21.15.44:0/2080239296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62941008d0 0x7f6294104910 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 --2- 172.21.15.44:0/2080239296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62940fff20 0x7f6294100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 -- 172.21.15.44:0/2080239296 >> 172.21.15.44:0/2080239296 conn(0x7f62940fb9b0 msgr2=0x7f62940fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 -- 172.21.15.44:0/2080239296 shutdown_connections 2024-09-13T17:31:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.108+0000 7f629c745640 1 -- 172.21.15.44:0/2080239296 wait complete. 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 Processor -- start 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 -- start start 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 0x7f6294196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294105070 0x7f62941966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f629419aac0 0x7f629419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419b4b0 con 0x7f62940fff20 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419b620 con 0x7f6294105070 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629c745640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419b790 con 0x7f629419aac0 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629a4e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 0x7f6294196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629a4e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 0x7f6294196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37272/0 (socket says 172.21.15.44:37272) 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629a4e1640 1 -- 172.21.15.44:0/1972194404 learned_addr learned my addr 172.21.15.44:0/1972194404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f6299ce0640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294105070 0x7f62941966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.109+0000 7f629ace2640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f629419aac0 0x7f629419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629a4e1640 1 -- 172.21.15.44:0/1972194404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f629419aac0 msgr2=0x7f629419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629a4e1640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f629419aac0 0x7f629419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629a4e1640 1 -- 172.21.15.44:0/1972194404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294105070 msgr2=0x7f62941966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629a4e1640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294105070 0x7f62941966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629a4e1640 1 -- 172.21.15.44:0/1972194404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009560 con 0x7f62940fff20 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629ace2640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f629419aac0 0x7f629419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629a4e1640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 0x7f6294196190 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f629000d950 tx=0x7f629000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f628b7fe640 1 -- 172.21.15.44:0/1972194404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290009a30 con 0x7f62940fff20 2024-09-13T17:31:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62941a2d70 con 0x7f62940fff20 2024-09-13T17:31:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f628b7fe640 1 -- 172.21.15.44:0/1972194404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62900044b0 con 0x7f62940fff20 2024-09-13T17:31:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f628b7fe640 1 -- 172.21.15.44:0/1972194404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62900047a0 con 0x7f62940fff20 2024-09-13T17:31:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.110+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62941a31e0 con 0x7f62940fff20 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.112+0000 7f628b7fe640 1 -- 172.21.15.44:0/1972194404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6290004020 con 0x7f62940fff20 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 Processor -- start 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 -- start start 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd51e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd528fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd5294f0 0x5645fd52b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd489c30 con 0x5645fd51e330 2024-09-13T17:31:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f628b7fe640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62800a1200 0x7f62800a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd4873c0 con 0x5645fd51ec50 2024-09-13T17:31:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629f463740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd482cb0 con 0x5645fd5294f0 2024-09-13T17:31:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629ace2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd51e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f6299ce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd5294f0 0x5645fd52b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629ace2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd51e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37280/0 (socket says 172.21.15.44:37280) 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629ace2640 1 -- 172.21.15.44:0/3384581114 learned_addr learned my addr 172.21.15.44:0/3384581114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f628b7fe640 1 -- 172.21.15.44:0/1972194404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f62900cca70 con 0x7f62940fff20 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd528fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.113+0000 7f629ace2640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62800a1200 0x7f62800a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6299ce0640 1 -- 172.21.15.44:0/3384581114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 msgr2=0x5645fd528fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd528fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6299ce0640 1 -- 172.21.15.44:0/3384581114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 msgr2=0x5645fd51e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd51e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6299ce0640 1 -- 172.21.15.44:0/3384581114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fd429910 con 0x5645fd5294f0 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd528fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f629ace2640 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd51e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd5294f0 0x5645fd52b8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f628402a940 tx=0x7f6284005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874bb964dbc45adf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f629ace2640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62800a1200 0x7f62800a36c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f628c014ec0 tx=0x7f628c009240 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6288ff9640 1 -- 172.21.15.44:0/3384581114 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284004230 con 0x5645fd5294f0 2024-09-13T17:31:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6288ff9640 1 -- 172.21.15.44:0/3384581114 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6284031df0 con 0x5645fd5294f0 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.114+0000 7f6288ff9640 1 -- 172.21.15.44:0/3384581114 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284003b30 con 0x5645fd5294f0 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 -- 172.21.15.44:0/3384581114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd5294f0 msgr2=0x5645fd52b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd5294f0 0x5645fd52b8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f628402a940 tx=0x7f6284005c00 comp rx=0 tx=0).stop 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 -- 172.21.15.44:0/3384581114 shutdown_connections 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd5294f0 0x5645fd52b8e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd528fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 --2- 172.21.15.44:0/3384581114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd51e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 -- 172.21.15.44:0/3384581114 >> 172.21.15.44:0/3384581114 conn(0x5645fd51a8c0 msgr2=0x5645fd51aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 -- 172.21.15.44:0/3384581114 shutdown_connections 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 -- 172.21.15.44:0/3384581114 wait complete. 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 -- start start 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.115+0000 7f629f463740 1 Processor -- start 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd528e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51ec50 0x5645fd5261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd5294f0 0x5645fd526770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd440e80 con 0x5645fd51e330 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd440b20 con 0x5645fd5294f0 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd406260 con 0x5645fd51ec50 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 10 client.0 _ll_register_callbacks cb 0x5645fd43bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd528e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd528e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37286/0 (socket says 172.21.15.44:37286) 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 -- 172.21.15.44:0/3876454208 learned_addr learned my addr 172.21.15.44:0/3876454208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51ec50 0x5645fd5261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629ace2640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd5294f0 0x5645fd526770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 -- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51ec50 msgr2=0x5645fd5261d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51ec50 0x5645fd5261d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 -- 172.21.15.44:0/3876454208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd5294f0 msgr2=0x5645fd526770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd5294f0 0x5645fd526770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 -- 172.21.15.44:0/3876454208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629410d400 con 0x5645fd51e330 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51ec50 0x5645fd5261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629ace2640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd5294f0 0x5645fd526770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd528e90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6290099500 tx=0x7f6290099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0755cfb70840355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f627dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f627dffb640 1 -- 172.21.15.44:0/3876454208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62900042d0 con 0x5645fd51e330 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f627dffb640 1 -- 172.21.15.44:0/3876454208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f6290099800 con 0x5645fd51e330 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645fd4ab680 con 0x5645fd51e330 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f627dffb640 1 -- 172.21.15.44:0/3876454208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629009d630 con 0x5645fd51e330 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 10 client.17271 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.116+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645fd481330 con 0x5645fd51e330 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f627dffb640 1 -- 172.21.15.44:0/3876454208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5645fd4ab680 con 0x5645fd51e330 2024-09-13T17:31:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f627dffb640 1 -- 172.21.15.44:0/3876454208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f62900b90b0 con 0x5645fd51e330 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f629f463740 10 client.17271 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f629f463740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f629f463740 20 client.17271 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f629f463740 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd538ee0 0x5645fd53b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f629f463740 4 client.17271 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645fd4765d0 con 0x5645fd538ee0 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.117+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd538ee0 0x5645fd53b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f627dffb640 10 client.17271 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd538ee0 0x5645fd53b2d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f54f9294a120d5a server_cookie=37175b12605b86a8 in_seq=0 out_seq=0 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f627dffb640 1 -- 172.21.15.44:0/3876454208 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5645fd4765d0 con 0x5645fd538ee0 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f627dffb640 10 client.17271 handle_command_reply: tid=0 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 client.17271 shutdown 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 msgr2=0x5645fd528e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd528e90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6290099500 tx=0x7f6290099400 comp rx=0 tx=0).stop 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 shutdown_connections 2024-09-13T17:31:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd538ee0 0x5645fd53b2d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51ec50 0x5645fd5261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd5294f0 0x5645fd526770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 --2- 172.21.15.44:0/3876454208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd51e330 0x5645fd528e90 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f627dffb640 0 client.17271 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 >> 172.21.15.44:0/3876454208 conn(0x5645fd51a8c0 msgr2=0x5645fd519b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 shutdown_connections 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.118+0000 7f629f463740 1 -- 172.21.15.44:0/3876454208 wait complete. 2024-09-13T17:31:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.119+0000 7f629f463740 20 client.17271 trim_cache size 0 max 16384 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 Processor -- start 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 -- start start 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd516120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd53fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd5289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd51b5b0 con 0x5645fd5294f0 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd51f800 con 0x5645fd51ec50 2024-09-13T17:31:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629f463740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd524700 con 0x5645fd51e330 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629ace2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd5289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629ace2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd5289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37298/0 (socket says 172.21.15.44:37298) 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629ace2640 1 -- 172.21.15.44:0/1070464353 learned_addr learned my addr 172.21.15.44:0/1070464353 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f6299ce0640 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd53fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.131+0000 7f629a4e1640 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd516120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629ace2640 1 -- 172.21.15.44:0/1070464353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 msgr2=0x5645fd516120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629ace2640 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd516120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629ace2640 1 -- 172.21.15.44:0/1070464353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 msgr2=0x5645fd53fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629ace2640 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd53fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629ace2640 1 -- 172.21.15.44:0/1070464353 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fd423300 con 0x5645fd5294f0 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629ace2640 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd5289a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f628c011fd0 tx=0x7f628c009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18643bd1d7961b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f62897fa640 1 -- 172.21.15.44:0/1070464353 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c014bb0 con 0x5645fd5294f0 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f62897fa640 1 -- 172.21.15.44:0/1070464353 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f628c01a050 con 0x5645fd5294f0 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f62897fa640 1 -- 172.21.15.44:0/1070464353 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c01fc80 con 0x5645fd5294f0 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 -- 172.21.15.44:0/1070464353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 msgr2=0x5645fd5289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd5289a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f628c011fd0 tx=0x7f628c009790 comp rx=0 tx=0).stop 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 -- 172.21.15.44:0/1070464353 shutdown_connections 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd5289a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd53fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 --2- 172.21.15.44:0/1070464353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd516120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.132+0000 7f629f463740 1 -- 172.21.15.44:0/1070464353 >> 172.21.15.44:0/1070464353 conn(0x5645fd51a8c0 msgr2=0x5645fd515a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 -- 172.21.15.44:0/1070464353 shutdown_connections 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 -- 172.21.15.44:0/1070464353 wait complete. 2024-09-13T17:31:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 -- start start 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 Processor -- start 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd52d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd52d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd58f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd58f8e0 con 0x5645fd5294f0 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd58fa20 con 0x5645fd51ec50 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd58fbe0 con 0x5645fd51e330 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629f463740 10 client.0 _ll_register_callbacks cb 0x7f6284005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629ace2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd58f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629ace2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd58f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37312/0 (socket says 172.21.15.44:37312) 2024-09-13T17:31:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629ace2640 1 -- 172.21.15.44:0/3893883611 learned_addr learned my addr 172.21.15.44:0/3893883611 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f6299ce0640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd52d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.133+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd52d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629ace2640 1 -- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 msgr2=0x5645fd52d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629ace2640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd52d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629ace2640 1 -- 172.21.15.44:0/3893883611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 msgr2=0x5645fd52d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629ace2640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd52d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629ace2640 1 -- 172.21.15.44:0/3893883611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c011d10 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629ace2640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd58f3a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f628c016040 tx=0x7f628c014be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4693ec6a18bff0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f627dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f627dffb640 1 -- 172.21.15.44:0/3893883611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c004e50 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f627dffb640 1 -- 172.21.15.44:0/3893883611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f628c003f80 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6284031df0 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f627dffb640 1 -- 172.21.15.44:0/3893883611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f628c01f950 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629f463740 10 client.17277 fetch_fsmap learned FSMap version 25 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645fd4488e0 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f627dffb640 1 -- 172.21.15.44:0/3893883611 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6284031df0 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.134+0000 7f627dffb640 1 -- 172.21.15.44:0/3893883611 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f628c03a070 con 0x5645fd5294f0 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629f463740 10 client.17277 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629f463740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:31:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629f463740 20 client.17277 populate_metadata read hostname 'smithi044' 2024-09-13T17:31:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629f463740 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd591310 0x5645fd593700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:31:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629f463740 4 client.17277 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:31:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645fd176300 con 0x5645fd591310 2024-09-13T17:31:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd591310 0x5645fd593700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:31:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f629a4e1640 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd591310 0x5645fd593700 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc034fa60ace49f server_cookie=388e5f9db4163afc in_seq=0 out_seq=0 2024-09-13T17:31:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.135+0000 7f627dffb640 10 client.17277 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f627dffb640 1 -- 172.21.15.44:0/3893883611 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645fd176300 con 0x5645fd591310 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f627dffb640 10 client.17277 handle_command_reply: tid=0 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 client.17277 shutdown 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 msgr2=0x5645fd58f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd58f3a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f628c016040 tx=0x7f628c014be0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 shutdown_connections 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5645fd591310 0x5645fd593700 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fd5294f0 0x5645fd58f3a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645fd51ec50 0x5645fd52d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f627dffb640 0 client.17277 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 --2- 172.21.15.44:0/3893883611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645fd51e330 0x5645fd52d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 >> 172.21.15.44:0/3893883611 conn(0x5645fd51a8c0 msgr2=0x5645fd525a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 shutdown_connections 2024-09-13T17:31:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.148+0000 7f629f463740 1 -- 172.21.15.44:0/3893883611 wait complete. 2024-09-13T17:31:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629f463740 20 client.17277 trim_cache size 0 max 16384 2024-09-13T17:31:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:31:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:31:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:31:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62800a1200 msgr2=0x7f62800a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62800a1200 0x7f62800a36c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f628c014ec0 tx=0x7f628c009240 comp rx=0 tx=0).stop 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 msgr2=0x7f6294196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 0x7f6294196190 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f629000d950 tx=0x7f629000de10 comp rx=0 tx=0).stop 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 shutdown_connections 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62800a1200 0x7f62800a36c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f629419aac0 0x7f629419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294105070 0x7f62941966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 --2- 172.21.15.44:0/1972194404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62940fff20 0x7f6294196190 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:31:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 >> 172.21.15.44:0/1972194404 conn(0x7f62940fb9b0 msgr2=0x7f62940fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:31:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 shutdown_connections 2024-09-13T17:31:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:31:55.149+0000 7f629c745640 1 -- 172.21.15.44:0/1972194404 wait complete. 2024-09-13T17:31:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:31:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:31:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:31:55.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:54 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:31:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:31:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:31:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:31:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:54 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:31:55.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:31:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:31:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:31:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:54 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:31:56.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:55 smithi070 ceph-mon[34502]: pgmap v159: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-13T17:31:56.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:55 smithi100 ceph-mon[35055]: pgmap v159: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-13T17:31:56.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:55 smithi044 ceph-mon[31175]: pgmap v159: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-13T17:31:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:57 smithi070 ceph-mon[34502]: pgmap v160: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-13T17:31:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:57 smithi100 ceph-mon[35055]: pgmap v160: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-13T17:31:58.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:57 smithi044 ceph-mon[31175]: pgmap v160: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-13T17:32:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:31:59 smithi070 ceph-mon[34502]: pgmap v161: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:32:00.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:31:59 smithi100 ceph-mon[35055]: pgmap v161: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:32:00.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:31:59 smithi044 ceph-mon[31175]: pgmap v161: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-13T17:32:02.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:01 smithi070 ceph-mon[34502]: pgmap v162: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-13T17:32:02.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:01 smithi100 ceph-mon[35055]: pgmap v162: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-13T17:32:02.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:01 smithi044 ceph-mon[31175]: pgmap v162: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-13T17:32:04.159 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:03 smithi044 ceph-mon[31175]: pgmap v163: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-13T17:32:04.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:03 smithi070 ceph-mon[34502]: pgmap v163: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-13T17:32:04.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:03 smithi100 ceph-mon[35055]: pgmap v163: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-13T17:32:06.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:05 smithi070 ceph-mon[34502]: pgmap v164: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-09-13T17:32:06.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:05 smithi100 ceph-mon[35055]: pgmap v164: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-09-13T17:32:06.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:05 smithi044 ceph-mon[31175]: pgmap v164: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-09-13T17:32:08.073 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.073 INFO:tasks.workunit.client.0.smithi044.stdout:189/600: open fd = 3 2024-09-13T17:32:08.073 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.073 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:190/600: open fd = 3 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:191/600: open fd = 3 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:192/600: open fd = 3 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.074 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:193/600: open fd = 3 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:194/600: open fd = 3 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:195/600: open fd = 3 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:196/600: open fd = 3 2024-09-13T17:32:08.075 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:197/600: open fd = 3 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:198/600: open fd = 3 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:199/600: open fd = 3 2024-09-13T17:32:08.076 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:200/600: open fd = 3 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:201/600: open fd = 3 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.077 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:202/600: open fd = 3 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:203/600: open fd = 3 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.078 INFO:tasks.workunit.client.0.smithi044.stdout:204/600: open fd = 3 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:205/600: open fd = 3 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:206/600: open fd = 3 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:207/600: open fd = 3 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.079 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:208/600: open fd = 3 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:209/600: open fd = 3 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:210/600: open fd = 3 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.080 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:211/600: open fd = 3 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:212/600: open fd = 3 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:213/600: open fd = 3 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.081 INFO:tasks.workunit.client.0.smithi044.stdout:214/600: open fd = 3 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:215/600: open fd = 3 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:216/600: open fd = 3 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:217/600: open fd = 3 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.082 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:218/600: open fd = 3 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:219/600: open fd = 3 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:220/600: open fd = 3 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.083 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:221/600: open fd = 3 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:222/600: open fd = 3 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:223/600: open fd = 3 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:224/600: open fd = 3 2024-09-13T17:32:08.084 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:225/600: open fd = 3 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:226/600: open fd = 3 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:227/600: open fd = 3 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.085 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:228/600: open fd = 3 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:229/600: open fd = 3 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:230/600: open fd = 3 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:231/600: open fd = 3 2024-09-13T17:32:08.086 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:232/600: open fd = 3 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:233/600: open fd = 3 2024-09-13T17:32:08.087 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.088 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.088 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.088 INFO:tasks.workunit.client.0.smithi044.stdout:234/600: open fd = 3 2024-09-13T17:32:08.088 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.088 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:235/600: open fd = 3 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:236/600: open fd = 3 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:237/600: open fd = 3 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.089 INFO:tasks.workunit.client.0.smithi044.stdout:238/600: open fd = 3 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:239/600: open fd = 3 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:240/600: open fd = 3 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:241/600: open fd = 3 2024-09-13T17:32:08.090 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:242/600: open fd = 3 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:243/600: open fd = 3 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:244/600: open fd = 3 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.091 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:245/600: open fd = 3 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:246/600: open fd = 3 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:247/600: open fd = 3 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.092 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:248/600: open fd = 3 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:249/600: open fd = 3 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:250/600: open fd = 3 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.093 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:32:08.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:07 smithi070 ceph-mon[34502]: pgmap v165: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-13T17:32:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 -- 172.21.15.44:0/2935948947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4240ff7d0 msgr2=0x7fa424103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 --2- 172.21.15.44:0/2935948947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4240ff7d0 0x7fa424103810 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa418009930 tx=0x7fa41802a420 comp rx=0 tx=0).stop 2024-09-13T17:32:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 -- 172.21.15.44:0/2935948947 shutdown_connections 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 --2- 172.21.15.44:0/2935948947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa424103f70 0x7fa42410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 --2- 172.21.15.44:0/2935948947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4240ff7d0 0x7fa424103810 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 --2- 172.21.15.44:0/2935948947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4240fee20 0x7fa4240ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 -- 172.21.15.44:0/2935948947 >> 172.21.15.44:0/2935948947 conn(0x7fa4240fa8f0 msgr2=0x7fa4240fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 -- 172.21.15.44:0/2935948947 shutdown_connections 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 -- 172.21.15.44:0/2935948947 wait complete. 2024-09-13T17:32:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.248+0000 7fa429eb7640 1 Processor -- start 2024-09-13T17:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 -- start start 2024-09-13T17:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4240fee20 0x7fa424197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa424103f70 0x7fa4241975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 0x7fa4241a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4241a2780 con 0x7fa424197ae0 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4241a28f0 con 0x7fa424103f70 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa429eb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4241a2ab0 con 0x7fa4240fee20 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 0x7fa4241a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 0x7fa4241a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40802/0 (socket says 172.21.15.44:40802) 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 -- 172.21.15.44:0/3199858685 learned_addr learned my addr 172.21.15.44:0/3199858685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa422ffd640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa424103f70 0x7fa4241975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa4237fe640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4240fee20 0x7fa424197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 -- 172.21.15.44:0/3199858685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4240fee20 msgr2=0x7fa424197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4240fee20 0x7fa424197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 -- 172.21.15.44:0/3199858685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa424103f70 msgr2=0x7fa4241975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa424103f70 0x7fa4241975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa423fff640 1 -- 172.21.15.44:0/3199858685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009560 con 0x7fa424197ae0 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.249+0000 7fa4237fe640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4240fee20 0x7fa424197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.250+0000 7fa423fff640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 0x7fa4241a20e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00eb10 tx=0x7fa41c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.250+0000 7fa420ff9640 1 -- 172.21.15.44:0/3199858685 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c004b20 con 0x7fa424197ae0 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.250+0000 7fa420ff9640 1 -- 172.21.15.44:0/3199858685 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41c004c80 con 0x7fa424197ae0 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.250+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4241a2d90 con 0x7fa424197ae0 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.250+0000 7fa420ff9640 1 -- 172.21.15.44:0/3199858685 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c005a10 con 0x7fa424197ae0 2024-09-13T17:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.250+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4241a3270 con 0x7fa424197ae0 2024-09-13T17:32:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.252+0000 7fa420ff9640 1 -- 172.21.15.44:0/3199858685 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa41c005020 con 0x7fa424197ae0 2024-09-13T17:32:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.252+0000 7fa42cbdb740 1 Processor -- start 2024-09-13T17:32:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 -- start start 2024-09-13T17:32:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862f3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8863008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88625ec30 con 0x55f8862f3c50 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88625c3c0 con 0x55f8862f3330 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f886257cb0 con 0x55f8862fe4f0 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa420ff9640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa40c0a1200 0x7fa40c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40804/0 (socket says 172.21.15.44:40804) 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 -- 172.21.15.44:0/2429284515 learned_addr learned my addr 172.21.15.44:0/2429284515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa423fff640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa40c0a1200 0x7fa40c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8863008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 -- 172.21.15.44:0/2429284515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 msgr2=0x55f8863008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8863008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 -- 172.21.15.44:0/2429284515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 msgr2=0x55f8862f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa4237fe640 1 -- 172.21.15.44:0/2429284515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8861fe910 con 0x55f8862f3c50 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa420ff9640 1 -- 172.21.15.44:0/3199858685 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa41c014070 con 0x7fa424197ae0 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.253+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8863008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.254+0000 7fa4237fe640 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fdfb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa41400bd40 tx=0x7fa41400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f108df9b370bc41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.254+0000 7fa423fff640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa40c0a1200 0x7fa40c0a36c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa41c09a090 tx=0x7fa41c06d520 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.254+0000 7fa409ffb640 1 -- 172.21.15.44:0/2429284515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41401a660 con 0x55f8862f3c50 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.254+0000 7fa409ffb640 1 -- 172.21.15.44:0/2429284515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41400be30 con 0x55f8862f3c50 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.254+0000 7fa409ffb640 1 -- 172.21.15.44:0/2429284515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414011aa0 con 0x55f8862f3c50 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2429284515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 msgr2=0x55f8862fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fdfb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa41400bd40 tx=0x7fa41400f730 comp rx=0 tx=0).stop 2024-09-13T17:32:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2429284515 shutdown_connections 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8863008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fdfb0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2429284515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2429284515 >> 172.21.15.44:0/2429284515 conn(0x55f8862ef8c0 msgr2=0x55f8862efca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2429284515 shutdown_connections 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2429284515 wait complete. 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 -- start start 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.255+0000 7fa42cbdb740 1 Processor -- start 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862fded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8862fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f886215e80 con 0x55f8862f3c50 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f886215b20 con 0x55f8862f3330 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8861db260 con 0x55f8862fe4f0 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa42cbdb740 10 client.0 _ll_register_callbacks cb 0x55f886210df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40820/0 (socket says 172.21.15.44:40820) 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 -- 172.21.15.44:0/748014694 learned_addr learned my addr 172.21.15.44:0/748014694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa422ffd640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8862fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa422ffd640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 -- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 msgr2=0x55f8862fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8862fb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 -- 172.21.15.44:0/748014694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 msgr2=0x55f8862fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 -- 172.21.15.44:0/748014694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41400b9a0 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa422ffd640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8862fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa4237fe640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fb150 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa414011480 tx=0x7fa414005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7387a9b4d1106c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.256+0000 7fa402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa402ffd640 1 -- 172.21.15.44:0/748014694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41401e070 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa402ffd640 1 -- 172.21.15.44:0/748014694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa414011a20 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f886280680 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa402ffd640 1 -- 172.21.15.44:0/748014694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa414005670 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 10 client.17289 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f886256330 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa402ffd640 1 -- 172.21.15.44:0/748014694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f886280680 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa402ffd640 1 -- 172.21.15.44:0/748014694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa414004c70 con 0x55f8862f3c50 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 10 client.17289 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 20 client.17289 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f88630ddd0 0x55f8863101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 4 client.17289 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f88624b5d0 con 0x55f88630ddd0 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.257+0000 7fa422ffd640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f88630ddd0 0x55f8863101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.258+0000 7fa402ffd640 10 client.17289 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.258+0000 7fa422ffd640 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f88630ddd0 0x55f8863101f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a067e3eb87ac164d server_cookie=be52d164e51a069c in_seq=0 out_seq=0 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa402ffd640 1 -- 172.21.15.44:0/748014694 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f88624b5d0 con 0x55f88630ddd0 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa402ffd640 10 client.17289 handle_command_reply: tid=0 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 client.17289 shutdown 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 msgr2=0x55f8862fb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fb150 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa414011480 tx=0x7fa414005270 comp rx=0 tx=0).stop 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 shutdown_connections 2024-09-13T17:32:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f88630ddd0 0x55f8863101f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862fe4f0 0x55f8862fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862fb150 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa402ffd640 0 client.17289 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/748014694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8862fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 >> 172.21.15.44:0/748014694 conn(0x55f8862ef8c0 msgr2=0x55f8862eeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 shutdown_connections 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 1 -- 172.21.15.44:0/748014694 wait complete. 2024-09-13T17:32:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.259+0000 7fa42cbdb740 20 client.17289 trim_cache size 0 max 16384 2024-09-13T17:32:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 Processor -- start 2024-09-13T17:32:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 -- start start 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862ef8c0 0x55f8862ef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8863055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88624b5d0 con 0x55f8862f3c50 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f886280680 con 0x55f8862f3330 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f886305ae0 con 0x55f8862ef8c0 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862f9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40828/0 (socket says 172.21.15.44:40828) 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 -- 172.21.15.44:0/2910438511 learned_addr learned my addr 172.21.15.44:0/2910438511 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862ef8c0 0x55f8862ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa4237fe640 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8863055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 -- 172.21.15.44:0/2910438511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862ef8c0 msgr2=0x55f8862ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862ef8c0 0x55f8862ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 -- 172.21.15.44:0/2910438511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 msgr2=0x55f8863055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8863055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 -- 172.21.15.44:0/2910438511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8861f8300 con 0x55f8862f3c50 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.272+0000 7fa422ffd640 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862f9a00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa418005ec0 tx=0x7fa418000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c56adc66bfc23c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa40a7fc640 1 -- 172.21.15.44:0/2910438511 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418042040 con 0x55f8862f3c50 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa40a7fc640 1 -- 172.21.15.44:0/2910438511 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa418049720 con 0x55f8862f3c50 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa40a7fc640 1 -- 172.21.15.44:0/2910438511 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41803d6b0 con 0x55f8862f3c50 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2910438511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 msgr2=0x55f8862f9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862f9a00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa418005ec0 tx=0x7fa418000bc0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2910438511 shutdown_connections 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862f3c50 0x55f8862f9a00 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3330 0x55f8863055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/2910438511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862ef8c0 0x55f8862ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2910438511 >> 172.21.15.44:0/2910438511 conn(0x55f8862fe030 msgr2=0x55f8862efca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2910438511 shutdown_connections 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.273+0000 7fa42cbdb740 1 -- 172.21.15.44:0/2910438511 wait complete. 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 -- start start 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 Processor -- start 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 0x55f886361120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862f3330 0x55f886361660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3c50 0x55f88635c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88635cdf0 con 0x55f8862ef8c0 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88635cf60 con 0x55f8862f3c50 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88635d120 con 0x55f8862f3330 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa42cbdb740 10 client.0 _ll_register_callbacks cb 0x7fa41400b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa422ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 0x55f886361120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa422ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 0x55f886361120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40832/0 (socket says 172.21.15.44:40832) 2024-09-13T17:32:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.274+0000 7fa422ffd640 1 -- 172.21.15.44:0/691011089 learned_addr learned my addr 172.21.15.44:0/691011089 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa4237fe640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862f3330 0x55f886361660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3c50 0x55f88635c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 -- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862f3330 msgr2=0x55f886361660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862f3330 0x55f886361660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 -- 172.21.15.44:0/691011089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3c50 msgr2=0x55f88635c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3c50 0x55f88635c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 -- 172.21.15.44:0/691011089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa418051040 con 0x55f8862ef8c0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa422ffd640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 0x55f886361120 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa41802a940 tx=0x7fa418005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51e8e2c9d99390c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa402ffd640 1 -- 172.21.15.44:0/691011089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418042030 con 0x55f8862ef8c0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa402ffd640 1 -- 172.21.15.44:0/691011089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fa418003870 con 0x55f8862ef8c0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa41400be30 con 0x55f8862ef8c0 2024-09-13T17:32:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa402ffd640 1 -- 172.21.15.44:0/691011089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa418049890 con 0x55f8862ef8c0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa42cbdb740 10 client.17295 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.275+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f88621d8e0 con 0x55f8862ef8c0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa402ffd640 1 -- 172.21.15.44:0/691011089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41400be30 con 0x55f8862ef8c0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa402ffd640 1 -- 172.21.15.44:0/691011089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa41803dc70 con 0x55f8862ef8c0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa42cbdb740 10 client.17295 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa42cbdb740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa42cbdb740 20 client.17295 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8862fe4f0 0x55f88636bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa42cbdb740 4 client.17295 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f885f4b300 con 0x55f8862fe4f0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa4237fe640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8862fe4f0 0x55f88636bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa4237fe640 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8862fe4f0 0x55f88636bd30 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1533fbb5bb12860 server_cookie=9ac446752ccce8c3 in_seq=0 out_seq=0 2024-09-13T17:32:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.276+0000 7fa402ffd640 10 client.17295 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:08.283 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:07 smithi100 ceph-mon[35055]: pgmap v165: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa402ffd640 1 -- 172.21.15.44:0/691011089 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f885f4b300 con 0x55f8862fe4f0 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa402ffd640 10 client.17295 handle_command_reply: tid=0 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 client.17295 shutdown 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 msgr2=0x55f886361120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 0x55f886361120 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa41802a940 tx=0x7fa418005850 comp rx=0 tx=0).stop 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 shutdown_connections 2024-09-13T17:32:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55f8862fe4f0 0x55f88636bd30 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8862f3330 0x55f886361660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8862f3c50 0x55f88635c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa402ffd640 0 client.17295 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 --2- 172.21.15.44:0/691011089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8862ef8c0 0x55f886361120 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 >> 172.21.15.44:0/691011089 conn(0x55f8862fe030 msgr2=0x55f8862f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 shutdown_connections 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 1 -- 172.21.15.44:0/691011089 wait complete. 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa42cbdb740 20 client.17295 trim_cache size 0 max 16384 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:32:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:32:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:32:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:32:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa40c0a1200 msgr2=0x7fa40c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa40c0a1200 0x7fa40c0a36c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa41c09a090 tx=0x7fa41c06d520 comp rx=0 tx=0).stop 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 msgr2=0x7fa4241a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 0x7fa4241a20e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00eb10 tx=0x7fa41c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 shutdown_connections 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa40c0a1200 0x7fa40c0a36c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa424197ae0 0x7fa4241a20e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa424103f70 0x7fa4241975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 --2- 172.21.15.44:0/3199858685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4240fee20 0x7fa424197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.291+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 >> 172.21.15.44:0/3199858685 conn(0x7fa4240fa8f0 msgr2=0x7fa4240fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.292+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 shutdown_connections 2024-09-13T17:32:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:08.292+0000 7fa429eb7640 1 -- 172.21.15.44:0/3199858685 wait complete. 2024-09-13T17:32:08.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:07 smithi044 ceph-mon[31175]: pgmap v165: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-13T17:32:10.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:09 smithi070 ceph-mon[34502]: pgmap v166: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 81 KiB/s wr, 35 op/s 2024-09-13T17:32:10.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:09 smithi100 ceph-mon[35055]: pgmap v166: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 81 KiB/s wr, 35 op/s 2024-09-13T17:32:10.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:09 smithi044 ceph-mon[31175]: pgmap v166: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 81 KiB/s wr, 35 op/s 2024-09-13T17:32:12.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:11 smithi070 ceph-mon[34502]: pgmap v167: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-13T17:32:12.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:11 smithi100 ceph-mon[35055]: pgmap v167: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-13T17:32:12.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:11 smithi044 ceph-mon[31175]: pgmap v167: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-13T17:32:13.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:32:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 -- 172.21.15.44:0/728590574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 msgr2=0x7f21e010e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 --2- 172.21.15.44:0/728590574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e010e9d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009900 tx=0x7f21d802a410 comp rx=0 tx=0).stop 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 -- 172.21.15.44:0/728590574 shutdown_connections 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 --2- 172.21.15.44:0/728590574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e00ff3e0 0x7f21e010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 --2- 172.21.15.44:0/728590574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e010e9d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 --2- 172.21.15.44:0/728590574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e00feac0 0x7f21e00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 -- 172.21.15.44:0/728590574 >> 172.21.15.44:0/728590574 conn(0x7f21e00fa8c0 msgr2=0x7f21e00fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 -- 172.21.15.44:0/728590574 shutdown_connections 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.328+0000 7f21e709c640 1 -- 172.21.15.44:0/728590574 wait complete. 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 Processor -- start 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 -- start start 2024-09-13T17:32:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e00feac0 0x7f21e01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e00ff3e0 0x7f21e0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e019d320 con 0x7f21e010c5e0 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e019d490 con 0x7f21e00feac0 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e709c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e019d650 con 0x7f21e00ff3e0 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e5639640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e5639640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55322/0 (socket says 172.21.15.44:55322) 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.329+0000 7f21e5639640 1 -- 172.21.15.44:0/1417186474 learned_addr learned my addr 172.21.15.44:0/1417186474 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e4e38640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e00feac0 0x7f21e01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21dffff640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e00ff3e0 0x7f21e0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e5639640 1 -- 172.21.15.44:0/1417186474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e00ff3e0 msgr2=0x7f21e0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e5639640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e00ff3e0 0x7f21e0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e5639640 1 -- 172.21.15.44:0/1417186474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e00feac0 msgr2=0x7f21e01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e5639640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e00feac0 0x7f21e01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e5639640 1 -- 172.21.15.44:0/1417186474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009560 con 0x7f21e010c5e0 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e5639640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e019cd20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f21d802aa00 tx=0x7f21d8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21ddffb640 1 -- 172.21.15.44:0/1417186474 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80044f0 con 0x7f21e010c5e0 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21ddffb640 1 -- 172.21.15.44:0/1417186474 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21d8032e10 con 0x7f21e010c5e0 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e019d8d0 con 0x7f21e010c5e0 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21ddffb640 1 -- 172.21.15.44:0/1417186474 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8003850 con 0x7f21e010c5e0 2024-09-13T17:32:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.330+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e01a51b0 con 0x7f21e010c5e0 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21ddffb640 1 -- 172.21.15.44:0/1417186474 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f21d80048c0 con 0x7f21e010c5e0 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 Processor -- start 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 -- start start 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbc71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddb31f20 con 0x5599ddbd1fd0 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddb2f6b0 con 0x5599ddbc6e10 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddb2afa0 con 0x5599ddbc7730 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21ddffb640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21d00a1280 0x7f21d00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.333+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55324/0 (socket says 172.21.15.44:55324) 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 -- 172.21.15.44:0/2601204357 learned_addr learned my addr 172.21.15.44:0/2601204357 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21e4e38640 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbc71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21e5639640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21d00a1280 0x7f21d00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 -- 172.21.15.44:0/2601204357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 msgr2=0x5599ddbd1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21ddffb640 1 -- 172.21.15.44:0/1417186474 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f21d80edcc0 con 0x7f21e010c5e0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 -- 172.21.15.44:0/2601204357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 msgr2=0x5599ddbc71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbc71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 -- 172.21.15.44:0/2601204357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ddae5840 con 0x5599ddbd1fd0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21dffff640 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd43c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f21cc012420 tx=0x7f21cc00e370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b002a3063a46776 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21e5639640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21d00a1280 0x7f21d00a3740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21d80b7520 tx=0x7f21d8009260 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21caffd640 1 -- 172.21.15.44:0/2601204357 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc018070 con 0x5599ddbd1fd0 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21caffd640 1 -- 172.21.15.44:0/2601204357 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21cc0054b0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.334+0000 7f21caffd640 1 -- 172.21.15.44:0/2601204357 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0103d0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 -- 172.21.15.44:0/2601204357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 msgr2=0x5599ddbd43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd43c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f21cc012420 tx=0x7f21cc00e370 comp rx=0 tx=0).stop 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 -- 172.21.15.44:0/2601204357 shutdown_connections 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd43c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/2601204357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbc71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 -- 172.21.15.44:0/2601204357 >> 172.21.15.44:0/2601204357 conn(0x5599ddbc3380 msgr2=0x5599ddbc3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 -- 172.21.15.44:0/2601204357 shutdown_connections 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 -- 172.21.15.44:0/2601204357 wait complete. 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 -- start start 2024-09-13T17:32:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.335+0000 7f21e9dc7740 1 Processor -- start 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc6e10 0x5599ddbd19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc7730 0x5599ddbced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbcf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddacc2a0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd803430 con 0x5599ddbc7730 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddaae680 con 0x5599ddbc6e10 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e9dc7740 10 client.0 _ll_register_callbacks cb 0x5599ddb24610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbcf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbcf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55340/0 (socket says 172.21.15.44:55340) 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 -- 172.21.15.44:0/1524833682 learned_addr learned my addr 172.21.15.44:0/1524833682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21e4e38640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc7730 0x5599ddbced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc6e10 0x5599ddbd19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 -- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc6e10 msgr2=0x5599ddbd19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc6e10 0x5599ddbd19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 -- 172.21.15.44:0/1524833682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc7730 msgr2=0x5599ddbced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc7730 0x5599ddbced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.336+0000 7f21dffff640 1 -- 172.21.15.44:0/1524833682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc00ecb0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21dffff640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbcf330 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e340 tx=0x7f21cc004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132be13afef0a15f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21bffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21bffff640 1 -- 172.21.15.44:0/1524833682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc018070 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21bffff640 1 -- 172.21.15.44:0/1524833682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f21cc004050 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599dd81e4e0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21bffff640 1 -- 172.21.15.44:0/1524833682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21cc004210 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21e9dc7740 10 client.17307 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599ddb29620 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21bffff640 1 -- 172.21.15.44:0/1524833682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599dd81e4e0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21bffff640 1 -- 172.21.15.44:0/1524833682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21cc010530 con 0x5599ddbd1fd0 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21e9dc7740 10 client.17307 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21e9dc7740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.337+0000 7f21e9dc7740 20 client.17307 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.338+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddbe1dd0 0x5599ddbe41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.338+0000 7f21e9dc7740 4 client.17307 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.338+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599dd82f7e0 con 0x5599ddbe1dd0 2024-09-13T17:32:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.338+0000 7f21e4e38640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddbe1dd0 0x5599ddbe41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.338+0000 7f21bffff640 10 client.17307 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.338+0000 7f21e4e38640 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddbe1dd0 0x5599ddbe41f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced4c292a94a34 server_cookie=99a9aae2bd7ff7c8 in_seq=0 out_seq=0 2024-09-13T17:32:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21bffff640 1 -- 172.21.15.44:0/1524833682 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5599dd82f7e0 con 0x5599ddbe1dd0 2024-09-13T17:32:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21bffff640 10 client.17307 handle_command_reply: tid=0 2024-09-13T17:32:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 client.17307 shutdown 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 msgr2=0x5599ddbcf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbcf330 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e340 tx=0x7f21cc004b00 comp rx=0 tx=0).stop 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 shutdown_connections 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddbe1dd0 0x5599ddbe41f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbcf330 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc7730 0x5599ddbced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21bffff640 0 client.17307 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/1524833682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc6e10 0x5599ddbd19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 >> 172.21.15.44:0/1524833682 conn(0x5599ddbc3380 msgr2=0x5599ddbc9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 shutdown_connections 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 1 -- 172.21.15.44:0/1524833682 wait complete. 2024-09-13T17:32:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.339+0000 7f21e9dc7740 20 client.17307 trim_cache size 0 max 16384 2024-09-13T17:32:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.353+0000 7f21e9dc7740 1 Processor -- start 2024-09-13T17:32:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 -- start start 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbc9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd81e4e0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddbe1c00 con 0x5599ddbc6e10 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddbc6b90 con 0x5599ddbc7730 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbc9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbc9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55348/0 (socket says 172.21.15.44:55348) 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 -- 172.21.15.44:0/627519608 learned_addr learned my addr 172.21.15.44:0/627519608 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21e4e38640 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 -- 172.21.15.44:0/627519608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 msgr2=0x5599ddbd92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 -- 172.21.15.44:0/627519608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 msgr2=0x5599ddbd8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 -- 172.21.15.44:0/627519608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ddacb790 con 0x5599ddbd1fd0 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21dffff640 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbc9fd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f21cc004c50 tx=0x7f21cc00e370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84109210a7c5752 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.354+0000 7f21cb7fe640 1 -- 172.21.15.44:0/627519608 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc018040 con 0x5599ddbd1fd0 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21cb7fe640 1 -- 172.21.15.44:0/627519608 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21cc004940 con 0x5599ddbd1fd0 2024-09-13T17:32:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21cb7fe640 1 -- 172.21.15.44:0/627519608 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc012030 con 0x5599ddbd1fd0 2024-09-13T17:32:13.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 -- 172.21.15.44:0/627519608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 msgr2=0x5599ddbc9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbc9fd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f21cc004c50 tx=0x7f21cc00e370 comp rx=0 tx=0).stop 2024-09-13T17:32:13.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 -- 172.21.15.44:0/627519608 shutdown_connections 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbc9fd0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/627519608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 -- 172.21.15.44:0/627519608 >> 172.21.15.44:0/627519608 conn(0x5599ddbc3380 msgr2=0x5599ddbc3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 -- 172.21.15.44:0/627519608 shutdown_connections 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 -- 172.21.15.44:0/627519608 wait complete. 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.355+0000 7f21e9dc7740 1 -- start start 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 Processor -- start 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddc37390 con 0x5599ddbd1fd0 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddc37500 con 0x5599ddbc6e10 2024-09-13T17:32:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ddc376c0 con 0x5599ddbc7730 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e9dc7740 10 client.0 _ll_register_callbacks cb 0x7f21d40096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55360/0 (socket says 172.21.15.44:55360) 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 -- 172.21.15.44:0/3787456271 learned_addr learned my addr 172.21.15.44:0/3787456271 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e4e38640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 -- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 msgr2=0x5599ddbd5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 -- 172.21.15.44:0/3787456271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 msgr2=0x5599ddbd4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21dffff640 1 -- 172.21.15.44:0/3787456271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599ddb29620 con 0x5599ddbd1fd0 2024-09-13T17:32:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.356+0000 7f21e4e38640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21dffff640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd57c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005740 tx=0x7f21cc010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e24fd670489a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21bffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21bffff640 1 -- 172.21.15.44:0/3787456271 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc018040 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21bffff640 1 -- 172.21.15.44:0/3787456271 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f21cc0051f0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21cc0054b0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21bffff640 1 -- 172.21.15.44:0/3787456271 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21cc010440 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 10 client.17313 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599ddaf0ea0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21bffff640 1 -- 172.21.15.44:0/3787456271 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21cc0054b0 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21bffff640 1 -- 172.21.15.44:0/3787456271 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21cc01d760 con 0x5599ddbd1fd0 2024-09-13T17:32:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 10 client.17313 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 20 client.17313 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddc389a0 0x5599ddc3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 4 client.17313 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.357+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599dd882e30 con 0x5599ddc389a0 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e4e38640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddc389a0 0x5599ddc3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21bffff640 10 client.17313 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e4e38640 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddc389a0 0x5599ddc3ad90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de46a5a1e7313b51 server_cookie=ce85fb9cc27f3511 in_seq=0 out_seq=0 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21bffff640 1 -- 172.21.15.44:0/3787456271 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5599dd882e30 con 0x5599ddc389a0 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21bffff640 10 client.17313 handle_command_reply: tid=0 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 client.17313 shutdown 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 msgr2=0x5599ddbd57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd57c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005740 tx=0x7f21cc010e10 comp rx=0 tx=0).stop 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 shutdown_connections 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5599ddc389a0 0x5599ddc3ad90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599ddbc7730 0x5599ddbd5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599ddbd1fd0 0x5599ddbd57c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21bffff640 0 client.17313 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 --2- 172.21.15.44:0/3787456271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5599ddbc6e10 0x5599ddbd4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 >> 172.21.15.44:0/3787456271 conn(0x5599ddbc3380 msgr2=0x5599ddbc9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 shutdown_connections 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.358+0000 7f21e9dc7740 1 -- 172.21.15.44:0/3787456271 wait complete. 2024-09-13T17:32:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e9dc7740 20 client.17313 trim_cache size 0 max 16384 2024-09-13T17:32:13.367 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:32:13.367 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:32:13.367 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:32:13.367 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21d00a1280 msgr2=0x7f21d00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21d00a1280 0x7f21d00a3740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21d80b7520 tx=0x7f21d8009260 comp rx=0 tx=0).stop 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 msgr2=0x7f21e019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e019cd20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f21d802aa00 tx=0x7f21d8032c20 comp rx=0 tx=0).stop 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 shutdown_connections 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21d00a1280 0x7f21d00a3740 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e00ff3e0 0x7f21e0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21e010c5e0 0x7f21e019cd20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 --2- 172.21.15.44:0/1417186474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e00feac0 0x7f21e01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 >> 172.21.15.44:0/1417186474 conn(0x7f21e00fa8c0 msgr2=0x7f21e00fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 shutdown_connections 2024-09-13T17:32:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.359+0000 7f21e709c640 1 -- 172.21.15.44:0/1417186474 wait complete. 2024-09-13T17:32:13.371 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:32:13.371 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:32:13.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 -- 172.21.15.44:0/2140026901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 msgr2=0x7f633810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 --2- 172.21.15.44:0/2140026901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f633810bb50 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6330009cd0 tx=0x7f633002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 -- 172.21.15.44:0/2140026901 shutdown_connections 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 --2- 172.21.15.44:0/2140026901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63380ff860 0x7f63381038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 --2- 172.21.15.44:0/2140026901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f633810bb50 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 --2- 172.21.15.44:0/2140026901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63380feeb0 0x7f63380ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 -- 172.21.15.44:0/2140026901 >> 172.21.15.44:0/2140026901 conn(0x7f63380fa900 msgr2=0x7f63380fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 -- 172.21.15.44:0/2140026901 shutdown_connections 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.475+0000 7f633dbae640 1 -- 172.21.15.44:0/2140026901 wait complete. 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 Processor -- start 2024-09-13T17:32:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 -- start start 2024-09-13T17:32:13.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63380feeb0 0x7f63381960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f6338196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633819aa20 0x7f633819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633819b410 con 0x7f6338104000 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633819b580 con 0x7f63380feeb0 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f633dbae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633819b6f0 con 0x7f633819aa20 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f6338196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f6338196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55376/0 (socket says 172.21.15.44:55376) 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f6336ffd640 1 -- 172.21.15.44:0/1515744175 learned_addr learned my addr 172.21.15.44:0/1515744175 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f63377fe640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63380feeb0 0x7f63381960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.476+0000 7f6337fff640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633819aa20 0x7f633819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6336ffd640 1 -- 172.21.15.44:0/1515744175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633819aa20 msgr2=0x7f633819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6336ffd640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633819aa20 0x7f633819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6336ffd640 1 -- 172.21.15.44:0/1515744175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63380feeb0 msgr2=0x7f63381960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6336ffd640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63380feeb0 0x7f63381960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6336ffd640 1 -- 172.21.15.44:0/1515744175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6330009900 con 0x7f6338104000 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6337fff640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633819aa20 0x7f633819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6336ffd640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f6338196630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f632c00c970 tx=0x7f632c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00cea0 con 0x7f6338104000 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f632c007bb0 con 0x7f6338104000 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c018660 con 0x7f6338104000 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63381a2cd0 con 0x7f6338104000 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.477+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63381a3130 con 0x7f6338104000 2024-09-13T17:32:13.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.478+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6338100380 con 0x7f6338104000 2024-09-13T17:32:13.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.479+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f632c007d20 con 0x7f6338104000 2024-09-13T17:32:13.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.480+0000 7f6334ff9640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63200a1280 0x7f63200a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.480+0000 7f63377fe640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63200a1280 0x7f63200a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.480+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f632c0cd080 con 0x7f6338104000 2024-09-13T17:32:13.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.481+0000 7f63377fe640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63200a1280 0x7f63200a3740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f632800abe0 tx=0x7f632800a2d0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.484+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f632c096150 con 0x7f6338104000 2024-09-13T17:32:13.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.695+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6338197440 con 0x7f6338104000 2024-09-13T17:32:13.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.695+0000 7f6334ff9640 1 -- 172.21.15.44:0/1515744175 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f632c099770 con 0x7f6338104000 2024-09-13T17:32:13.697 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:32:13.697 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:32:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.697+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63200a1280 msgr2=0x7f63200a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63200a1280 0x7f63200a3740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f632800abe0 tx=0x7f632800a2d0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 msgr2=0x7f6338196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f6338196630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f632c00c970 tx=0x7f632c00cd70 comp rx=0 tx=0).stop 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 shutdown_connections 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63200a1280 0x7f63200a3740 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633819aa20 0x7f633819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6338104000 0x7f6338196630 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 --2- 172.21.15.44:0/1515744175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63380feeb0 0x7f63381960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 >> 172.21.15.44:0/1515744175 conn(0x7f63380fa900 msgr2=0x7f6338068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 shutdown_connections 2024-09-13T17:32:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.698+0000 7f633dbae640 1 -- 172.21.15.44:0/1515744175 wait complete. 2024-09-13T17:32:13.709 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 -- 172.21.15.44:0/819648011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0010a350 msgr2=0x7fca0010c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 --2- 172.21.15.44:0/819648011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0010a350 0x7fca0010c7a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009ca0 tx=0x7fc9f802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 -- 172.21.15.44:0/819648011 shutdown_connections 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 --2- 172.21.15.44:0/819648011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca00101800 0x7fca00109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 --2- 172.21.15.44:0/819648011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0010a350 0x7fca0010c7a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 --2- 172.21.15.44:0/819648011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca00100e50 0x7fca00101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 -- 172.21.15.44:0/819648011 >> 172.21.15.44:0/819648011 conn(0x7fca000fa930 msgr2=0x7fca000fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 -- 172.21.15.44:0/819648011 shutdown_connections 2024-09-13T17:32:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.815+0000 7fca054c5640 1 -- 172.21.15.44:0/819648011 wait complete. 2024-09-13T17:32:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 Processor -- start 2024-09-13T17:32:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 -- start start 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca00100e50 0x7fca00196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 0x7fca001966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca0010a350 0x7fca0019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0019b060 con 0x7fca00101800 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0019b1d0 con 0x7fca00100e50 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fca054c5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0019b390 con 0x7fca0010a350 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 0x7fca001966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.816+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 0x7fca001966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55384/0 (socket says 172.21.15.44:55384) 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/2467321027 learned_addr learned my addr 172.21.15.44:0/2467321027 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9feffd640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca00100e50 0x7fca00196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca0010a350 0x7fca0019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/2467321027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca0010a350 msgr2=0x7fca0019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca0010a350 0x7fca0019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/2467321027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca00100e50 msgr2=0x7fca00196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca00100e50 0x7fca00196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/2467321027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8009900 con 0x7fca00101800 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 0x7fca001966c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400d770 tx=0x7fc9f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9e7fff640 1 -- 172.21.15.44:0/2467321027 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4009c30 con 0x7fca00101800 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9e7fff640 1 -- 172.21.15.44:0/2467321027 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9f400de80 con 0x7fca00101800 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0019b670 con 0x7fca00101800 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fc9e7fff640 1 -- 172.21.15.44:0/2467321027 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4004d40 con 0x7fca00101800 2024-09-13T17:32:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.817+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca001a2ef0 con 0x7fca00101800 2024-09-13T17:32:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.819+0000 7fc9e7fff640 1 -- 172.21.15.44:0/2467321027 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc9f4004ea0 con 0x7fca00101800 2024-09-13T17:32:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 Processor -- start 2024-09-13T17:32:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 -- start start 2024-09-13T17:32:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590c7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560859022cc0 con 0x5608590b8f90 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560859020450 con 0x5608590c7d60 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fca081e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085901bd40 con 0x5608590b8670 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55398/0 (socket says 172.21.15.44:55398) 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 -- 172.21.15.44:0/1661677131 learned_addr learned my addr 172.21.15.44:0/1661677131 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 -- 172.21.15.44:0/1661677131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 msgr2=0x5608590b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 -- 172.21.15.44:0/1661677131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 msgr2=0x5608590ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 -- 172.21.15.44:0/1661677131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560858fe3620 con 0x5608590b8f90 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9e7fff640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9e80a1280 0x7fc9e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9e80a1280 0x7fc9e80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9feffd640 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590c7820 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000bd00 tx=0x7fc9f00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4faf5f69c6bf9c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.820+0000 7fc9e7fff640 1 -- 172.21.15.44:0/2467321027 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc9f40cc980 con 0x7fca00101800 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fc9e57fa640 1 -- 172.21.15.44:0/1661677131 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0005220 con 0x5608590b8f90 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fc9e57fa640 1 -- 172.21.15.44:0/1661677131 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9f0015070 con 0x5608590b8f90 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fc9e57fa640 1 -- 172.21.15.44:0/1661677131 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011440 con 0x5608590b8f90 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9e80a1280 0x7fc9e80a3740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80098d0 tx=0x7fc9f8030f20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 -- 172.21.15.44:0/1661677131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 msgr2=0x5608590c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590c7820 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000bd00 tx=0x7fc9f00107a0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 -- 172.21.15.44:0/1661677131 shutdown_connections 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590c7820 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 --2- 172.21.15.44:0/1661677131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 -- 172.21.15.44:0/1661677131 >> 172.21.15.44:0/1661677131 conn(0x5608590b4d00 msgr2=0x5608590b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 -- 172.21.15.44:0/1661677131 shutdown_connections 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.821+0000 7fca081e2740 1 -- 172.21.15.44:0/1661677131 wait complete. 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 -- start start 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 Processor -- start 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8f90 0x5608590c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560858fda180 con 0x5608590b8670 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560858cf4430 con 0x5608590c7d60 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560858f9f5e0 con 0x5608590b8f90 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fca081e2740 10 client.0 _ll_register_callbacks cb 0x56085900baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55404/0 (socket says 172.21.15.44:55404) 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 -- 172.21.15.44:0/1329922889 learned_addr learned my addr 172.21.15.44:0/1329922889 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 -- 172.21.15.44:0/1329922889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8f90 msgr2=0x5608590c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8f90 0x5608590c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 -- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 msgr2=0x5608590c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.822+0000 7fc9feffd640 1 -- 172.21.15.44:0/1329922889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f000b900 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9feffd640 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590c73e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005d00 tx=0x7fc9f0005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df93edcd53765bba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/1329922889 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0020070 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/1329922889 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc9f0017070 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560859045b80 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/1329922889 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9f00048d0 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 10 client.17331 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56085901a3c0 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/1329922889 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560859045b80 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/1329922889 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9f0025b50 con 0x5608590b8670 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 10 client.17331 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 10 client.17331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:32:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 20 client.17331 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5608590d7620 0x5608590d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 4 client.17331 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.823+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56085900e1f0 con 0x5608590d7620 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.824+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5608590d7620 0x5608590d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.824+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5608590d7620 0x5608590d9a10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f56cdc5da641e9f server_cookie=d0156c4b9c9f5641 in_seq=0 out_seq=0 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.824+0000 7fc9e2ff5640 10 client.17331 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/1329922889 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56085900e1f0 con 0x5608590d7620 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fc9e2ff5640 10 client.17331 handle_command_reply: tid=0 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 client.17331 shutdown 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 msgr2=0x5608590c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590c73e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005d00 tx=0x7fc9f0005130 comp rx=0 tx=0).stop 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 shutdown_connections 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5608590d7620 0x5608590d9a10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8f90 0x5608590c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fc9e2ff5640 0 client.17331 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 --2- 172.21.15.44:0/1329922889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590c73e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 >> 172.21.15.44:0/1329922889 conn(0x5608590b4d00 msgr2=0x5608590baa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 shutdown_connections 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 1 -- 172.21.15.44:0/1329922889 wait complete. 2024-09-13T17:32:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.825+0000 7fca081e2740 20 client.17331 trim_cache size 0 max 16384 2024-09-13T17:32:13.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 Processor -- start 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 -- start start 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590b8f90 0x5608590bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590c7d60 0x5608590cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608590c0410 con 0x5608590b8670 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608590c5c10 con 0x5608590b8f90 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fca081e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608590af3e0 con 0x5608590c7d60 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55406/0 (socket says 172.21.15.44:55406) 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fc9feffd640 1 -- 172.21.15.44:0/2704471760 learned_addr learned my addr 172.21.15.44:0/2704471760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.836+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590b8f90 0x5608590bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9feffd640 1 -- 172.21.15.44:0/2704471760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590c7d60 msgr2=0x5608590cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590c7d60 0x5608590cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9feffd640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590c7d60 0x5608590cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9feffd640 1 -- 172.21.15.44:0/2704471760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590b8f90 msgr2=0x5608590bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9feffd640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590b8f90 0x5608590bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9feffd640 1 -- 172.21.15.44:0/2704471760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560858fbc770 con 0x5608590b8670 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590c7d60 0x5608590cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590b8f90 0x5608590bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9feffd640 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590b46a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005780 tx=0x7fc9f00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfa62876acf97e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9e5ffb640 1 -- 172.21.15.44:0/2704471760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0020040 con 0x5608590b8670 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9e5ffb640 1 -- 172.21.15.44:0/2704471760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9f00053b0 con 0x5608590b8670 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fc9e5ffb640 1 -- 172.21.15.44:0/2704471760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00235d0 con 0x5608590b8670 2024-09-13T17:32:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fca081e2740 1 -- 172.21.15.44:0/2704471760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 msgr2=0x5608590b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fca081e2740 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590b46a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005780 tx=0x7fc9f00107a0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fca081e2740 1 -- 172.21.15.44:0/2704471760 shutdown_connections 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fca081e2740 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590c7d60 0x5608590cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fca081e2740 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590b8f90 0x5608590bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.837+0000 7fca081e2740 1 --2- 172.21.15.44:0/2704471760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8670 0x5608590b46a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- 172.21.15.44:0/2704471760 >> 172.21.15.44:0/2704471760 conn(0x5608590b4d00 msgr2=0x5608590bf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- 172.21.15.44:0/2704471760 shutdown_connections 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- 172.21.15.44:0/2704471760 wait complete. 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- start start 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 Processor -- start 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608590cbad0 con 0x5608590b8f90 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608590cbc40 con 0x5608590c7d60 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608590cbe00 con 0x5608590b8670 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fca081e2740 10 client.0 _ll_register_callbacks cb 0x7fc9f000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55416/0 (socket says 172.21.15.44:55416) 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/3349531798 learned_addr learned my addr 172.21.15.44:0/3349531798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.838+0000 7fc9feffd640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/3349531798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 msgr2=0x5608590cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 msgr2=0x5608590cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9fe7fc640 1 -- 172.21.15.44:0/3349531798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56085901a3c0 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9ff7fe640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9feffd640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9fe7fc640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590cb020 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009610 tx=0x7fc9f40953f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11566a323b52dd89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/3349531798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4099e70 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/3349531798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc9f4099770 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9f0015070 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/3349531798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9f409d5a0 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 10 client.17337 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560858fe1e80 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/3349531798 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9f0015070 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/3349531798 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9f40ca030 con 0x5608590b8f90 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 10 client.17337 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 10 client.17337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 20 client.17337 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x56085912ee60 0x560859131280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 4 client.17337 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.839+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560858d0f4e0 con 0x56085912ee60 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fc9feffd640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x56085912ee60 0x560859131280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fc9feffd640 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x56085912ee60 0x560859131280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a108cc140d179e3 server_cookie=c1b50f12250fe863 in_seq=0 out_seq=0 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fc9e2ff5640 10 client.17337 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fc9e2ff5640 1 -- 172.21.15.44:0/3349531798 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560858d0f4e0 con 0x56085912ee60 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fc9e2ff5640 10 client.17337 handle_command_reply: tid=0 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 client.17337 shutdown 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 msgr2=0x5608590cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590cb020 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009610 tx=0x7fc9f40953f0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 shutdown_connections 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x56085912ee60 0x560859131280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5608590c7d60 0x5608590cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608590b8f90 0x5608590cb020 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fc9e2ff5640 0 client.17337 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 --2- 172.21.15.44:0/3349531798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608590b8670 0x5608590cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 >> 172.21.15.44:0/3349531798 conn(0x5608590b4d00 msgr2=0x5608590ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 shutdown_connections 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.840+0000 7fca081e2740 1 -- 172.21.15.44:0/3349531798 wait complete. 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca081e2740 20 client.17337 trim_cache size 0 max 16384 2024-09-13T17:32:13.849 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:32:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9e80a1280 msgr2=0x7fc9e80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9e80a1280 0x7fc9e80a3740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80098d0 tx=0x7fc9f8030f20 comp rx=0 tx=0).stop 2024-09-13T17:32:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 msgr2=0x7fca001966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 0x7fca001966c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400d770 tx=0x7fc9f400dc30 comp rx=0 tx=0).stop 2024-09-13T17:32:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 shutdown_connections 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9e80a1280 0x7fc9e80a3740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca0010a350 0x7fca0019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca00101800 0x7fca001966c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 --2- 172.21.15.44:0/2467321027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca00100e50 0x7fca00196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 >> 172.21.15.44:0/2467321027 conn(0x7fca000fa930 msgr2=0x7fca000fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 shutdown_connections 2024-09-13T17:32:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.841+0000 7fca054c5640 1 -- 172.21.15.44:0/2467321027 wait complete. 2024-09-13T17:32:13.853 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:32:13.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 -- 172.21.15.44:0/800268867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990100940 msgr2=0x7fe990104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 --2- 172.21.15.44:0/800268867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990100940 0x7fe990104980 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe980009930 tx=0x7fe98002a420 comp rx=0 tx=0).stop 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 -- 172.21.15.44:0/800268867 shutdown_connections 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 --2- 172.21.15.44:0/800268867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9901050e0 0x7fe99010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 --2- 172.21.15.44:0/800268867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe990100940 0x7fe990104980 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 --2- 172.21.15.44:0/800268867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9900fff90 0x7fe990100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.956+0000 7fe9986b1640 1 -- 172.21.15.44:0/800268867 >> 172.21.15.44:0/800268867 conn(0x7fe9900fba20 msgr2=0x7fe9900fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 -- 172.21.15.44:0/800268867 shutdown_connections 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 -- 172.21.15.44:0/800268867 wait complete. 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 Processor -- start 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 -- start start 2024-09-13T17:32:13.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9900fff90 0x7fe990196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9901050e0 0x7fe9901966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 0x7fe99019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019b4a0 con 0x7fe99019aab0 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019b610 con 0x7fe9900fff90 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe9986b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019b780 con 0x7fe9901050e0 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe996c4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 0x7fe99019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe996c4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 0x7fe99019af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55426/0 (socket says 172.21.15.44:55426) 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.957+0000 7fe996c4e640 1 -- 172.21.15.44:0/1724326163 learned_addr learned my addr 172.21.15.44:0/1724326163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe99644d640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9900fff90 0x7fe990196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9901050e0 0x7fe9901966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe996c4e640 1 -- 172.21.15.44:0/1724326163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9901050e0 msgr2=0x7fe9901966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe996c4e640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9901050e0 0x7fe9901966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe996c4e640 1 -- 172.21.15.44:0/1724326163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9900fff90 msgr2=0x7fe990196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe996c4e640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9900fff90 0x7fe990196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe996c4e640 1 -- 172.21.15.44:0/1724326163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980009560 con 0x7fe99019aab0 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe996c4e640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 0x7fe99019af60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe98800cce0 tx=0x7fe98800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe9877fe640 1 -- 172.21.15.44:0/1724326163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800f450 con 0x7fe99019aab0 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9901a2d60 con 0x7fe99019aab0 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe9877fe640 1 -- 172.21.15.44:0/1724326163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe98800fa40 con 0x7fe99019aab0 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe9877fe640 1 -- 172.21.15.44:0/1724326163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880189a0 con 0x7fe99019aab0 2024-09-13T17:32:13.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.958+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901a3210 con 0x7fe99019aab0 2024-09-13T17:32:13.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.961+0000 7fe99b3d7740 1 Processor -- start 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.961+0000 7fe9877fe640 1 -- 172.21.15.44:0/1724326163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe98800fbb0 con 0x7fe99019aab0 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 -- start start 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8fecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ff7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 0x555ac8ff9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8f56cc0 con 0x555ac8ff7950 2024-09-13T17:32:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8f54450 con 0x555ac8fec790 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8f4fd40 con 0x555ac8fed0b0 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 0x555ac8ff9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 0x555ac8ff9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55436/0 (socket says 172.21.15.44:55436) 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 -- 172.21.15.44:0/74091660 learned_addr learned my addr 172.21.15.44:0/74091660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8fecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe99644d640 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ff7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe9877fe640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe97c0a1200 0x7fe97c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 -- 172.21.15.44:0/74091660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 msgr2=0x555ac8ff7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ff7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 -- 172.21.15.44:0/74091660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 msgr2=0x555ac8fecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8fecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe995c4c640 1 -- 172.21.15.44:0/74091660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ac8f17620 con 0x555ac8ff7950 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.962+0000 7fe996c4e640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe97c0a1200 0x7fe97c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe9877fe640 1 -- 172.21.15.44:0/1724326163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe9880cd550 con 0x7fe99019aab0 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe995c4c640 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 0x555ac8ff9d40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe980033000 tx=0x7fe980000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7c113904ae787a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe984ff9640 1 -- 172.21.15.44:0/74091660 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980004520 con 0x555ac8ff7950 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe984ff9640 1 -- 172.21.15.44:0/74091660 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe980044070 con 0x555ac8ff7950 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe984ff9640 1 -- 172.21.15.44:0/74091660 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980003c00 con 0x555ac8ff7950 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe99b3d7740 1 -- 172.21.15.44:0/74091660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 msgr2=0x555ac8ff9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe996c4e640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe97c0a1200 0x7fe97c0a36c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe988099fe0 tx=0x7fe988085040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.963+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 0x555ac8ff9d40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe980033000 tx=0x7fe980000bc0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- 172.21.15.44:0/74091660 shutdown_connections 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ff7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8ff7950 0x555ac8ff9d40 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/74091660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8fecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- 172.21.15.44:0/74091660 >> 172.21.15.44:0/74091660 conn(0x555ac8fe8d00 msgr2=0x555ac8fe90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- 172.21.15.44:0/74091660 shutdown_connections 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- 172.21.15.44:0/74091660 wait complete. 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- start start 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 Processor -- start 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ff6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 0x555ac8ff4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8ff7950 0x555ac8ff48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8f0e180 con 0x555ac8fed0b0 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8c28430 con 0x555ac8fec790 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8ed35e0 con 0x555ac8ff7950 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99b3d7740 10 client.0 _ll_register_callbacks cb 0x555ac8f3faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99644d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 0x555ac8ff4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99644d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 0x555ac8ff4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55446/0 (socket says 172.21.15.44:55446) 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.964+0000 7fe99644d640 1 -- 172.21.15.44:0/1754232796 learned_addr learned my addr 172.21.15.44:0/1754232796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ff6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8ff7950 0x555ac8ff48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99644d640 1 -- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8ff7950 msgr2=0x555ac8ff48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99644d640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8ff7950 0x555ac8ff48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99644d640 1 -- 172.21.15.44:0/1754232796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 msgr2=0x555ac8ff6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99644d640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ff6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99644d640 1 -- 172.21.15.44:0/1754232796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99010d470 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99644d640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 0x555ac8ff4390 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00bf60 tx=0x7fe98c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46840ed875c66303 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe979ffb640 1 -- 172.21.15.44:0/1754232796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c004230 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe979ffb640 1 -- 172.21.15.44:0/1754232796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe98c004390 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ac8f79b80 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe979ffb640 1 -- 172.21.15.44:0/1754232796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe98c0106a0 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99b3d7740 10 client.17349 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ac8f4e3c0 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.965+0000 7fe979ffb640 1 -- 172.21.15.44:0/1754232796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555ac8f79b80 con 0x555ac8fed0b0 2024-09-13T17:32:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe979ffb640 1 -- 172.21.15.44:0/1754232796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe98c003a10 con 0x555ac8fed0b0 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe99b3d7740 10 client.17349 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe99b3d7740 10 client.17349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe99b3d7740 20 client.17349 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac90072e0 0x555ac9009700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe99b3d7740 4 client.17349 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ac8f421f0 con 0x555ac90072e0 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac90072e0 0x555ac9009700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe979ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.966+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac90072e0 0x555ac9009700 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8332de04f479c78 server_cookie=87c27120be1aabb1 in_seq=0 out_seq=0 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe979ffb640 1 -- 172.21.15.44:0/1754232796 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555ac8f421f0 con 0x555ac90072e0 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe979ffb640 10 client.17349 handle_command_reply: tid=0 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 client.17349 shutdown 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 msgr2=0x555ac8ff4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 0x555ac8ff4390 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00bf60 tx=0x7fe98c00bf90 comp rx=0 tx=0).stop 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 shutdown_connections 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac90072e0 0x555ac9009700 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8ff7950 0x555ac8ff48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe979ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fed0b0 0x555ac8ff4390 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1754232796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ff6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 >> 172.21.15.44:0/1754232796 conn(0x555ac8fe8d00 msgr2=0x555ac8feea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 shutdown_connections 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1754232796 wait complete. 2024-09-13T17:32:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.967+0000 7fe99b3d7740 20 client.17349 trim_cache size 0 max 16384 2024-09-13T17:32:13.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.978+0000 7fe99b3d7740 1 Processor -- start 2024-09-13T17:32:13.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.978+0000 7fe99b3d7740 1 -- start start 2024-09-13T17:32:13.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fe8d00 0x555ac8fe86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 0x555ac9003100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac9003640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8fec510 con 0x555ac8fec790 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8ff3330 con 0x555ac8fe8d00 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac8ff6b00 con 0x555ac8fed0b0 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 0x555ac9003100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 0x555ac9003100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55454/0 (socket says 172.21.15.44:55454) 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 -- 172.21.15.44:0/1535483999 learned_addr learned my addr 172.21.15.44:0/1535483999 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 -- 172.21.15.44:0/1535483999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 msgr2=0x555ac9003640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fe8d00 0x555ac8fe86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac9003640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 -- 172.21.15.44:0/1535483999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fe8d00 msgr2=0x555ac8fe86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fe8d00 0x555ac8fe86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 -- 172.21.15.44:0/1535483999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ac8ef0770 con 0x555ac8fec790 2024-09-13T17:32:13.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe995c4c640 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fe8d00 0x555ac8fe86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe99644d640 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 0x555ac9003100 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00b5d0 tx=0x7fe98c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdfd30257b11ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe9857fa640 1 -- 172.21.15.44:0/1535483999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0041b0 con 0x555ac8fec790 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe9857fa640 1 -- 172.21.15.44:0/1535483999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe98c003a80 con 0x555ac8fec790 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.979+0000 7fe9857fa640 1 -- 172.21.15.44:0/1535483999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c004680 con 0x555ac8fec790 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1535483999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 msgr2=0x555ac9003100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 0x555ac9003100 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00b5d0 tx=0x7fe98c005730 comp rx=0 tx=0).stop 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1535483999 shutdown_connections 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac9003640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fec790 0x555ac9003100 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/1535483999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fe8d00 0x555ac8fe86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1535483999 >> 172.21.15.44:0/1535483999 conn(0x555ac8ff7040 msgr2=0x555ac8ff2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1535483999 shutdown_connections 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- 172.21.15.44:0/1535483999 wait complete. 2024-09-13T17:32:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- start start 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 Processor -- start 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 0x555ac8ffb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ffb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ffbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac905d8b0 con 0x555ac8fe8d00 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac905da20 con 0x555ac8fec790 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.980+0000 7fe99b3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac905dbe0 con 0x555ac8fed0b0 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99b3d7740 10 client.0 _ll_register_callbacks cb 0x7fe980009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe995c4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 0x555ac8ffb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe995c4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 0x555ac8ffb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55458/0 (socket says 172.21.15.44:55458) 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe995c4c640 1 -- 172.21.15.44:0/501780721 learned_addr learned my addr 172.21.15.44:0/501780721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ffb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe995c4c640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ffbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 -- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 msgr2=0x555ac8ffbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ffbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 -- 172.21.15.44:0/501780721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 msgr2=0x555ac8ffb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 0x555ac8ffb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 -- 172.21.15.44:0/501780721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ac8f4e3c0 con 0x555ac8fec790 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe995c4c640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 0x555ac8ffb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99644d640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ffb780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x555ac8ff6f20 tx=0x7fe98c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e8216366335354 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe979ffb640 1 -- 172.21.15.44:0/501780721 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c005250 con 0x555ac8fec790 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe980044070 con 0x555ac8fec790 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe979ffb640 1 -- 172.21.15.44:0/501780721 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fe98c000e40 con 0x555ac8fec790 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe979ffb640 1 -- 172.21.15.44:0/501780721 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe98c00bec0 con 0x555ac8fec790 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99b3d7740 10 client.26681 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.981+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ac8f15e80 con 0x555ac8fec790 2024-09-13T17:32:13.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe979ffb640 1 -- 172.21.15.44:0/501780721 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe980044070 con 0x555ac8fec790 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe979ffb640 1 -- 172.21.15.44:0/501780721 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe98c0106a0 con 0x555ac8fec790 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe99b3d7740 10 client.26681 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe99b3d7740 10 client.26681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe99b3d7740 20 client.26681 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac8ff7950 0x555ac9060df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe99b3d7740 4 client.26681 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ac8c434e0 con 0x555ac8ff7950 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.982+0000 7fe995c4c640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac8ff7950 0x555ac9060df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe979ffb640 10 client.26681 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe995c4c640 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac8ff7950 0x555ac9060df0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e53c36071310f04c server_cookie=3f26bbaf133dd34f in_seq=0 out_seq=0 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe979ffb640 1 -- 172.21.15.44:0/501780721 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ac8c434e0 con 0x555ac8ff7950 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe979ffb640 10 client.26681 handle_command_reply: tid=0 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 client.26681 shutdown 2024-09-13T17:32:13.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 msgr2=0x555ac8ffb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ffb780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x555ac8ff6f20 tx=0x7fe98c003c80 comp rx=0 tx=0).stop 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 shutdown_connections 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x555ac8ff7950 0x555ac9060df0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ac8fed0b0 0x555ac8ffbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ac8fec790 0x555ac8ffb780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 --2- 172.21.15.44:0/501780721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ac8fe8d00 0x555ac8ffb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe979ffb640 0 client.26681 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 >> 172.21.15.44:0/501780721 conn(0x555ac8ff7040 msgr2=0x555ac8ff4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 shutdown_connections 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 1 -- 172.21.15.44:0/501780721 wait complete. 2024-09-13T17:32:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe99b3d7740 20 client.26681 trim_cache size 0 max 16384 2024-09-13T17:32:13.992 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:32:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe97c0a1200 msgr2=0x7fe97c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe9986b1640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe97c0a1200 0x7fe97c0a36c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe988099fe0 tx=0x7fe988085040 comp rx=0 tx=0).stop 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 msgr2=0x7fe99019af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.983+0000 7fe9986b1640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 0x7fe99019af60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe98800cce0 tx=0x7fe98800a620 comp rx=0 tx=0).stop 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 shutdown_connections 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe97c0a1200 0x7fe97c0a36c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9901050e0 0x7fe9901966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99019aab0 0x7fe99019af60 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 --2- 172.21.15.44:0/1724326163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9900fff90 0x7fe990196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 >> 172.21.15.44:0/1724326163 conn(0x7fe9900fba20 msgr2=0x7fe9900fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 shutdown_connections 2024-09-13T17:32:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:13.984+0000 7fe9986b1640 1 -- 172.21.15.44:0/1724326163 wait complete. 2024-09-13T17:32:13.994 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:13 smithi044 ceph-mon[31175]: pgmap v168: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-13T17:32:13.994 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1515744175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:32:13.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:32:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 -- 172.21.15.44:0/1470323307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a00fef20 msgr2=0x7ff7a00ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/1470323307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a00fef20 0x7ff7a00ff300 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff794009930 tx=0x7ff79402a420 comp rx=0 tx=0).stop 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 -- 172.21.15.44:0/1470323307 shutdown_connections 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/1470323307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a00ff8d0 0x7ff7a0103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/1470323307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0104070 0x7ff7a010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/1470323307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a00fef20 0x7ff7a00ff300 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 -- 172.21.15.44:0/1470323307 >> 172.21.15.44:0/1470323307 conn(0x7ff7a00fa970 msgr2=0x7ff7a00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 -- 172.21.15.44:0/1470323307 shutdown_connections 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.098+0000 7ff7a70c9640 1 -- 172.21.15.44:0/1470323307 wait complete. 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 Processor -- start 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 -- start start 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a00fef20 0x7ff7a01965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a0104070 0x7ff7a0196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 0x7ff7a019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019b8e0 con 0x7ff7a019aef0 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019ba50 con 0x7ff7a00fef20 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a70c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019bbc0 con 0x7ff7a0104070 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a5666640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 0x7ff7a019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a5666640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 0x7ff7a019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55474/0 (socket says 172.21.15.44:55474) 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a5666640 1 -- 172.21.15.44:0/3924235384 learned_addr learned my addr 172.21.15.44:0/3924235384 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a00fef20 0x7ff7a01965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.099+0000 7ff79ffff640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a0104070 0x7ff7a0196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a5666640 1 -- 172.21.15.44:0/3924235384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a0104070 msgr2=0x7ff7a0196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a5666640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a0104070 0x7ff7a0196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a5666640 1 -- 172.21.15.44:0/3924235384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a00fef20 msgr2=0x7ff7a01965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a5666640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a00fef20 0x7ff7a01965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a5666640 1 -- 172.21.15.44:0/3924235384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794009560 con 0x7ff7a019aef0 2024-09-13T17:32:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a5666640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 0x7ff7a019b3a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff79800eaa0 tx=0x7ff79800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:14.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff79dffb640 1 -- 172.21.15.44:0/3924235384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800cdb0 con 0x7ff7a019aef0 2024-09-13T17:32:14.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff79dffb640 1 -- 172.21.15.44:0/3924235384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff798005020 con 0x7ff7a019aef0 2024-09-13T17:32:14.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a01a31a0 con 0x7ff7a019aef0 2024-09-13T17:32:14.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff79dffb640 1 -- 172.21.15.44:0/3924235384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7980116d0 con 0x7ff7a019aef0 2024-09-13T17:32:14.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.100+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a01a3600 con 0x7ff7a019aef0 2024-09-13T17:32:14.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.102+0000 7ff79dffb640 1 -- 172.21.15.44:0/3924235384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff798005190 con 0x7ff7a019aef0 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.102+0000 7ff7a9de6740 1 Processor -- start 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 -- start start 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a988bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 0x55c8a9896410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a9898d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a97f5cc0 con 0x55c8a988c0b0 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a97f3450 con 0x55c8a988b790 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a97eed40 con 0x55c8a9896950 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff79dffb640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7900a1200 0x7ff7900a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 0x55c8a9896410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 0x55c8a9896410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55478/0 (socket says 172.21.15.44:55478) 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 -- 172.21.15.44:0/2763367626 learned_addr learned my addr 172.21.15.44:0/2763367626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a5666640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7900a1200 0x7ff7900a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff79ffff640 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a988bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff79ffff640 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a9898d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff79dffb640 1 -- 172.21.15.44:0/3924235384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff7980cd1c0 con 0x7ff7a019aef0 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 -- 172.21.15.44:0/2763367626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 msgr2=0x55c8a9898d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a9898d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 -- 172.21.15.44:0/2763367626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 msgr2=0x55c8a988bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a988bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 -- 172.21.15.44:0/2763367626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a97b6620 con 0x55c8a988c0b0 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.103+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 0x55c8a9896410 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff794009900 tx=0x7ff794005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f130a9389eac9d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a5666640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7900a1200 0x7ff7900a36c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7980ca6a0 tx=0x7ff798005680 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff78affd640 1 -- 172.21.15.44:0/2763367626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794004230 con 0x55c8a988c0b0 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff78affd640 1 -- 172.21.15.44:0/2763367626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff794031df0 con 0x55c8a988c0b0 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff78affd640 1 -- 172.21.15.44:0/2763367626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794003c00 con 0x55c8a988c0b0 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2763367626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 msgr2=0x55c8a9896410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 0x55c8a9896410 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff794009900 tx=0x7ff794005c00 comp rx=0 tx=0).stop 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2763367626 shutdown_connections 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a9898d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988c0b0 0x55c8a9896410 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2763367626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a988bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2763367626 >> 172.21.15.44:0/2763367626 conn(0x55c8a9887d00 msgr2=0x55c8a98880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:14.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2763367626 shutdown_connections 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.104+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2763367626 wait complete. 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 -- start start 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 Processor -- start 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a9895ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988c0b0 0x55c8a9896400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a98934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a97ad180 con 0x55c8a9896950 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a94c7430 con 0x55c8a988b790 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a97725e0 con 0x55c8a988c0b0 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a9de6740 10 client.0 _ll_register_callbacks cb 0x55c8a97deaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a98934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a9895ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a9895ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:51492/0 (socket says 172.21.15.44:51492) 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff79ffff640 1 -- 172.21.15.44:0/3716375229 learned_addr learned my addr 172.21.15.44:0/3716375229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.105+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988c0b0 0x55c8a9896400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff79ffff640 1 -- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988c0b0 msgr2=0x55c8a9896400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff79ffff640 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988c0b0 0x55c8a9896400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff79ffff640 1 -- 172.21.15.44:0/3716375229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 msgr2=0x55c8a98934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff79ffff640 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a98934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff79ffff640 1 -- 172.21.15.44:0/3716375229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a010c400 con 0x55c8a988b790 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff79ffff640 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a9895ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff78c012ea0 tx=0x7ff78c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4200f42c913aa4aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff77ffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff77ffff640 1 -- 172.21.15.44:0/3716375229 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c016070 con 0x55c8a988b790 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff77ffff640 1 -- 172.21.15.44:0/3716375229 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff78c011e80 con 0x55c8a988b790 2024-09-13T17:32:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8a9818b80 con 0x55c8a988b790 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff77ffff640 1 -- 172.21.15.44:0/3716375229 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff78c005020 con 0x55c8a988b790 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff7a9de6740 10 client.26693 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c8a97ed3c0 con 0x55c8a988b790 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.106+0000 7ff77ffff640 1 -- 172.21.15.44:0/3716375229 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c8a9818b80 con 0x55c8a988b790 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff77ffff640 1 -- 172.21.15.44:0/3716375229 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff78c004d00 con 0x55c8a988b790 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a9de6740 10 client.26693 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a9de6740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a9de6740 20 client.26693 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a9de6740 4 client.26693 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8a97e11f0 con 0x55c8a98a6280 2024-09-13T17:32:14.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff77ffff640 10 client.26693 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.107+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98a86a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926693f8cf6192b1 server_cookie=6122250c8cd5c6fe in_seq=0 out_seq=0 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff77ffff640 1 -- 172.21.15.44:0/3716375229 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c8a97e11f0 con 0x55c8a98a6280 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff77ffff640 10 client.26693 handle_command_reply: tid=0 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 client.26693 shutdown 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 msgr2=0x55c8a9895ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a9895ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff78c012ea0 tx=0x7ff78c004730 comp rx=0 tx=0).stop 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 shutdown_connections 2024-09-13T17:32:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98a86a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988c0b0 0x55c8a9896400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff77ffff640 0 client.26693 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a98934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/3716375229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988b790 0x55c8a9895ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 >> 172.21.15.44:0/3716375229 conn(0x55c8a9887d00 msgr2=0x55c8a988da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 shutdown_connections 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 1 -- 172.21.15.44:0/3716375229 wait complete. 2024-09-13T17:32:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.108+0000 7ff7a9de6740 20 client.26693 trim_cache size 0 max 16384 2024-09-13T17:32:14.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.119+0000 7ff7a9de6740 1 Processor -- start 2024-09-13T17:32:14.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.119+0000 7ff7a9de6740 1 -- start start 2024-09-13T17:32:14.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988b790 0x55c8a98877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9891b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a9892090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9818b80 con 0x55c8a9896950 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9899400 con 0x55c8a988c0b0 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a98a2ff0 con 0x55c8a988b790 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a9892090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a9892090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55486/0 (socket says 172.21.15.44:55486) 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 -- 172.21.15.44:0/2660563728 learned_addr learned my addr 172.21.15.44:0/2660563728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988b790 0x55c8a98877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 -- 172.21.15.44:0/2660563728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988b790 msgr2=0x55c8a98877a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988b790 0x55c8a98877a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 -- 172.21.15.44:0/2660563728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 msgr2=0x55c8a9891b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9891b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9891b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 -- 172.21.15.44:0/2660563728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a978f770 con 0x55c8a9896950 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9891b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff79ffff640 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a9892090 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005fd0 tx=0x7ff78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5b9be79d469f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff78b7fe640 1 -- 172.21.15.44:0/2660563728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c016070 con 0x55c8a9896950 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff78b7fe640 1 -- 172.21.15.44:0/2660563728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff78c033020 con 0x55c8a9896950 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.120+0000 7ff78b7fe640 1 -- 172.21.15.44:0/2660563728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c022d40 con 0x55c8a9896950 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2660563728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 msgr2=0x55c8a9892090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a9892090 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005fd0 tx=0x7ff78c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2660563728 shutdown_connections 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a9896950 0x55c8a9892090 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9891b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/2660563728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a988b790 0x55c8a98877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2660563728 >> 172.21.15.44:0/2660563728 conn(0x55c8a9887d00 msgr2=0x55c8a98ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2660563728 shutdown_connections 2024-09-13T17:32:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 -- 172.21.15.44:0/2660563728 wait complete. 2024-09-13T17:32:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 -- start start 2024-09-13T17:32:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.121+0000 7ff7a9de6740 1 Processor -- start 2024-09-13T17:32:14.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 0x55c8a989d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9899c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a989a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a989a680 con 0x55c8a988b790 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a989a7f0 con 0x55c8a988c0b0 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a989a9b0 con 0x55c8a9896950 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a9de6740 10 client.0 _ll_register_callbacks cb 0x7ff794005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 0x55c8a989d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 0x55c8a989d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55496/0 (socket says 172.21.15.44:55496) 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 -- 172.21.15.44:0/1297510056 learned_addr learned my addr 172.21.15.44:0/1297510056 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9899c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a989a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 -- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 msgr2=0x55c8a989a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a989a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 -- 172.21.15.44:0/1297510056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 msgr2=0x55c8a9899c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9899c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 -- 172.21.15.44:0/1297510056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a97ed3c0 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff79ffff640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 0x55c8a989d7a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff78c012780 tx=0x7ff78c012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b64abfb4f1d8214 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.122+0000 7ff77ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff77ffff640 1 -- 172.21.15.44:0/1297510056 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c016070 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff77ffff640 1 -- 172.21.15.44:0/1297510056 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7ff78c034030 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff794031df0 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff77ffff640 1 -- 172.21.15.44:0/1297510056 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff78c021380 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 10 client.17373 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c8a97b4e80 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff77ffff640 1 -- 172.21.15.44:0/1297510056 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff794031df0 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff77ffff640 1 -- 172.21.15.44:0/1297510056 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff78c021a20 con 0x55c8a988b790 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 10 client.17373 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 20 client.17373 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98ff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 4 client.17373 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8a94e24e0 con 0x55c8a98a6280 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.123+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff77ffff640 10 client.17373 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a4e65640 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98ff720 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b021e9a844618611 server_cookie=adf57b78e7275b9b in_seq=0 out_seq=0 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff77ffff640 1 -- 172.21.15.44:0/1297510056 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8a94e24e0 con 0x55c8a98a6280 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff77ffff640 10 client.17373 handle_command_reply: tid=0 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 client.17373 shutdown 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 msgr2=0x55c8a989d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 0x55c8a989d7a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff78c012780 tx=0x7ff78c012540 comp rx=0 tx=0).stop 2024-09-13T17:32:14.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 shutdown_connections 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c8a98a6280 0x55c8a98ff720 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8a9896950 0x55c8a989a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff77ffff640 0 client.17373 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8a988c0b0 0x55c8a9899c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 --2- 172.21.15.44:0/1297510056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8a988b790 0x55c8a989d7a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 >> 172.21.15.44:0/1297510056 conn(0x55c8a9887d00 msgr2=0x55c8a9891a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 shutdown_connections 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 1 -- 172.21.15.44:0/1297510056 wait complete. 2024-09-13T17:32:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.124+0000 7ff7a9de6740 20 client.17373 trim_cache size 0 max 16384 2024-09-13T17:32:14.133 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:32:14.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7900a1200 msgr2=0x7ff7900a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7900a1200 0x7ff7900a36c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7980ca6a0 tx=0x7ff798005680 comp rx=0 tx=0).stop 2024-09-13T17:32:14.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 msgr2=0x7ff7a019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 0x7ff7a019b3a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff79800eaa0 tx=0x7ff79800ef60 comp rx=0 tx=0).stop 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 shutdown_connections 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7900a1200 0x7ff7900a36c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a0104070 0x7ff7a0196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7a019aef0 0x7ff7a019b3a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 --2- 172.21.15.44:0/3924235384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a00fef20 0x7ff7a01965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 >> 172.21.15.44:0/3924235384 conn(0x7ff7a00fa970 msgr2=0x7ff7a0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 shutdown_connections 2024-09-13T17:32:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:14.125+0000 7ff7a70c9640 1 -- 172.21.15.44:0/3924235384 wait complete. 2024-09-13T17:32:14.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:13 smithi070 ceph-mon[34502]: pgmap v168: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-13T17:32:14.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1515744175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:32:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:13 smithi100 ceph-mon[35055]: pgmap v168: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-13T17:32:14.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1515744175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:32:15.137 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:32:15.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:32:15.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 -- 172.21.15.44:0/1971062378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80ff810 msgr2=0x7f31d8103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 --2- 172.21.15.44:0/1971062378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80ff810 0x7f31d8103850 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009930 tx=0x7f31c802a420 comp rx=0 tx=0).stop 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 -- 172.21.15.44:0/1971062378 shutdown_connections 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 --2- 172.21.15.44:0/1971062378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 0x7f31d810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 --2- 172.21.15.44:0/1971062378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80ff810 0x7f31d8103850 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 --2- 172.21.15.44:0/1971062378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d80fee60 0x7f31d80ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 -- 172.21.15.44:0/1971062378 >> 172.21.15.44:0/1971062378 conn(0x7f31d8075c60 msgr2=0x7f31d80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 -- 172.21.15.44:0/1971062378 shutdown_connections 2024-09-13T17:32:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.256+0000 7f31df5db640 1 -- 172.21.15.44:0/1971062378 wait complete. 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 Processor -- start 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 -- start start 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 0x7f31d81961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 0x7f31d8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d819ab10 0x7f31d819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d819b500 con 0x7f31d80fee60 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d819b670 con 0x7f31d819ab10 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31df5db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d819b7e0 con 0x7f31d8103fb0 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 0x7f31d81961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 0x7f31d81961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55508/0 (socket says 172.21.15.44:55508) 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 -- 172.21.15.44:0/4141169690 learned_addr learned my addr 172.21.15.44:0/4141169690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31ddb78640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d819ab10 0x7f31d819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dcb76640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 0x7f31d8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 -- 172.21.15.44:0/4141169690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 msgr2=0x7f31d8196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 0x7f31d8196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 -- 172.21.15.44:0/4141169690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d819ab10 msgr2=0x7f31d819afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d819ab10 0x7f31d819afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dd377640 1 -- 172.21.15.44:0/4141169690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c8009560 con 0x7f31d80fee60 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.257+0000 7f31dcb76640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 0x7f31d8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.258+0000 7f31dd377640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 0x7f31d81961e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f31d400d950 tx=0x7f31d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.258+0000 7f31ce7fc640 1 -- 172.21.15.44:0/4141169690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d4009c30 con 0x7f31d80fee60 2024-09-13T17:32:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.258+0000 7f31ce7fc640 1 -- 172.21.15.44:0/4141169690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31d4009d90 con 0x7f31d80fee60 2024-09-13T17:32:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.258+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d81a2dc0 con 0x7f31d80fee60 2024-09-13T17:32:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.258+0000 7f31ce7fc640 1 -- 172.21.15.44:0/4141169690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d400f620 con 0x7f31d80fee60 2024-09-13T17:32:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.258+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d81a3230 con 0x7f31d80fee60 2024-09-13T17:32:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.260+0000 7f31ce7fc640 1 -- 172.21.15.44:0/4141169690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f31d40109f0 con 0x7f31d80fee60 2024-09-13T17:32:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.260+0000 7f31e23d8740 1 Processor -- start 2024-09-13T17:32:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.260+0000 7f31e23d8740 1 -- start start 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c408fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c409a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 0x5619c409cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31e23d8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3ffac60 con 0x5619c409a980 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31e23d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3ff83f0 con 0x5619c408f7c0 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31e23d8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3ff3ce0 con 0x5619c40900e0 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31ce7fc640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31c40a1280 0x7f31c40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 0x5619c409cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dd377640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c409a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 0x5619c409cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55520/0 (socket says 172.21.15.44:55520) 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 -- 172.21.15.44:0/2858154236 learned_addr learned my addr 172.21.15.44:0/2858154236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31ddb78640 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c408fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31ce7fc640 1 -- 172.21.15.44:0/4141169690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f31d40ccc10 con 0x7f31d80fee60 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31ddb78640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31c40a1280 0x7f31c40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 -- 172.21.15.44:0/2858154236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 msgr2=0x5619c409a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c409a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 -- 172.21.15.44:0/2858154236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 msgr2=0x5619c408fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c408fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31dcb76640 1 -- 172.21.15.44:0/2858154236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c3faf970 con 0x5619c409a980 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.261+0000 7f31ddb78640 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c408fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:15.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.262+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 0x5619c409cd70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f31d8072f50 tx=0x7f31c8004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ba0c9e483dcbbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.262+0000 7f31ddb78640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31c40a1280 0x7f31c40a3740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009fd0 tx=0x7f31d0009550 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.262+0000 7f31c37fe640 1 -- 172.21.15.44:0/2858154236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80035f0 con 0x5619c409a980 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.262+0000 7f31c37fe640 1 -- 172.21.15.44:0/2858154236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c8003be0 con 0x5619c409a980 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.262+0000 7f31c37fe640 1 -- 172.21.15.44:0/2858154236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c803caf0 con 0x5619c409a980 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 -- 172.21.15.44:0/2858154236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 msgr2=0x5619c409cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 0x5619c409cd70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f31d8072f50 tx=0x7f31c8004170 comp rx=0 tx=0).stop 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 -- 172.21.15.44:0/2858154236 shutdown_connections 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c409a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c409a980 0x5619c409cd70 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2858154236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c408fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 -- 172.21.15.44:0/2858154236 >> 172.21.15.44:0/2858154236 conn(0x5619c408bd50 msgr2=0x5619c408c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 -- 172.21.15.44:0/2858154236 shutdown_connections 2024-09-13T17:32:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 -- 172.21.15.44:0/2858154236 wait complete. 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 -- start start 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.263+0000 7f31e23d8740 1 Processor -- start 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c409a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 0x5619c4097580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c409a980 0x5619c4097b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3fb1f80 con 0x5619c40900e0 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3fb1c20 con 0x5619c408f7c0 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3f7eab0 con 0x5619c409a980 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 10 client.0 _ll_register_callbacks cb 0x5619c3fed350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 0x5619c4097580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 0x5619c4097580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55532/0 (socket says 172.21.15.44:55532) 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 -- 172.21.15.44:0/2381694820 learned_addr learned my addr 172.21.15.44:0/2381694820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31ddb78640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c409a980 0x5619c4097b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dd377640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c409a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 -- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c409a980 msgr2=0x5619c4097b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c409a980 0x5619c4097b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 -- 172.21.15.44:0/2381694820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 msgr2=0x5619c409a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c409a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 -- 172.21.15.44:0/2381694820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d810c340 con 0x5619c40900e0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dd377640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c409a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 0x5619c4097580 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009900 tx=0x7f31c8004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b624575a3f3757af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31c0ff9640 1 -- 172.21.15.44:0/2381694820 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8049040 con 0x5619c40900e0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31c0ff9640 1 -- 172.21.15.44:0/2381694820 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f31c80037f0 con 0x5619c40900e0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.264+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619c3ce7590 con 0x5619c40900e0 2024-09-13T17:32:15.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31c0ff9640 1 -- 172.21.15.44:0/2381694820 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c803c650 con 0x5619c40900e0 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 10 client.17385 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619c3ff2360 con 0x5619c40900e0 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31c0ff9640 1 -- 172.21.15.44:0/2381694820 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5619c3ce7590 con 0x5619c40900e0 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31c0ff9640 1 -- 172.21.15.44:0/2381694820 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31c80457c0 con 0x5619c40900e0 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 10 client.17385 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 20 client.17385 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c40aa360 0x5619c40ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 4 client.17385 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619c3cf8860 con 0x5619c40aa360 2024-09-13T17:32:15.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.265+0000 7f31dd377640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c40aa360 0x5619c40ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.266+0000 7f31dd377640 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c40aa360 0x5619c40ac750 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4be88d4df1ba539 server_cookie=dda6572c30b3a75 in_seq=0 out_seq=0 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.266+0000 7f31c0ff9640 10 client.17385 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.266+0000 7f31c0ff9640 1 -- 172.21.15.44:0/2381694820 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5619c3cf8860 con 0x5619c40aa360 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.266+0000 7f31c0ff9640 10 client.17385 handle_command_reply: tid=0 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.266+0000 7f31e23d8740 1 client.17385 shutdown 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 msgr2=0x5619c4097580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 0x5619c4097580 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009900 tx=0x7f31c8004590 comp rx=0 tx=0).stop 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 shutdown_connections 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c40aa360 0x5619c40ac750 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c409a980 0x5619c4097b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c40900e0 0x5619c4097580 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31c0ff9640 0 client.17385 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2381694820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408f7c0 0x5619c409a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 >> 172.21.15.44:0/2381694820 conn(0x5619c408bd50 msgr2=0x5619c40911a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 shutdown_connections 2024-09-13T17:32:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 1 -- 172.21.15.44:0/2381694820 wait complete. 2024-09-13T17:32:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.267+0000 7f31e23d8740 20 client.17385 trim_cache size 0 max 16384 2024-09-13T17:32:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.281+0000 7f31e23d8740 1 Processor -- start 2024-09-13T17:32:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.281+0000 7f31e23d8740 1 -- start start 2024-09-13T17:32:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c408bd50 0x5619c40a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c40900e0 0x5619c4095e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31e23d8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c3ce7590 con 0x5619c408f7c0 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31e23d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c4087910 con 0x5619c40900e0 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31e23d8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c40a73e0 con 0x5619c408bd50 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55546/0 (socket says 172.21.15.44:55546) 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 -- 172.21.15.44:0/2701249159 learned_addr learned my addr 172.21.15.44:0/2701249159 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31ddb78640 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c40900e0 0x5619c4095e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dd377640 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c408bd50 0x5619c40a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 -- 172.21.15.44:0/2701249159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c408bd50 msgr2=0x5619c40a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c408bd50 0x5619c40a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 -- 172.21.15.44:0/2701249159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c40900e0 msgr2=0x5619c4095e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c40900e0 0x5619c4095e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 -- 172.21.15.44:0/2701249159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c3f941c0 con 0x5619c408f7c0 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dd377640 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c408bd50 0x5619c40a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31dcb76640 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a1b90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f31c8045b50 tx=0x7f31c8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214fd8327cb6fb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31c3fff640 1 -- 172.21.15.44:0/2701249159 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8049040 con 0x5619c408f7c0 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31c3fff640 1 -- 172.21.15.44:0/2701249159 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c8003f40 con 0x5619c408f7c0 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.282+0000 7f31c3fff640 1 -- 172.21.15.44:0/2701249159 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c803c8b0 con 0x5619c408f7c0 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 -- 172.21.15.44:0/2701249159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 msgr2=0x5619c40a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a1b90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f31c8045b50 tx=0x7f31c8005740 comp rx=0 tx=0).stop 2024-09-13T17:32:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 -- 172.21.15.44:0/2701249159 shutdown_connections 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c40900e0 0x5619c4095e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a1b90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 --2- 172.21.15.44:0/2701249159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c408bd50 0x5619c40a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 -- 172.21.15.44:0/2701249159 >> 172.21.15.44:0/2701249159 conn(0x5619c409a460 msgr2=0x5619c4092840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 -- 172.21.15.44:0/2701249159 shutdown_connections 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 -- 172.21.15.44:0/2701249159 wait complete. 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 -- start start 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.283+0000 7f31e23d8740 1 Processor -- start 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408bd50 0x5619c40a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c4100860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c4100da0 con 0x5619c408f7c0 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c4100ee0 con 0x5619c408bd50 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c4101050 con 0x5619c40900e0 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31e23d8740 10 client.0 _ll_register_callbacks cb 0x7f31d0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55558/0 (socket says 172.21.15.44:55558) 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 -- 172.21.15.44:0/304804887 learned_addr learned my addr 172.21.15.44:0/304804887 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31ddb78640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c4100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dd377640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408bd50 0x5619c40a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 -- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 msgr2=0x5619c4100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c4100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 -- 172.21.15.44:0/304804887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408bd50 msgr2=0x5619c40a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408bd50 0x5619c40a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 -- 172.21.15.44:0/304804887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c3ff2360 con 0x5619c408f7c0 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dd377640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408bd50 0x5619c40a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31dcb76640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a0cc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f31c8045ba0 tx=0x7f31c8045710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befd032fadc74cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.284+0000 7f31c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31c0ff9640 1 -- 172.21.15.44:0/304804887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8049070 con 0x5619c408f7c0 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31c0ff9640 1 -- 172.21.15.44:0/304804887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f31c8000e40 con 0x5619c408f7c0 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31c8003be0 con 0x5619c408f7c0 2024-09-13T17:32:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31c0ff9640 1 -- 172.21.15.44:0/304804887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c8003a00 con 0x5619c408f7c0 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 10 client.17391 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619c3fb9990 con 0x5619c408f7c0 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31c0ff9640 1 -- 172.21.15.44:0/304804887 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c8003be0 con 0x5619c408f7c0 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31c0ff9640 1 -- 172.21.15.44:0/304804887 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31c8064070 con 0x5619c408f7c0 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 10 client.17391 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 20 client.17391 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c409a980 0x5619c4104400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 4 client.17391 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:32:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5619c3d4beb0 con 0x5619c409a980 2024-09-13T17:32:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.285+0000 7f31dd377640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c409a980 0x5619c4104400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.286+0000 7f31dd377640 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c409a980 0x5619c4104400 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d081ca44b023d8c server_cookie=29f5ef81faa76feb in_seq=0 out_seq=0 2024-09-13T17:32:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.286+0000 7f31c0ff9640 10 client.17391 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31c0ff9640 1 -- 172.21.15.44:0/304804887 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5619c3d4beb0 con 0x5619c409a980 2024-09-13T17:32:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31c0ff9640 10 client.17391 handle_command_reply: tid=0 2024-09-13T17:32:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 client.17391 shutdown 2024-09-13T17:32:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 msgr2=0x5619c40a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a0cc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f31c8045ba0 tx=0x7f31c8045710 comp rx=0 tx=0).stop 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 shutdown_connections 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5619c409a980 0x5619c4104400 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c40900e0 0x5619c4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619c408f7c0 0x5619c40a0cc0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31c0ff9640 0 client.17391 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 --2- 172.21.15.44:0/304804887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5619c408bd50 0x5619c40a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 >> 172.21.15.44:0/304804887 conn(0x5619c409a460 msgr2=0x5619c4095dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 shutdown_connections 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 1 -- 172.21.15.44:0/304804887 wait complete. 2024-09-13T17:32:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.287+0000 7f31e23d8740 20 client.17391 trim_cache size 0 max 16384 2024-09-13T17:32:15.297 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:32:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:32:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "e8d283e3-99ca-4183-80c5-5c3ad7e4e7aa", 2024-09-13T17:32:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:32:15.297 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:32:15.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31c40a1280 msgr2=0x7f31c40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31c40a1280 0x7f31c40a3740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009fd0 tx=0x7f31d0009550 comp rx=0 tx=0).stop 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 msgr2=0x7f31d81961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 0x7f31d81961e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f31d400d950 tx=0x7f31d400de10 comp rx=0 tx=0).stop 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 shutdown_connections 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31c40a1280 0x7f31c40a3740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d8103fb0 0x7f31d8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d819ab10 0x7f31d819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 --2- 172.21.15.44:0/4141169690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31d80fee60 0x7f31d81961e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 >> 172.21.15.44:0/4141169690 conn(0x7f31d8075c60 msgr2=0x7f31d80677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 shutdown_connections 2024-09-13T17:32:15.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.288+0000 7f31df5db640 1 -- 172.21.15.44:0/4141169690 wait complete. 2024-09-13T17:32:15.301 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:32:15.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 -- 172.21.15.44:0/2097960565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 msgr2=0x7f1a600ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 --2- 172.21.15.44:0/2097960565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a600ff260 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009930 tx=0x7f1a5402a420 comp rx=0 tx=0).stop 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 -- 172.21.15.44:0/2097960565 shutdown_connections 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 --2- 172.21.15.44:0/2097960565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a600ff830 0x7f1a60103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 --2- 172.21.15.44:0/2097960565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60103fd0 0x7f1a6010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 --2- 172.21.15.44:0/2097960565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a600ff260 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 -- 172.21.15.44:0/2097960565 >> 172.21.15.44:0/2097960565 conn(0x7f1a600fa930 msgr2=0x7f1a600fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 -- 172.21.15.44:0/2097960565 shutdown_connections 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.407+0000 7f1a67019640 1 -- 172.21.15.44:0/2097960565 wait complete. 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 Processor -- start 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 -- start start 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a60197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a60103fd0 0x7f1a60197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60197c20 0x7f1a601a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a601a28c0 con 0x7f1a600fee80 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a601a2a30 con 0x7f1a60197c20 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a67019640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a601a2bf0 con 0x7f1a60103fd0 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a60197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a60197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55566/0 (socket says 172.21.15.44:55566) 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 -- 172.21.15.44:0/195044007 learned_addr learned my addr 172.21.15.44:0/195044007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a60103fd0 0x7f1a60197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a655b6640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60197c20 0x7f1a601a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 -- 172.21.15.44:0/195044007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a60103fd0 msgr2=0x7f1a60197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a60103fd0 0x7f1a60197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 -- 172.21.15.44:0/195044007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60197c20 msgr2=0x7f1a601a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60197c20 0x7f1a601a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a64db5640 1 -- 172.21.15.44:0/195044007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54009560 con 0x7f1a600fee80 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.408+0000 7f1a655b6640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60197c20 0x7f1a601a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.409+0000 7f1a64db5640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a60197100 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1a5402a940 tx=0x7f1a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.409+0000 7f1a5dffb640 1 -- 172.21.15.44:0/195044007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54004cf0 con 0x7f1a600fee80 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.409+0000 7f1a5dffb640 1 -- 172.21.15.44:0/195044007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a54003a80 con 0x7f1a600fee80 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.409+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a601a2e70 con 0x7f1a600fee80 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.409+0000 7f1a5dffb640 1 -- 172.21.15.44:0/195044007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5403ca70 con 0x7f1a600fee80 2024-09-13T17:32:15.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.409+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a601a3290 con 0x7f1a600fee80 2024-09-13T17:32:15.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.411+0000 7f1a5dffb640 1 -- 172.21.15.44:0/195044007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1a54003bf0 con 0x7f1a600fee80 2024-09-13T17:32:15.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.411+0000 7f1a69e1a740 1 Processor -- start 2024-09-13T17:32:15.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.411+0000 7f1a69e1a740 1 -- start start 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f791790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f79179630 0x555f79187ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f791883e0 0x555f7918a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f790e3f20 con 0x555f79178d10 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f790e16b0 con 0x555f79179630 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f790dcfa0 con 0x555f791883e0 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f791790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f791790f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55580/0 (socket says 172.21.15.44:55580) 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 -- 172.21.15.44:0/1741254234 learned_addr learned my addr 172.21.15.44:0/1741254234 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a64db5640 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f79179630 0x555f79187ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f791883e0 0x555f7918a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 -- 172.21.15.44:0/1741254234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f791883e0 msgr2=0x555f7918a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f791883e0 0x555f7918a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 -- 172.21.15.44:0/1741254234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f79179630 msgr2=0x555f79187ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f79179630 0x555f79187ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 -- 172.21.15.44:0/1741254234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f79097840 con 0x555f79178d10 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f791883e0 0x555f7918a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a5dffb640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1a500a1200 0x7f1a500a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a655b6640 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f791790f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fd0 tx=0x7f1a5800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4777f7550c6bec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a64db5640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1a500a1200 0x7f1a500a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a5dffb640 1 -- 172.21.15.44:0/195044007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1a540ee3e0 con 0x7f1a600fee80 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a4affd640 1 -- 172.21.15.44:0/1741254234 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58011700 con 0x555f79178d10 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a4affd640 1 -- 172.21.15.44:0/1741254234 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a58011cf0 con 0x555f79178d10 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.412+0000 7f1a4affd640 1 -- 172.21.15.44:0/1741254234 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58010c20 con 0x555f79178d10 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 -- 172.21.15.44:0/1741254234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 msgr2=0x555f791790f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a64db5640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1a500a1200 0x7f1a500a36c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a540eb8c0 tx=0x7f1a54009260 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f791790f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fd0 tx=0x7f1a5800f7f0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 -- 172.21.15.44:0/1741254234 shutdown_connections 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f791883e0 0x555f7918a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f79179630 0x555f79187ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/1741254234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f791790f0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 -- 172.21.15.44:0/1741254234 >> 172.21.15.44:0/1741254234 conn(0x555f79175380 msgr2=0x555f79175760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 -- 172.21.15.44:0/1741254234 shutdown_connections 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.413+0000 7f1a69e1a740 1 -- 172.21.15.44:0/1741254234 wait complete. 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 -- start start 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 Processor -- start 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79187e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79185070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79185800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7907e2a0 con 0x555f79179630 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f78db5430 con 0x555f791883e0 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f79060680 con 0x555f79178d10 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a69e1a740 10 client.0 _ll_register_callbacks cb 0x555f790d6610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79185070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79185070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55590/0 (socket says 172.21.15.44:55590) 2024-09-13T17:32:15.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 -- 172.21.15.44:0/4137258826 learned_addr learned my addr 172.21.15.44:0/4137258826 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a655b6640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79185800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a655b6640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79187e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 -- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 msgr2=0x555f79187e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79187e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 -- 172.21.15.44:0/4137258826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 msgr2=0x555f79185800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79185800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.414+0000 7f1a5ffff640 1 -- 172.21.15.44:0/4137258826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a58009c00 con 0x555f79179630 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a655b6640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79185800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79185070 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1a60069730 tx=0x7f1a4c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b24cbb5ace36f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a3ffff640 1 -- 172.21.15.44:0/4137258826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c011850 con 0x555f79179630 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f78dd04e0 con 0x555f79179630 2024-09-13T17:32:15.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a3ffff640 1 -- 172.21.15.44:0/4137258826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1a4c015070 con 0x555f79179630 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a3ffff640 1 -- 172.21.15.44:0/4137258826 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4c0119b0 con 0x555f79179630 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 10 client.17403 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555f790db620 con 0x555f79179630 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a3ffff640 1 -- 172.21.15.44:0/4137258826 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f78dd04e0 con 0x555f79179630 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a3ffff640 1 -- 172.21.15.44:0/4137258826 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1a4c01acd0 con 0x555f79179630 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 10 client.17403 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 20 client.17403 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f7919a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 4 client.17403 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.415+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f78de17e0 con 0x555f79198310 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.416+0000 7f1a655b6640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f7919a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.416+0000 7f1a655b6640 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f7919a730 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd35aa5e67b50e36 server_cookie=84bdb3eef1eba3ca in_seq=0 out_seq=0 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.416+0000 7f1a3ffff640 10 client.17403 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a3ffff640 1 -- 172.21.15.44:0/4137258826 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555f78de17e0 con 0x555f79198310 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a3ffff640 10 client.17403 handle_command_reply: tid=0 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 client.17403 shutdown 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 msgr2=0x555f79185070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79185070 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1a60069730 tx=0x7f1a4c00f7f0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 shutdown_connections 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f7919a730 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79185800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79185070 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a3ffff640 0 client.17403 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/4137258826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79187e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.417+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 >> 172.21.15.44:0/4137258826 conn(0x555f79175380 msgr2=0x555f7917b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.418+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 shutdown_connections 2024-09-13T17:32:15.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.418+0000 7f1a69e1a740 1 -- 172.21.15.44:0/4137258826 wait complete. 2024-09-13T17:32:15.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.418+0000 7f1a69e1a740 20 client.17403 trim_cache size 0 max 16384 2024-09-13T17:32:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.431+0000 7f1a69e1a740 1 Processor -- start 2024-09-13T17:32:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.431+0000 7f1a69e1a740 1 -- start start 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79174e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79180300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79180840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f79196b10 con 0x555f79179630 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7917fe40 con 0x555f791883e0 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f791861b0 con 0x555f79178d10 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79180300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79180300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55592/0 (socket says 172.21.15.44:55592) 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 -- 172.21.15.44:0/432833113 learned_addr learned my addr 172.21.15.44:0/432833113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a655b6640 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79174e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a655b6640 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79180840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 -- 172.21.15.44:0/432833113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 msgr2=0x555f79174e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79174e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 -- 172.21.15.44:0/432833113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 msgr2=0x555f79180840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79180840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 -- 172.21.15.44:0/432833113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f7907d790 con 0x555f79179630 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a655b6640 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79180840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79180300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c010010 tx=0x7f1a4c010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8c7b5be3bf91c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a4b7fe640 1 -- 172.21.15.44:0/432833113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c004220 con 0x555f79179630 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a4b7fe640 1 -- 172.21.15.44:0/432833113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a4c010610 con 0x555f79179630 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.432+0000 7f1a4b7fe640 1 -- 172.21.15.44:0/432833113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0052a0 con 0x555f79179630 2024-09-13T17:32:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 -- 172.21.15.44:0/432833113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 msgr2=0x555f79180300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79180300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c010010 tx=0x7f1a4c010540 comp rx=0 tx=0).stop 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 -- 172.21.15.44:0/432833113 shutdown_connections 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f79180840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79179630 0x555f79180300 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/432833113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79178d10 0x555f79174e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 -- 172.21.15.44:0/432833113 >> 172.21.15.44:0/432833113 conn(0x555f79175380 msgr2=0x555f79175760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 -- 172.21.15.44:0/432833113 shutdown_connections 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 -- 172.21.15.44:0/432833113 wait complete. 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 -- start start 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.433+0000 7f1a69e1a740 1 Processor -- start 2024-09-13T17:32:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f7918b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79179630 0x555f7918bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f7918ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7918f090 con 0x555f79178d10 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7918f200 con 0x555f791883e0 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7918f3c0 con 0x555f79179630 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a69e1a740 10 client.0 _ll_register_callbacks cb 0x7f1a4c00a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f7918b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f7918b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55594/0 (socket says 172.21.15.44:55594) 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 -- 172.21.15.44:0/3882617783 learned_addr learned my addr 172.21.15.44:0/3882617783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f7918ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79179630 0x555f7918bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 -- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79179630 msgr2=0x555f7918bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79179630 0x555f7918bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 -- 172.21.15.44:0/3882617783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 msgr2=0x555f7918ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f7918ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 -- 172.21.15.44:0/3882617783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f790db620 con 0x555f79178d10 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a655b6640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f7918b790 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fd0 tx=0x7f1a4c00a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ca6e98631f6d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.434+0000 7f1a3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a3ffff640 1 -- 172.21.15.44:0/3882617783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5802f070 con 0x555f79178d10 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a3ffff640 1 -- 172.21.15.44:0/3882617783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f1a58010d00 con 0x555f79178d10 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a58011cf0 con 0x555f79178d10 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a3ffff640 1 -- 172.21.15.44:0/3882617783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a58010ec0 con 0x555f79178d10 2024-09-13T17:32:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 10 client.17409 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555f790a2ea0 con 0x555f79178d10 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a3ffff640 1 -- 172.21.15.44:0/3882617783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a58011cf0 con 0x555f79178d10 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a3ffff640 1 -- 172.21.15.44:0/3882617783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1a58011500 con 0x555f79178d10 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 10 client.17409 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 20 client.17409 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f791f1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 4 client.17409 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f78e34e30 con 0x555f79198310 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.435+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f791f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a5ffff640 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f791f1900 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7805a09b930f4e7 server_cookie=705ab7f8e65f5478 in_seq=0 out_seq=0 2024-09-13T17:32:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a3ffff640 10 client.17409 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a3ffff640 1 -- 172.21.15.44:0/3882617783 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555f78e34e30 con 0x555f79198310 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a3ffff640 10 client.17409 handle_command_reply: tid=0 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 client.17409 shutdown 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 msgr2=0x555f7918b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f7918b790 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fd0 tx=0x7f1a4c00a8a0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 shutdown_connections 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x555f79198310 0x555f791f1900 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f79179630 0x555f7918bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555f791883e0 0x555f7918ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 --2- 172.21.15.44:0/3882617783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f79178d10 0x555f7918b790 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a3ffff640 0 client.17409 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 >> 172.21.15.44:0/3882617783 conn(0x555f79175380 msgr2=0x555f79180300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 shutdown_connections 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 1 -- 172.21.15.44:0/3882617783 wait complete. 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.436+0000 7f1a69e1a740 20 client.17409 trim_cache size 0 max 16384 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:32:15.446 INFO:teuthology.orchestra.run.smithi044.stdout: "e8d283e3-99ca-4183-80c5-5c3ad7e4e7aa": { 2024-09-13T17:32:15.447 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:32:15.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "e8d283e3-99ca-4183-80c5-5c3ad7e4e7aa", 2024-09-13T17:32:15.447 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:32:15.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:32:15.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:32:15.447 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:32:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1a500a1200 msgr2=0x7f1a500a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1a500a1200 0x7f1a500a36c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a540eb8c0 tx=0x7f1a54009260 comp rx=0 tx=0).stop 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 msgr2=0x7f1a60197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a60197100 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1a5402a940 tx=0x7f1a540046c0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 shutdown_connections 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1a500a1200 0x7f1a500a36c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a60103fd0 0x7f1a60197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a60197c20 0x7f1a601a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 --2- 172.21.15.44:0/195044007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a600fee80 0x7f1a60197100 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 >> 172.21.15.44:0/195044007 conn(0x7f1a600fa930 msgr2=0x7f1a600fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 shutdown_connections 2024-09-13T17:32:15.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:15.437+0000 7f1a67019640 1 -- 172.21.15.44:0/195044007 wait complete. 2024-09-13T17:32:15.450 INFO:tasks.cephfs.filesystem:scrub status for tag:e8d283e3-99ca-4183-80c5-5c3ad7e4e7aa - {'path': '/', 'tag': 'e8d283e3-99ca-4183-80c5-5c3ad7e4e7aa', 'options': 'recursive,force'} 2024-09-13T17:32:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:15 smithi070 ceph-mon[34502]: pgmap v169: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 77 KiB/s wr, 33 op/s 2024-09-13T17:32:16.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:15 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:32:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:15 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:15 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:32:16.198 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:15 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:15 smithi100 ceph-mon[35055]: pgmap v169: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 77 KiB/s wr, 33 op/s 2024-09-13T17:32:16.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:15 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:32:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:15 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:15 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:32:16.275 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:15 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:16.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:15 smithi044 ceph-mon[31175]: pgmap v169: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 77 KiB/s wr, 33 op/s 2024-09-13T17:32:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:15 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:32:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:15 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:15 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:32:16.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:15 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:18.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:17 smithi070 ceph-mon[34502]: pgmap v170: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:18.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:17 smithi100 ceph-mon[35055]: pgmap v170: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:18.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:17 smithi044 ceph-mon[31175]: pgmap v170: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:19.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:18 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:32:19.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:18 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:32:19.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:18 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:32:20.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:19 smithi070 ceph-mon[34502]: pgmap v171: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:32:20.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:19 smithi100 ceph-mon[35055]: pgmap v171: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:32:20.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:19 smithi044 ceph-mon[31175]: pgmap v171: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:32:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 -- 172.21.15.44:0/2418492895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c105040 msgr2=0x7f0e2c10cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 --2- 172.21.15.44:0/2418492895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c105040 0x7f0e2c10cb90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009930 tx=0x7f0e2402a420 comp rx=0 tx=0).stop 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 -- 172.21.15.44:0/2418492895 shutdown_connections 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 --2- 172.21.15.44:0/2418492895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c105040 0x7f0e2c10cb90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 --2- 172.21.15.44:0/2418492895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c1008a0 0x7f0e2c1048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 --2- 172.21.15.44:0/2418492895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e2c0ffef0 0x7f0e2c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 -- 172.21.15.44:0/2418492895 >> 172.21.15.44:0/2418492895 conn(0x7f0e2c0fb960 msgr2=0x7f0e2c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 -- 172.21.15.44:0/2418492895 shutdown_connections 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.392+0000 7f0e3395e640 1 -- 172.21.15.44:0/2418492895 wait complete. 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 Processor -- start 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 -- start start 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c0ffef0 0x7f0e2c1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e2c105040 0x7f0e2c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 0x7f0e2c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19b470 con 0x7f0e2c19aa80 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19b5e0 con 0x7f0e2c0ffef0 2024-09-13T17:32:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e3395e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19b750 con 0x7f0e2c105040 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e31efb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 0x7f0e2c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e31efb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 0x7f0e2c19af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41424/0 (socket says 172.21.15.44:41424) 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e31efb640 1 -- 172.21.15.44:0/4002128722 learned_addr learned my addr 172.21.15.44:0/4002128722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e2c105040 0x7f0e2c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.393+0000 7f0e316fa640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c0ffef0 0x7f0e2c1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e31efb640 1 -- 172.21.15.44:0/4002128722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e2c105040 msgr2=0x7f0e2c196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e31efb640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e2c105040 0x7f0e2c196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e31efb640 1 -- 172.21.15.44:0/4002128722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c0ffef0 msgr2=0x7f0e2c1961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e31efb640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c0ffef0 0x7f0e2c1961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e31efb640 1 -- 172.21.15.44:0/4002128722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24009560 con 0x7f0e2c19aa80 2024-09-13T17:32:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e316fa640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c0ffef0 0x7f0e2c1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e31efb640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 0x7f0e2c19af30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009900 tx=0x7f0e24032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.394+0000 7f0e227fc640 1 -- 172.21.15.44:0/4002128722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240044f0 con 0x7f0e2c19aa80 2024-09-13T17:32:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.395+0000 7f0e227fc640 1 -- 172.21.15.44:0/4002128722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e24005490 con 0x7f0e2c19aa80 2024-09-13T17:32:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.395+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c1a2d30 con 0x7f0e2c19aa80 2024-09-13T17:32:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.395+0000 7f0e227fc640 1 -- 172.21.15.44:0/4002128722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24003850 con 0x7f0e2c19aa80 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.395+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1a31b0 con 0x7f0e2c19aa80 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.397+0000 7f0e227fc640 1 -- 172.21.15.44:0/4002128722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0e24004850 con 0x7f0e2c19aa80 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 Processor -- start 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 -- start start 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e33710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e227fc640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0e180a1280 0x7f0e180a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e3dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4d9ec30 con 0x55dca4e3e4f0 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4d9c3c0 con 0x55dca4e33330 2024-09-13T17:32:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e36682740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4d97cb0 con 0x55dca4e33c50 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e316fa640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0e180a1280 0x7f0e180a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e31efb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e30ef9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e3dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e227fc640 1 -- 172.21.15.44:0/4002128722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0e240edbb0 con 0x7f0e2c19aa80 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e30ef9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e3dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:46408/0 (socket says 172.21.15.44:46408) 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e31efb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41430/0 (socket says 172.21.15.44:41430) 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e30ef9640 1 -- 172.21.15.44:0/2105075431 learned_addr learned my addr 172.21.15.44:0/2105075431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.398+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e33710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e31efb640 1 -- 172.21.15.44:0/2105075431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 msgr2=0x55dca4e3dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e31efb640 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e3dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e31efb640 1 -- 172.21.15.44:0/2105075431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 msgr2=0x55dca4e33710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e31efb640 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e33710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e31efb640 1 -- 172.21.15.44:0/2105075431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dca4d3e910 con 0x55dca4e3e4f0 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e3dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e316fa640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0e180a1280 0x7f0e180a3740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e28005fd0 tx=0x7f0e28009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e31efb640 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e408e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0e240eafe0 tx=0x7f0e24041600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b8186443b4f42f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e177fe640 1 -- 172.21.15.44:0/2105075431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240bb0a0 con 0x55dca4e3e4f0 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e177fe640 1 -- 172.21.15.44:0/2105075431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e240c7830 con 0x55dca4e3e4f0 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.399+0000 7f0e177fe640 1 -- 172.21.15.44:0/2105075431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240eb260 con 0x55dca4e3e4f0 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 -- 172.21.15.44:0/2105075431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 msgr2=0x55dca4e408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e408e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0e240eafe0 tx=0x7f0e24041600 comp rx=0 tx=0).stop 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 -- 172.21.15.44:0/2105075431 shutdown_connections 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e3dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e408e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 --2- 172.21.15.44:0/2105075431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e33710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 -- 172.21.15.44:0/2105075431 >> 172.21.15.44:0/2105075431 conn(0x55dca4e2f8c0 msgr2=0x55dca4e2fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 -- 172.21.15.44:0/2105075431 shutdown_connections 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 -- 172.21.15.44:0/2105075431 wait complete. 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 -- start start 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.400+0000 7f0e36682740 1 Processor -- start 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e3dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33c50 0x55dca4e3b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e3b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4d55e80 con 0x55dca4e3e4f0 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4d55b20 con 0x55dca4e33c50 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4d1b260 con 0x55dca4e33330 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e36682740 10 client.0 _ll_register_callbacks cb 0x55dca4d50df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e316fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e3dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e316fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e3dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:46414/0 (socket says 172.21.15.44:46414) 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e31efb640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e3b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e316fa640 1 -- 172.21.15.44:0/3978357293 learned_addr learned my addr 172.21.15.44:0/3978357293 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.401+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33c50 0x55dca4e3b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e316fa640 1 -- 172.21.15.44:0/3978357293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33c50 msgr2=0x55dca4e3b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e316fa640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33c50 0x55dca4e3b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e316fa640 1 -- 172.21.15.44:0/3978357293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 msgr2=0x55dca4e3b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e316fa640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e3b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e316fa640 1 -- 172.21.15.44:0/3978357293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1a31b0 con 0x55dca4e33330 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e31efb640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e3b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e316fa640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e3dee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e28005c40 tx=0x7f0e28004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc5d5522b59a610 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3978357293 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800ee90 con 0x55dca4e33330 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3978357293 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0e28017070 con 0x55dca4e33330 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dca4dc0680 con 0x55dca4e33330 2024-09-13T17:32:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3978357293 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e2801e420 con 0x55dca4e33330 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e36682740 10 client.26794 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.402+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dca4d96330 con 0x55dca4e33330 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3978357293 <== mon.1 v2:172.21.15.100:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dca4dc0680 con 0x55dca4e33330 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3978357293 <== mon.1 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e28024020 con 0x55dca4e33330 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e36682740 10 client.26794 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e36682740 10 client.26794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e36682740 20 client.26794 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e36682740 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4e4dc70 0x55dca4e50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e36682740 4 client.26794 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dca4d8b5d0 con 0x55dca4e4dc70 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4e4dc70 0x55dca4e50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e14ff9640 10 client.26794 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.403+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4e4dc70 0x55dca4e50090 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f5d5ddb981ccff server_cookie=9a30b1937cf438d7 in_seq=0 out_seq=0 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3978357293 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dca4d8b5d0 con 0x55dca4e4dc70 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e14ff9640 10 client.26794 handle_command_reply: tid=0 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 client.26794 shutdown 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 msgr2=0x55dca4e3dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e3dee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e28005c40 tx=0x7f0e28004ab0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 shutdown_connections 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4e4dc70 0x55dca4e50090 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e3b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e14ff9640 0 client.26794 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33c50 0x55dca4e3b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 --2- 172.21.15.44:0/3978357293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e3dee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 >> 172.21.15.44:0/3978357293 conn(0x55dca4e2f8c0 msgr2=0x55dca4e2eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 shutdown_connections 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.404+0000 7f0e36682740 1 -- 172.21.15.44:0/3978357293 wait complete. 2024-09-13T17:32:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.405+0000 7f0e36682740 20 client.26794 trim_cache size 0 max 16384 2024-09-13T17:32:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 Processor -- start 2024-09-13T17:32:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 -- start start 2024-09-13T17:32:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e3da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e44f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e45470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4e30a30 con 0x55dca4e3e4f0 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4e4a900 con 0x55dca4e33330 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.417+0000 7f0e36682740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4e39690 con 0x55dca4e33c50 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e45470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e45470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41444/0 (socket says 172.21.15.44:41444) 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 -- 172.21.15.44:0/4092867764 learned_addr learned my addr 172.21.15.44:0/4092867764 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e316fa640 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e3da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e44f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 -- 172.21.15.44:0/4092867764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 msgr2=0x55dca4e44f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e44f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 -- 172.21.15.44:0/4092867764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 msgr2=0x55dca4e3da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e3da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 -- 172.21.15.44:0/4092867764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dca4d38300 con 0x55dca4e3e4f0 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e31efb640 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e45470 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0e240eb090 tx=0x7f0e240058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d2ad101d758ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e17fff640 1 -- 172.21.15.44:0/4092867764 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240ba9a0 con 0x55dca4e3e4f0 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e17fff640 1 -- 172.21.15.44:0/4092867764 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e240c2e40 con 0x55dca4e3e4f0 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.418+0000 7f0e17fff640 1 -- 172.21.15.44:0/4092867764 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24041270 con 0x55dca4e3e4f0 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 -- 172.21.15.44:0/4092867764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 msgr2=0x55dca4e45470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e45470 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0e240eb090 tx=0x7f0e240058b0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 -- 172.21.15.44:0/4092867764 shutdown_connections 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33c50 0x55dca4e44f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e3e4f0 0x55dca4e45470 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 --2- 172.21.15.44:0/4092867764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e33330 0x55dca4e3da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 -- 172.21.15.44:0/4092867764 >> 172.21.15.44:0/4092867764 conn(0x55dca4e2f8c0 msgr2=0x55dca4e39a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 -- 172.21.15.44:0/4092867764 shutdown_connections 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 -- 172.21.15.44:0/4092867764 wait complete. 2024-09-13T17:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 -- start start 2024-09-13T17:32:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.419+0000 7f0e36682740 1 Processor -- start 2024-09-13T17:32:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e45280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 0x55dca4e41590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e3e4f0 0x55dca4e41ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4e421d0 con 0x55dca4e33c50 2024-09-13T17:32:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4e42340 con 0x55dca4e3e4f0 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca4ea4130 con 0x55dca4e33330 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e36682740 10 client.0 _ll_register_callbacks cb 0x7f0e24004b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 0x55dca4e41590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 0x55dca4e41590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41454/0 (socket says 172.21.15.44:41454) 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 -- 172.21.15.44:0/3117333114 learned_addr learned my addr 172.21.15.44:0/3117333114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e316fa640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e45280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e31efb640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e3e4f0 0x55dca4e41ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 -- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 msgr2=0x55dca4e45280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e45280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 -- 172.21.15.44:0/3117333114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e3e4f0 msgr2=0x55dca4e41ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e3e4f0 0x55dca4e41ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 -- 172.21.15.44:0/3117333114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dca4d3e910 con 0x55dca4e33c50 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e30ef9640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 0x55dca4e41590 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00bfd0 tx=0x7f0e1c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ef9ffc6222ca97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.420+0000 7f0e14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3117333114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c005690 con 0x55dca4e33c50 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e240c7830 con 0x55dca4e33c50 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3117333114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0e1c004940 con 0x55dca4e33c50 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3117333114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e1c004b00 con 0x55dca4e33c50 2024-09-13T17:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 10 client.17427 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dca4d5d8e0 con 0x55dca4e33c50 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3117333114 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e240c7830 con 0x55dca4e33c50 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3117333114 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e1c02a440 con 0x55dca4e33c50 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 10 client.17427 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 20 client.17427 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4ea5940 0x55dca4ea7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 4 client.17427 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.421+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dca4a8b300 con 0x55dca4ea5940 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.422+0000 7f0e316fa640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4ea5940 0x55dca4ea7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.422+0000 7f0e316fa640 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4ea5940 0x55dca4ea7d60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3985bdcc90c940ea server_cookie=f20ac3fe796bad7a in_seq=0 out_seq=0 2024-09-13T17:32:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.422+0000 7f0e14ff9640 10 client.17427 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e14ff9640 1 -- 172.21.15.44:0/3117333114 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dca4a8b300 con 0x55dca4ea5940 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e14ff9640 10 client.17427 handle_command_reply: tid=0 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 client.17427 shutdown 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 msgr2=0x55dca4e41590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 0x55dca4e41590 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00bfd0 tx=0x7f0e1c005900 comp rx=0 tx=0).stop 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 shutdown_connections 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dca4ea5940 0x55dca4ea7d60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dca4e3e4f0 0x55dca4e41ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dca4e33c50 0x55dca4e41590 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e14ff9640 0 client.17427 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 --2- 172.21.15.44:0/3117333114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dca4e33330 0x55dca4e45280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 >> 172.21.15.44:0/3117333114 conn(0x55dca4e2f8c0 msgr2=0x55dca4e39850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 shutdown_connections 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 1 -- 172.21.15.44:0/3117333114 wait complete. 2024-09-13T17:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.435+0000 7f0e36682740 20 client.17427 trim_cache size 0 max 16384 2024-09-13T17:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0e180a1280 msgr2=0x7f0e180a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0e180a1280 0x7f0e180a3740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e28005fd0 tx=0x7f0e28009500 comp rx=0 tx=0).stop 2024-09-13T17:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 msgr2=0x7f0e2c19af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 0x7f0e2c19af30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009900 tx=0x7f0e24032ba0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 shutdown_connections 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0e180a1280 0x7f0e180a3740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e2c105040 0x7f0e2c196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e2c19aa80 0x7f0e2c19af30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 --2- 172.21.15.44:0/4002128722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c0ffef0 0x7f0e2c1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 >> 172.21.15.44:0/4002128722 conn(0x7f0e2c0fb960 msgr2=0x7f0e2c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 shutdown_connections 2024-09-13T17:32:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:21.436+0000 7f0e3395e640 1 -- 172.21.15.44:0/4002128722 wait complete. 2024-09-13T17:32:22.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:21 smithi070 ceph-mon[34502]: pgmap v172: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:32:22.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:21 smithi100 ceph-mon[35055]: pgmap v172: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:32:22.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:21 smithi044 ceph-mon[31175]: pgmap v172: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:32:24.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:23 smithi070 ceph-mon[34502]: pgmap v173: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:32:24.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:23 smithi100 ceph-mon[35055]: pgmap v173: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:32:24.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:23 smithi044 ceph-mon[31175]: pgmap v173: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:32:26.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:25 smithi070 ceph-mon[34502]: pgmap v174: 129 pgs: 129 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-13T17:32:26.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:25 smithi100 ceph-mon[35055]: pgmap v174: 129 pgs: 129 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-13T17:32:26.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:25 smithi044 ceph-mon[31175]: pgmap v174: 129 pgs: 129 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-13T17:32:28.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:27 smithi070 ceph-mon[34502]: pgmap v175: 129 pgs: 129 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-13T17:32:28.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:27 smithi100 ceph-mon[35055]: pgmap v175: 129 pgs: 129 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-13T17:32:28.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:27 smithi044 ceph-mon[31175]: pgmap v175: 129 pgs: 129 active+clean; 4.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-13T17:32:30.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:29 smithi100 ceph-mon[35055]: pgmap v176: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-13T17:32:30.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:29 smithi044 ceph-mon[31175]: pgmap v176: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-13T17:32:30.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:29 smithi070 ceph-mon[34502]: pgmap v176: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-13T17:32:32.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:31 smithi100 ceph-mon[35055]: pgmap v177: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-13T17:32:32.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:31 smithi044 ceph-mon[31175]: pgmap v177: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-13T17:32:32.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:31 smithi070 ceph-mon[34502]: pgmap v177: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-13T17:32:34.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:33 smithi100 ceph-mon[35055]: pgmap v178: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-13T17:32:34.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:33 smithi044 ceph-mon[31175]: pgmap v178: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-13T17:32:34.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:33 smithi070 ceph-mon[34502]: pgmap v178: 129 pgs: 129 active+clean; 4.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-13T17:32:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 -- 172.21.15.44:0/3261007836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940fef60 msgr2=0x7fbb94102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/3261007836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940fef60 0x7fbb94102fa0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802a410 comp rx=0 tx=0).stop 2024-09-13T17:32:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 -- 172.21.15.44:0/3261007836 shutdown_connections 2024-09-13T17:32:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/3261007836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb94103700 0x7fbb9410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/3261007836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940fef60 0x7fbb94102fa0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/3261007836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb940fe5b0 0x7fbb940fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 -- 172.21.15.44:0/3261007836 >> 172.21.15.44:0/3261007836 conn(0x7fbb94075350 msgr2=0x7fbb940fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 -- 172.21.15.44:0/3261007836 shutdown_connections 2024-09-13T17:32:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 -- 172.21.15.44:0/3261007836 wait complete. 2024-09-13T17:32:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.534+0000 7fbb9a42b640 1 Processor -- start 2024-09-13T17:32:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 -- start start 2024-09-13T17:32:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb940fe5b0 0x7fbb9410c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb94103700 0x7fbb9410ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 0x7fbb941a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941a48d0 con 0x7fbb9410d3f0 2024-09-13T17:32:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941a4a40 con 0x7fbb940fe5b0 2024-09-13T17:32:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb9a42b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941a4c00 con 0x7fbb94103700 2024-09-13T17:32:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 0x7fbb941a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 0x7fbb941a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41428/0 (socket says 172.21.15.44:41428) 2024-09-13T17:32:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 -- 172.21.15.44:0/2721351935 learned_addr learned my addr 172.21.15.44:0/2721351935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb93fff640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb940fe5b0 0x7fbb9410c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb937fe640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb94103700 0x7fbb9410ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 -- 172.21.15.44:0/2721351935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb94103700 msgr2=0x7fbb9410ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb94103700 0x7fbb9410ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 -- 172.21.15.44:0/2721351935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb940fe5b0 msgr2=0x7fbb9410c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb940fe5b0 0x7fbb9410c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.535+0000 7fbb989c8640 1 -- 172.21.15.44:0/2721351935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88009560 con 0x7fbb9410d3f0 2024-09-13T17:32:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.536+0000 7fbb989c8640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 0x7fbb941a4300 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eb10 tx=0x7fbb8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.536+0000 7fbb917fa640 1 -- 172.21.15.44:0/2721351935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004b20 con 0x7fbb9410d3f0 2024-09-13T17:32:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.536+0000 7fbb917fa640 1 -- 172.21.15.44:0/2721351935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb8c004c80 con 0x7fbb9410d3f0 2024-09-13T17:32:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.536+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb941a4ee0 con 0x7fbb9410d3f0 2024-09-13T17:32:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.536+0000 7fbb917fa640 1 -- 172.21.15.44:0/2721351935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c005a10 con 0x7fbb9410d3f0 2024-09-13T17:32:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.536+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb941a5310 con 0x7fbb9410d3f0 2024-09-13T17:32:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.538+0000 7fbb917fa640 1 -- 172.21.15.44:0/2721351935 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbb8c005020 con 0x7fbb9410d3f0 2024-09-13T17:32:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 Processor -- start 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 -- start start 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c37710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c41fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb917fa640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb7c0a1230 0x7fbb7c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049ba2c30 con 0x561049c37330 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049ba03c0 con 0x561049c424f0 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb9d229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049b9bcb0 con 0x561049c37c50 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c37710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c37710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41432/0 (socket says 172.21.15.44:41432) 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb937fe640 1 -- 172.21.15.44:0/1822981767 learned_addr learned my addr 172.21.15.44:0/1822981767 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb93fff640 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c41fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb917fa640 1 -- 172.21.15.44:0/2721351935 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbb8c014070 con 0x7fbb9410d3f0 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb937fe640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb7c0a1230 0x7fbb7c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.539+0000 7fbb989c8640 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 -- 172.21.15.44:0/1822981767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 msgr2=0x561049c41fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c41fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 -- 172.21.15.44:0/1822981767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 msgr2=0x561049c448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 -- 172.21.15.44:0/1822981767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561049b42910 con 0x561049c37330 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb93fff640 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c41fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c37710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005c40 tx=0x7fbb88004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2cbdd2f41289a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb937fe640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb7c0a1230 0x7fbb7c0a36f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb94066ea0 tx=0x7fbb88047000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb7a7fc640 1 -- 172.21.15.44:0/1822981767 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88031480 con 0x561049c37330 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb7a7fc640 1 -- 172.21.15.44:0/1822981767 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb88045040 con 0x561049c37330 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.540+0000 7fbb7a7fc640 1 -- 172.21.15.44:0/1822981767 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88049430 con 0x561049c37330 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 -- 172.21.15.44:0/1822981767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 msgr2=0x561049c37710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c37710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005c40 tx=0x7fbb88004540 comp rx=0 tx=0).stop 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 -- 172.21.15.44:0/1822981767 shutdown_connections 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c41fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1822981767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c37710 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 -- 172.21.15.44:0/1822981767 >> 172.21.15.44:0/1822981767 conn(0x561049c338c0 msgr2=0x561049c33ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 -- 172.21.15.44:0/1822981767 shutdown_connections 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 -- 172.21.15.44:0/1822981767 wait complete. 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.541+0000 7fbb9d229740 1 -- start start 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 Processor -- start 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37330 0x561049c41f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c3f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 0x561049c3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049b59e80 con 0x561049c424f0 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049b59b20 con 0x561049c37330 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049b1f260 con 0x561049c37c50 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb9d229740 10 client.0 _ll_register_callbacks cb 0x561049b54df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 0x561049c3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 0x561049c3f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41436/0 (socket says 172.21.15.44:41436) 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 -- 172.21.15.44:0/210555299 learned_addr learned my addr 172.21.15.44:0/210555299 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb93fff640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37330 0x561049c41f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb937fe640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c3f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 -- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 msgr2=0x561049c3f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c3f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 -- 172.21.15.44:0/210555299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37330 msgr2=0x561049c41f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37330 0x561049c41f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 -- 172.21.15.44:0/210555299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb940680a0 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb989c8640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 0x561049c3f810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c09a040 tx=0x7fbb8c099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ac0b5d378e9809 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.542+0000 7fbb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb737fe640 1 -- 172.21.15.44:0/210555299 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0cd3f0 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb737fe640 1 -- 172.21.15.44:0/210555299 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbb8c09aa40 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561049bc4680 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb737fe640 1 -- 172.21.15.44:0/210555299 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8c09a340 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 10 client.17439 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561049b9a330 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb737fe640 1 -- 172.21.15.44:0/210555299 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561049bc4680 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb737fe640 1 -- 172.21.15.44:0/210555299 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb8c0b0020 con 0x561049c424f0 2024-09-13T17:32:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 10 client.17439 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 20 client.17439 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049c51ef0 0x561049c54310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 4 client.17439 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.543+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561049b8f5d0 con 0x561049c51ef0 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.544+0000 7fbb93fff640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049c51ef0 0x561049c54310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.544+0000 7fbb93fff640 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049c51ef0 0x561049c54310 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b966e5494190e3 server_cookie=3c923ca3d74e34d6 in_seq=0 out_seq=0 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.544+0000 7fbb737fe640 10 client.17439 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb737fe640 1 -- 172.21.15.44:0/210555299 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561049b8f5d0 con 0x561049c51ef0 2024-09-13T17:32:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb737fe640 10 client.17439 handle_command_reply: tid=0 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 client.17439 shutdown 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 msgr2=0x561049c3f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 0x561049c3f810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c09a040 tx=0x7fbb8c099d60 comp rx=0 tx=0).stop 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 shutdown_connections 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049c51ef0 0x561049c54310 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c3f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c424f0 0x561049c3f810 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb737fe640 0 client.17439 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 --2- 172.21.15.44:0/210555299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37330 0x561049c41f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 >> 172.21.15.44:0/210555299 conn(0x561049c338c0 msgr2=0x561049c32b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 shutdown_connections 2024-09-13T17:32:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 1 -- 172.21.15.44:0/210555299 wait complete. 2024-09-13T17:32:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.545+0000 7fbb9d229740 20 client.17439 trim_cache size 0 max 16384 2024-09-13T17:32:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 Processor -- start 2024-09-13T17:32:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 -- start start 2024-09-13T17:32:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37c50 0x561049c491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c424f0 0x561049c496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049c34970 con 0x561049c37330 2024-09-13T17:32:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049c371b0 con 0x561049c37c50 2024-09-13T17:32:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb9d229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049c34610 con 0x561049c424f0 2024-09-13T17:32:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c41a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41450/0 (socket says 172.21.15.44:41450) 2024-09-13T17:32:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 -- 172.21.15.44:0/4262139447 learned_addr learned my addr 172.21.15.44:0/4262139447 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 -- 172.21.15.44:0/4262139447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c424f0 msgr2=0x561049c496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb989c8640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c424f0 0x561049c496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb937fe640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37c50 0x561049c491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c424f0 0x561049c496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 -- 172.21.15.44:0/4262139447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37c50 msgr2=0x561049c491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37c50 0x561049c491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 -- 172.21.15.44:0/4262139447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561049b3c300 con 0x561049c37330 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb989c8640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c424f0 0x561049c496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb937fe640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37c50 0x561049c491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.558+0000 7fbb93fff640 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c41a20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fbb84000c00 tx=0x7fbb84005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81139fea5ab5cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb7affd640 1 -- 172.21.15.44:0/4262139447 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84017070 con 0x561049c37330 2024-09-13T17:32:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb7affd640 1 -- 172.21.15.44:0/4262139447 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb84018040 con 0x561049c37330 2024-09-13T17:32:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb7affd640 1 -- 172.21.15.44:0/4262139447 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84005020 con 0x561049c37330 2024-09-13T17:32:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 -- 172.21.15.44:0/4262139447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 msgr2=0x561049c41a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c41a20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fbb84000c00 tx=0x7fbb84005f90 comp rx=0 tx=0).stop 2024-09-13T17:32:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 -- 172.21.15.44:0/4262139447 shutdown_connections 2024-09-13T17:32:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c424f0 0x561049c496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c37c50 0x561049c491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 --2- 172.21.15.44:0/4262139447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c41a20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 -- 172.21.15.44:0/4262139447 >> 172.21.15.44:0/4262139447 conn(0x561049c338c0 msgr2=0x561049c3dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 -- 172.21.15.44:0/4262139447 shutdown_connections 2024-09-13T17:32:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 -- 172.21.15.44:0/4262139447 wait complete. 2024-09-13T17:32:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.559+0000 7fbb9d229740 1 -- start start 2024-09-13T17:32:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 Processor -- start 2024-09-13T17:32:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c45c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049c49ae0 con 0x561049c37330 2024-09-13T17:32:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049ca83f0 con 0x561049c424f0 2024-09-13T17:32:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561049ca8580 con 0x561049c37c50 2024-09-13T17:32:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb9d229740 10 client.0 _ll_register_callbacks cb 0x7fbb84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c45c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c45c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41456/0 (socket says 172.21.15.44:41456) 2024-09-13T17:32:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb93fff640 1 -- 172.21.15.44:0/1624605553 learned_addr learned my addr 172.21.15.44:0/1624605553 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb937fe640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.560+0000 7fbb989c8640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb93fff640 1 -- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 msgr2=0x561049c461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb93fff640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb93fff640 1 -- 172.21.15.44:0/1624605553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 msgr2=0x561049c466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb93fff640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb93fff640 1 -- 172.21.15.44:0/1624605553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb84010430 con 0x561049c37330 2024-09-13T17:32:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb93fff640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c45c60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009c30 tx=0x7fbb8400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bfc92caf64a07e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb737fe640 1 -- 172.21.15.44:0/1624605553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84017030 con 0x561049c37330 2024-09-13T17:32:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb737fe640 1 -- 172.21.15.44:0/1624605553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbb84005030 con 0x561049c37330 2024-09-13T17:32:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb88045040 con 0x561049c37330 2024-09-13T17:32:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb737fe640 1 -- 172.21.15.44:0/1624605553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb84010e60 con 0x561049c37330 2024-09-13T17:32:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb9d229740 10 client.17445 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561049b618e0 con 0x561049c37330 2024-09-13T17:32:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb737fe640 1 -- 172.21.15.44:0/1624605553 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb88045040 con 0x561049c37330 2024-09-13T17:32:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.561+0000 7fbb737fe640 1 -- 172.21.15.44:0/1624605553 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb84004d20 con 0x561049c37330 2024-09-13T17:32:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb9d229740 10 client.17445 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb9d229740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb9d229740 20 client.17445 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049ca9c20 0x561049cac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb9d229740 4 client.17445 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:32:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56104988f300 con 0x561049ca9c20 2024-09-13T17:32:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb937fe640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049ca9c20 0x561049cac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb737fe640 10 client.17445 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.562+0000 7fbb937fe640 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049ca9c20 0x561049cac040 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1770a6f76c37a14 server_cookie=c06bdc30d0ddcd62 in_seq=0 out_seq=0 2024-09-13T17:32:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.577+0000 7fbb737fe640 1 -- 172.21.15.44:0/1624605553 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56104988f300 con 0x561049ca9c20 2024-09-13T17:32:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb737fe640 10 client.17445 handle_command_reply: tid=0 2024-09-13T17:32:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 client.17445 shutdown 2024-09-13T17:32:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 msgr2=0x561049c45c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c45c60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009c30 tx=0x7fbb8400fb00 comp rx=0 tx=0).stop 2024-09-13T17:32:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 shutdown_connections 2024-09-13T17:32:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561049ca9c20 0x561049cac040 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561049c37c50 0x561049c461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561049c424f0 0x561049c466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 --2- 172.21.15.44:0/1624605553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561049c37330 0x561049c45c60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb737fe640 0 client.17445 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 >> 172.21.15.44:0/1624605553 conn(0x561049c338c0 msgr2=0x561049c322f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 shutdown_connections 2024-09-13T17:32:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 1 -- 172.21.15.44:0/1624605553 wait complete. 2024-09-13T17:32:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.578+0000 7fbb9d229740 20 client.17445 trim_cache size 0 max 16384 2024-09-13T17:32:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:32:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:32:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:32:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb7c0a1230 msgr2=0x7fbb7c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb7c0a1230 0x7fbb7c0a36f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb94066ea0 tx=0x7fbb88047000 comp rx=0 tx=0).stop 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 msgr2=0x7fbb941a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 0x7fbb941a4300 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eb10 tx=0x7fbb8c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 shutdown_connections 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbb7c0a1230 0x7fbb7c0a36f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb94103700 0x7fbb9410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9410d3f0 0x7fbb941a4300 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 --2- 172.21.15.44:0/2721351935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb940fe5b0 0x7fbb9410c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 >> 172.21.15.44:0/2721351935 conn(0x7fbb94075350 msgr2=0x7fbb94101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 shutdown_connections 2024-09-13T17:32:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:34.579+0000 7fbb9a42b640 1 -- 172.21.15.44:0/2721351935 wait complete. 2024-09-13T17:32:36.250 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:35 smithi070 ceph-mon[34502]: pgmap v179: 129 pgs: 129 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-13T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:35 smithi100 ceph-mon[35055]: pgmap v179: 129 pgs: 129 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-13T17:32:36.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:35 smithi044 ceph-mon[31175]: pgmap v179: 129 pgs: 129 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-13T17:32:38.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:37 smithi100 ceph-mon[35055]: pgmap v180: 129 pgs: 129 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:32:38.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:37 smithi044 ceph-mon[31175]: pgmap v180: 129 pgs: 129 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:32:38.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:37 smithi070 ceph-mon[34502]: pgmap v180: 129 pgs: 129 active+clean; 5.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:32:40.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:39 smithi100 ceph-mon[35055]: pgmap v181: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:32:40.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:39 smithi044 ceph-mon[31175]: pgmap v181: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:32:40.449 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:39 smithi070 ceph-mon[34502]: pgmap v181: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:32:42.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:41 smithi100 ceph-mon[35055]: pgmap v182: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:42.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:41 smithi044 ceph-mon[31175]: pgmap v182: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:42.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:41 smithi070 ceph-mon[34502]: pgmap v182: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:44.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:43 smithi100 ceph-mon[35055]: pgmap v183: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:44.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:43 smithi044 ceph-mon[31175]: pgmap v183: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:44.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:43 smithi070 ceph-mon[34502]: pgmap v183: 129 pgs: 129 active+clean; 5.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:32:45.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:32:45.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 -- 172.21.15.44:0/4227091590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30841050a0 msgr2=0x7f308410cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 --2- 172.21.15.44:0/4227091590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30841050a0 0x7f308410cbf0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f307c009cd0 tx=0x7f307c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 -- 172.21.15.44:0/4227091590 shutdown_connections 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 --2- 172.21.15.44:0/4227091590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30841050a0 0x7f308410cbf0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 --2- 172.21.15.44:0/4227091590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3084100900 0x7f3084104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 --2- 172.21.15.44:0/4227091590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30840fff50 0x7f3084100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 -- 172.21.15.44:0/4227091590 >> 172.21.15.44:0/4227091590 conn(0x7f30840fb9e0 msgr2=0x7f30840fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 -- 172.21.15.44:0/4227091590 shutdown_connections 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.568+0000 7f308985a640 1 -- 172.21.15.44:0/4227091590 wait complete. 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 Processor -- start 2024-09-13T17:32:45.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 -- start start 2024-09-13T17:32:45.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f30841966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 0x7f308419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419b4a0 con 0x7f308419aab0 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419b610 con 0x7f30840fff50 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f308985a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419b780 con 0x7f30841050a0 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f30837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 0x7f308419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f30837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 0x7f308419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50342/0 (socket says 172.21.15.44:50342) 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f30837fe640 1 -- 172.21.15.44:0/1021741512 learned_addr learned my addr 172.21.15.44:0/1021741512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f3082ffd640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.569+0000 7f30827fc640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f30841966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f30837fe640 1 -- 172.21.15.44:0/1021741512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 msgr2=0x7f30841966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f30837fe640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f30841966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f30837fe640 1 -- 172.21.15.44:0/1021741512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 msgr2=0x7f3084196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f30837fe640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f30837fe640 1 -- 172.21.15.44:0/1021741512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009900 con 0x7f308419aab0 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f30837fe640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 0x7f308419af60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f307c004af0 tx=0x7f307c030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f3088858640 1 -- 172.21.15.44:0/1021741512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c031b80 con 0x7f308419aab0 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f3088858640 1 -- 172.21.15.44:0/1021741512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f307c003e00 con 0x7f308419aab0 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841a2d60 con 0x7f308419aab0 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f3088858640 1 -- 172.21.15.44:0/1021741512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c045410 con 0x7f308419aab0 2024-09-13T17:32:45.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.570+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841a3210 con 0x7f308419aab0 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.572+0000 7f3088858640 1 -- 172.21.15.44:0/1021741512 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f307c00a040 con 0x7f308419aab0 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 Processor -- start 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 -- start start 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d5358511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d535851730 0x55d53585ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d53585bfd0 0x55d53585e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5357bbf20 con 0x55d535850e10 2024-09-13T17:32:45.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5357b96b0 con 0x55d535851730 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f308c654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5357b4fa0 con 0x55d53585bfd0 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f30827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d5358511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f3088858640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f30827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d5358511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50348/0 (socket says 172.21.15.44:50348) 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f30827fc640 1 -- 172.21.15.44:0/2042292144 learned_addr learned my addr 172.21.15.44:0/2042292144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d535851730 0x55d53585ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f30837fe640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f30827fc640 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d53585bfd0 0x55d53585e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f3088858640 1 -- 172.21.15.44:0/1021741512 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f307c0ee330 con 0x7f308419aab0 2024-09-13T17:32:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.573+0000 7f3082ffd640 1 -- 172.21.15.44:0/2042292144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d53585bfd0 msgr2=0x55d53585e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d53585bfd0 0x55d53585e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3082ffd640 1 -- 172.21.15.44:0/2042292144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 msgr2=0x55d5358511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d5358511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3082ffd640 1 -- 172.21.15.44:0/2042292144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53576f840 con 0x55d535851730 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d535851730 0x55d53585ba90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3074009c00 tx=0x7f3074011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe88bc8d0c06c0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f30837fe640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f307c0b7be0 tx=0x7f307c02a8e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3069ffb640 1 -- 172.21.15.44:0/2042292144 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074004230 con 0x55d535851730 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3069ffb640 1 -- 172.21.15.44:0/2042292144 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3074009e70 con 0x55d535851730 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.574+0000 7f3069ffb640 1 -- 172.21.15.44:0/2042292144 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074004b20 con 0x55d535851730 2024-09-13T17:32:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 -- 172.21.15.44:0/2042292144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d535851730 msgr2=0x55d53585ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d535851730 0x55d53585ba90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3074009c00 tx=0x7f3074011810 comp rx=0 tx=0).stop 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 -- 172.21.15.44:0/2042292144 shutdown_connections 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d53585bfd0 0x55d53585e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d535851730 0x55d53585ba90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 --2- 172.21.15.44:0/2042292144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d5358511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 -- 172.21.15.44:0/2042292144 >> 172.21.15.44:0/2042292144 conn(0x55d53584d380 msgr2=0x55d53584d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 -- 172.21.15.44:0/2042292144 shutdown_connections 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 -- 172.21.15.44:0/2042292144 wait complete. 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 -- start start 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.575+0000 7f308c654740 1 Processor -- start 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d53585b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d535858cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535859250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5357562a0 con 0x55d535851730 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53548d430 con 0x55d53585bfd0 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d535738680 con 0x55d535850e10 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f308c654740 10 client.0 _ll_register_callbacks cb 0x55d5357ae610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d535858cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d535858cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50358/0 (socket says 172.21.15.44:50358) 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 -- 172.21.15.44:0/417187232 learned_addr learned my addr 172.21.15.44:0/417187232 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f30827fc640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535859250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f30827fc640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d53585b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 -- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 msgr2=0x55d53585b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d53585b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 -- 172.21.15.44:0/417187232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 msgr2=0x55d535859250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535859250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 -- 172.21.15.44:0/417187232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3074009830 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f30827fc640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d53585b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3082ffd640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d535858cb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3074004620 tx=0x7f3074019af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2960b89874a0a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3062ffd640 1 -- 172.21.15.44:0/417187232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074024470 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.576+0000 7f3062ffd640 1 -- 172.21.15.44:0/417187232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f30740245d0 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5354a84e0 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f3062ffd640 1 -- 172.21.15.44:0/417187232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3074024790 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 10 client.17457 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d5357b3620 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f3062ffd640 1 -- 172.21.15.44:0/417187232 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d5354a84e0 con 0x55d535851730 2024-09-13T17:32:45.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f3062ffd640 1 -- 172.21.15.44:0/417187232 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f30740239b0 con 0x55d535851730 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 10 client.17457 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 20 client.17457 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d53586bdd0 0x55d53586e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 4 client.17457 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5354b97e0 con 0x55d53586bdd0 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.577+0000 7f30827fc640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d53586bdd0 0x55d53586e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.578+0000 7f30827fc640 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d53586bdd0 0x55d53586e1f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4155edbfdbac581 server_cookie=9f051e89961de08d in_seq=0 out_seq=0 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.578+0000 7f3062ffd640 10 client.17457 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.578+0000 7f3062ffd640 1 -- 172.21.15.44:0/417187232 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d5354b97e0 con 0x55d53586bdd0 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.578+0000 7f3062ffd640 10 client.17457 handle_command_reply: tid=0 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.578+0000 7f308c654740 1 client.17457 shutdown 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 msgr2=0x55d535858cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d535858cb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3074004620 tx=0x7f3074019af0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 shutdown_connections 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d53586bdd0 0x55d53586e1f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535859250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d535858cb0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 --2- 172.21.15.44:0/417187232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d53585b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 >> 172.21.15.44:0/417187232 conn(0x55d53584d380 msgr2=0x55d5358530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f3062ffd640 0 client.17457 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 shutdown_connections 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 1 -- 172.21.15.44:0/417187232 wait complete. 2024-09-13T17:32:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.579+0000 7f308c654740 20 client.17457 trim_cache size 0 max 16384 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 Processor -- start 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 -- start start 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d535859230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d53584cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535857560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53584b9c0 con 0x55d535851730 2024-09-13T17:32:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d535868ae0 con 0x55d53585bfd0 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.593+0000 7f308c654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d535851fb0 con 0x55d535850e10 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d53584cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d53584cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50368/0 (socket says 172.21.15.44:50368) 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 -- 172.21.15.44:0/1206406546 learned_addr learned my addr 172.21.15.44:0/1206406546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f30827fc640 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d535859230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 -- 172.21.15.44:0/1206406546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 msgr2=0x55d535859230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d535859230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 -- 172.21.15.44:0/1206406546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 msgr2=0x55d535857560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535857560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 -- 172.21.15.44:0/1206406546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d535755790 con 0x55d535851730 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f3082ffd640 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d53584cd00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3074011d10 tx=0x7f3074005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11029e7ac1db13a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f306a7fc640 1 -- 172.21.15.44:0/1206406546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074011dc0 con 0x55d535851730 2024-09-13T17:32:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f306a7fc640 1 -- 172.21.15.44:0/1206406546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3074018390 con 0x55d535851730 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.594+0000 7f306a7fc640 1 -- 172.21.15.44:0/1206406546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074018800 con 0x55d535851730 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 -- 172.21.15.44:0/1206406546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 msgr2=0x55d53584cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d53584cd00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3074011d10 tx=0x7f3074005970 comp rx=0 tx=0).stop 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 -- 172.21.15.44:0/1206406546 shutdown_connections 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d535857560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535851730 0x55d53584cd00 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 --2- 172.21.15.44:0/1206406546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535850e10 0x55d535859230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 -- 172.21.15.44:0/1206406546 >> 172.21.15.44:0/1206406546 conn(0x55d53584d380 msgr2=0x55d535853f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 -- 172.21.15.44:0/1206406546 shutdown_connections 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 -- 172.21.15.44:0/1206406546 wait complete. 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 -- start start 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.595+0000 7f308c654740 1 Processor -- start 2024-09-13T17:32:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d535862ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535851730 0x55d5358631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d53585f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53585f950 con 0x55d535850e10 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53585fac0 con 0x55d53585bfd0 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53585fc80 con 0x55d535851730 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f308c654740 10 client.0 _ll_register_callbacks cb 0x7f307800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d535862ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d535862ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50370/0 (socket says 172.21.15.44:50370) 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 -- 172.21.15.44:0/2861466032 learned_addr learned my addr 172.21.15.44:0/2861466032 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535851730 0x55d5358631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d53585f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 -- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535851730 msgr2=0x55d5358631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535851730 0x55d5358631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 -- 172.21.15.44:0/2861466032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 msgr2=0x55d53585f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d53585f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 -- 172.21.15.44:0/2861466032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5357b3620 con 0x55d535850e10 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f30827fc640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d535862ca0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x55d53584e700 tx=0x7f3078004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b308f4e557c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f3062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f3062ffd640 1 -- 172.21.15.44:0/2861466032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078017040 con 0x55d535850e10 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.596+0000 7f3062ffd640 1 -- 172.21.15.44:0/2861466032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f3078012db0 con 0x55d535850e10 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3074009e70 con 0x55d535850e10 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f3062ffd640 1 -- 172.21.15.44:0/2861466032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3078021750 con 0x55d535850e10 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 10 client.17463 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d53577aea0 con 0x55d535850e10 2024-09-13T17:32:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f3062ffd640 1 -- 172.21.15.44:0/2861466032 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3074009e70 con 0x55d535850e10 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f3062ffd640 1 -- 172.21.15.44:0/2861466032 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3078010c60 con 0x55d535850e10 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 20 client.17463 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5358c35e0 0x55d5358c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 4 client.17463 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d53550ce30 con 0x55d5358c35e0 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.597+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5358c35e0 0x55d5358c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5358c35e0 0x55d5358c5a00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c19f7753122b4e17 server_cookie=5b2c7ee79fcecb2f in_seq=0 out_seq=0 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f3062ffd640 10 client.17463 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f3062ffd640 1 -- 172.21.15.44:0/2861466032 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d53550ce30 con 0x55d5358c35e0 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f3062ffd640 10 client.17463 handle_command_reply: tid=0 2024-09-13T17:32:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 client.17463 shutdown 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 msgr2=0x55d535862ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d535862ca0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x55d53584e700 tx=0x7f3078004730 comp rx=0 tx=0).stop 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 shutdown_connections 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5358c35e0 0x55d5358c5a00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d535851730 0x55d5358631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d53585bfd0 0x55d53585f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f3062ffd640 0 client.17463 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 --2- 172.21.15.44:0/2861466032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d535850e10 0x55d535862ca0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 >> 172.21.15.44:0/2861466032 conn(0x55d53584d380 msgr2=0x55d535857520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 shutdown_connections 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 1 -- 172.21.15.44:0/2861466032 wait complete. 2024-09-13T17:32:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.598+0000 7f308c654740 20 client.17463 trim_cache size 0 max 16384 2024-09-13T17:32:45.608 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:32:45.608 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:32:45.608 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:32:45.608 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 msgr2=0x7f306c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f307c0b7be0 tx=0x7f307c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 msgr2=0x7f308419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 0x7f308419af60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f307c004af0 tx=0x7f307c030e50 comp rx=0 tx=0).stop 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 shutdown_connections 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f30841966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aab0 0x7f308419af60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 --2- 172.21.15.44:0/1021741512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 >> 172.21.15.44:0/1021741512 conn(0x7f30840fb9e0 msgr2=0x7f30840fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 shutdown_connections 2024-09-13T17:32:45.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.599+0000 7f308985a640 1 -- 172.21.15.44:0/1021741512 wait complete. 2024-09-13T17:32:45.612 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:32:45.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:32:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 -- 172.21.15.44:0/13887900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 msgr2=0x7f7ea0100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 --2- 172.21.15.44:0/13887900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea0100370 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009930 tx=0x7f7e9c02a420 comp rx=0 tx=0).stop 2024-09-13T17:32:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 -- 172.21.15.44:0/13887900 shutdown_connections 2024-09-13T17:32:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 --2- 172.21.15.44:0/13887900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0100940 0x7f7ea0104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 --2- 172.21.15.44:0/13887900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea01050e0 0x7f7ea010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 --2- 172.21.15.44:0/13887900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea0100370 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 -- 172.21.15.44:0/13887900 >> 172.21.15.44:0/13887900 conn(0x7f7ea00fba20 msgr2=0x7f7ea00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 -- 172.21.15.44:0/13887900 shutdown_connections 2024-09-13T17:32:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.713+0000 7f7ea819c640 1 -- 172.21.15.44:0/13887900 wait complete. 2024-09-13T17:32:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 Processor -- start 2024-09-13T17:32:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 -- start start 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea01050e0 0x7f7ea0072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0073230 0x7f7ea01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01a49a0 con 0x7f7ea00fff90 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01a4b10 con 0x7f7ea01050e0 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea819c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01a4cd0 con 0x7f7ea0073230 2024-09-13T17:32:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea00727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50390/0 (socket says 172.21.15.44:50390) 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 -- 172.21.15.44:0/855787439 learned_addr learned my addr 172.21.15.44:0/855787439 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea6739640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0073230 0x7f7ea01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5737640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea01050e0 0x7f7ea0072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 -- 172.21.15.44:0/855787439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0073230 msgr2=0x7f7ea01a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0073230 0x7f7ea01a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 -- 172.21.15.44:0/855787439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea01050e0 msgr2=0x7f7ea0072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea01050e0 0x7f7ea0072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.714+0000 7f7ea5f38640 1 -- 172.21.15.44:0/855787439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009840 con 0x7f7ea00fff90 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.715+0000 7f7ea5f38640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea00727b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005ab0 tx=0x7f7e9c0329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.715+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c032df0 con 0x7f7ea00fff90 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.715+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e9c005490 con 0x7f7ea00fff90 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.715+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c009560 con 0x7f7ea00fff90 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.715+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0035f0 con 0x7f7ea00fff90 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.715+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea01a5260 con 0x7f7ea00fff90 2024-09-13T17:32:45.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.716+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea0101460 con 0x7f7ea00fff90 2024-09-13T17:32:45.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.717+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7e9c0046c0 con 0x7f7ea00fff90 2024-09-13T17:32:45.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.718+0000 7f7e96ffd640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a12b0 0x7f7e8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:45.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.718+0000 7f7ea5737640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a12b0 0x7f7e8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:45.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.718+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7e9c0eda20 con 0x7f7ea00fff90 2024-09-13T17:32:45.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.718+0000 7f7ea5737640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a12b0 0x7f7e8c0a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0073f30 tx=0x7f7e90005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:45.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.722+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7e9c0b6af0 con 0x7f7ea00fff90 2024-09-13T17:32:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.936+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ea0100370 con 0x7f7ea00fff90 2024-09-13T17:32:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.936+0000 7f7e96ffd640 1 -- 172.21.15.44:0/855787439 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f7e9c0ba110 con 0x7f7ea00fff90 2024-09-13T17:32:45.938 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:32:45.939 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:32:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a12b0 msgr2=0x7f7e8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a12b0 0x7f7e8c0a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0073f30 tx=0x7f7e90005f90 comp rx=0 tx=0).stop 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 msgr2=0x7f7ea00727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea00727b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005ab0 tx=0x7f7e9c0329f0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 shutdown_connections 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a12b0 0x7f7e8c0a3770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea0073230 0x7f7ea01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea01050e0 0x7f7ea0072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 --2- 172.21.15.44:0/855787439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea00fff90 0x7f7ea00727b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:45.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 >> 172.21.15.44:0/855787439 conn(0x7f7ea00fba20 msgr2=0x7f7ea00fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:45.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 shutdown_connections 2024-09-13T17:32:45.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:45.939+0000 7f7ea819c640 1 -- 172.21.15.44:0/855787439 wait complete. 2024-09-13T17:32:45.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:32:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 -- 172.21.15.44:0/1143788927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c740ff830 msgr2=0x7f7c74103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 --2- 172.21.15.44:0/1143788927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c740ff830 0x7f7c74103870 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009930 tx=0x7f7c6002a420 comp rx=0 tx=0).stop 2024-09-13T17:32:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 -- 172.21.15.44:0/1143788927 shutdown_connections 2024-09-13T17:32:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 --2- 172.21.15.44:0/1143788927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74103fd0 0x7f7c7410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 --2- 172.21.15.44:0/1143788927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c740ff830 0x7f7c74103870 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 --2- 172.21.15.44:0/1143788927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740fee80 0x7f7c740ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 -- 172.21.15.44:0/1143788927 >> 172.21.15.44:0/1143788927 conn(0x7f7c740fa930 msgr2=0x7f7c740fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 -- 172.21.15.44:0/1143788927 shutdown_connections 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.054+0000 7f7c7b108640 1 -- 172.21.15.44:0/1143788927 wait complete. 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 Processor -- start 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 -- start start 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c740fee80 0x7f7c74100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 0x7f7c74100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c74101390 0x7f7c741a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741a49f0 con 0x7f7c74103fd0 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741a4b60 con 0x7f7c740fee80 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c7b108640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741a4d20 con 0x7f7c74101390 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 0x7f7c74100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 0x7f7c74100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50418/0 (socket says 172.21.15.44:50418) 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 -- 172.21.15.44:0/2952503346 learned_addr learned my addr 172.21.15.44:0/2952503346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c796a5640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c74101390 0x7f7c741a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c740fee80 0x7f7c74100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 -- 172.21.15.44:0/2952503346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c74101390 msgr2=0x7f7c741a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c74101390 0x7f7c741a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 -- 172.21.15.44:0/2952503346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c740fee80 msgr2=0x7f7c74100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c740fee80 0x7f7c74100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.055+0000 7f7c73fff640 1 -- 172.21.15.44:0/2952503346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60009560 con 0x7f7c74103fd0 2024-09-13T17:32:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.056+0000 7f7c73fff640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 0x7f7c74100e50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005ab0 tx=0x7f7c60032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.056+0000 7f7c71ffb640 1 -- 172.21.15.44:0/2952503346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60032e10 con 0x7f7c74103fd0 2024-09-13T17:32:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.056+0000 7f7c71ffb640 1 -- 172.21.15.44:0/2952503346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c600046c0 con 0x7f7c74103fd0 2024-09-13T17:32:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.056+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c741a4fa0 con 0x7f7c74103fd0 2024-09-13T17:32:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.056+0000 7f7c71ffb640 1 -- 172.21.15.44:0/2952503346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60003850 con 0x7f7c74103fd0 2024-09-13T17:32:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.056+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741a53c0 con 0x7f7c74103fd0 2024-09-13T17:32:46.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.058+0000 7f7c71ffb640 1 -- 172.21.15.44:0/2952503346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7c60004830 con 0x7f7c74103fd0 2024-09-13T17:32:46.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.058+0000 7f7c7de2c740 1 Processor -- start 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.058+0000 7f7c7de2c740 1 -- start start 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e5790 0x561bdd3e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd34fcc0 con 0x561bdd3e60b0 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd34d450 con 0x561bdd3f0950 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd348d40 con 0x561bdd3e5790 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50422/0 (socket says 172.21.15.44:50422) 2024-09-13T17:32:46.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 -- 172.21.15.44:0/3014774790 learned_addr learned my addr 172.21.15.44:0/3014774790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c796a5640 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e5790 0x561bdd3e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c73fff640 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 -- 172.21.15.44:0/3014774790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e5790 msgr2=0x561bdd3e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e5790 0x561bdd3e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c71ffb640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7c640a1230 0x7f7c640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 -- 172.21.15.44:0/3014774790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 msgr2=0x561bdd3f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 -- 172.21.15.44:0/3014774790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bdd310620 con 0x561bdd3e60b0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c73fff640 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c71ffb640 1 -- 172.21.15.44:0/2952503346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7c600edb30 con 0x7f7c74103fd0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c796a5640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7c640a1230 0x7f7c640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.059+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f0410 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800b760 tx=0x7f7c6800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5504835de66b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c5effd640 1 -- 172.21.15.44:0/3014774790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003dd0 con 0x561bdd3e60b0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c5effd640 1 -- 172.21.15.44:0/3014774790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c6800be70 con 0x561bdd3e60b0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c5effd640 1 -- 172.21.15.44:0/3014774790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68004e40 con 0x561bdd3e60b0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3014774790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 msgr2=0x561bdd3f0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c796a5640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7c640a1230 0x7f7c640a36f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009fb0 tx=0x7f7c6c00d040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f0410 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800b760 tx=0x7f7c6800bc20 comp rx=0 tx=0).stop 2024-09-13T17:32:46.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3014774790 shutdown_connections 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f0410 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3014774790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e5790 0x561bdd3e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3014774790 >> 172.21.15.44:0/3014774790 conn(0x561bdd3e1d00 msgr2=0x561bdd3e20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3014774790 shutdown_connections 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.060+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3014774790 wait complete. 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 -- start start 2024-09-13T17:32:46.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 Processor -- start 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e60b0 0x561bdd3ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd307180 con 0x561bdd3e5790 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd021430 con 0x561bdd3f0950 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd2cc5e0 con 0x561bdd3e60b0 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c7de2c740 10 client.0 _ll_register_callbacks cb 0x561bdd338aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c78ea4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c78ea4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50430/0 (socket says 172.21.15.44:50430) 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c78ea4640 1 -- 172.21.15.44:0/497690285 learned_addr learned my addr 172.21.15.44:0/497690285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c73fff640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e60b0 0x561bdd3ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.061+0000 7f7c796a5640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c78ea4640 1 -- 172.21.15.44:0/497690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e60b0 msgr2=0x561bdd3ed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e60b0 0x561bdd3ed2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c78ea4640 1 -- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 msgr2=0x561bdd3ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c78ea4640 1 -- 172.21.15.44:0/497690285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c68004370 con 0x561bdd3e5790 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c796a5640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3eff30 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7c680039c0 tx=0x7f7c68003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64e35dc582e817e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c53fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c53fff640 1 -- 172.21.15.44:0/497690285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68004800 con 0x561bdd3e5790 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c53fff640 1 -- 172.21.15.44:0/497690285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7c68011650 con 0x561bdd3e5790 2024-09-13T17:32:46.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bdd372b80 con 0x561bdd3e5790 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c53fff640 1 -- 172.21.15.44:0/497690285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c68011810 con 0x561bdd3e5790 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c7de2c740 10 client.17481 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561bdd3473c0 con 0x561bdd3e5790 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.062+0000 7f7c53fff640 1 -- 172.21.15.44:0/497690285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561bdd372b80 con 0x561bdd3e5790 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c53fff640 1 -- 172.21.15.44:0/497690285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7c68025970 con 0x561bdd3e5790 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c7de2c740 10 client.17481 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c7de2c740 10 client.17481 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c7de2c740 20 client.17481 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4001f0 0x561bdd4025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c7de2c740 4 client.17481 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bdd33b1f0 con 0x561bdd4001f0 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c73fff640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4001f0 0x561bdd4025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c73fff640 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4001f0 0x561bdd4025e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bd9041299d144b6 server_cookie=146c00402b592919 in_seq=0 out_seq=0 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.063+0000 7f7c53fff640 10 client.17481 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c53fff640 1 -- 172.21.15.44:0/497690285 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561bdd33b1f0 con 0x561bdd4001f0 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c53fff640 10 client.17481 handle_command_reply: tid=0 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 client.17481 shutdown 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 msgr2=0x561bdd3eff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3eff30 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7c680039c0 tx=0x7f7c68003e00 comp rx=0 tx=0).stop 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 shutdown_connections 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4001f0 0x561bdd4025e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3e60b0 0x561bdd3ed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3f0950 0x561bdd3ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/497690285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3eff30 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 >> 172.21.15.44:0/497690285 conn(0x561bdd3e1d00 msgr2=0x561bdd3e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c53fff640 0 client.17481 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:46.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 shutdown_connections 2024-09-13T17:32:46.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 1 -- 172.21.15.44:0/497690285 wait complete. 2024-09-13T17:32:46.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.064+0000 7f7c7de2c740 20 client.17481 trim_cache size 0 max 16384 2024-09-13T17:32:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.075+0000 7f7c7de2c740 1 Processor -- start 2024-09-13T17:32:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.075+0000 7f7c7de2c740 1 -- start start 2024-09-13T17:32:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e5790 0x561bdd3e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd3ee730 con 0x561bdd3e60b0 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd3e6620 con 0x561bdd3e5790 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd3ee3d0 con 0x561bdd3f0950 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50434/0 (socket says 172.21.15.44:50434) 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 -- 172.21.15.44:0/3795857391 learned_addr learned my addr 172.21.15.44:0/3795857391 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 -- 172.21.15.44:0/3795857391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 msgr2=0x561bdd3f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 -- 172.21.15.44:0/3795857391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e5790 msgr2=0x561bdd3e16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e5790 0x561bdd3e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e5790 0x561bdd3e16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 -- 172.21.15.44:0/3795857391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bdd2e9770 con 0x561bdd3e60b0 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e5790 0x561bdd3e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c73fff640 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7610 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7c60000c00 tx=0x7f7c600f0070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91c0e515d742d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c5f7fe640 1 -- 172.21.15.44:0/3795857391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600ba9d0 con 0x561bdd3e60b0 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c5f7fe640 1 -- 172.21.15.44:0/3795857391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c600be750 con 0x561bdd3e60b0 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.076+0000 7f7c5f7fe640 1 -- 172.21.15.44:0/3795857391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600b8e90 con 0x561bdd3e60b0 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3795857391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 msgr2=0x561bdd3f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7610 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7c60000c00 tx=0x7f7c600f0070 comp rx=0 tx=0).stop 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3795857391 shutdown_connections 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7610 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/3795857391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e5790 0x561bdd3e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3795857391 >> 172.21.15.44:0/3795857391 conn(0x561bdd3e1d00 msgr2=0x561bdd406f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3795857391 shutdown_connections 2024-09-13T17:32:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 -- 172.21.15.44:0/3795857391 wait complete. 2024-09-13T17:32:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 -- start start 2024-09-13T17:32:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.077+0000 7f7c7de2c740 1 Processor -- start 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd3f44b0 con 0x561bdd3e5790 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd3f4620 con 0x561bdd3e60b0 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdd3f47e0 con 0x561bdd3f0950 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c7de2c740 10 client.0 _ll_register_callbacks cb 0x7f7c68004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50442/0 (socket says 172.21.15.44:50442) 2024-09-13T17:32:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 -- 172.21.15.44:0/1955352436 learned_addr learned my addr 172.21.15.44:0/1955352436 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c796a5640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c73fff640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 -- 172.21.15.44:0/1955352436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 msgr2=0x561bdd3f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 -- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e60b0 msgr2=0x561bdd3f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 -- 172.21.15.44:0/1955352436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741a53c0 con 0x561bdd3e5790 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c78ea4640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3f7600 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7c68004340 tx=0x7f7c680052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad6c2a5a7d64c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.078+0000 7f7c53fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c53fff640 1 -- 172.21.15.44:0/1955352436 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68025ea0 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c53fff640 1 -- 172.21.15.44:0/1955352436 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f7c68000e40 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c6800be70 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c53fff640 1 -- 172.21.15.44:0/1955352436 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6800b4a0 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 10 client.17487 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561bdd30ee80 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c53fff640 1 -- 172.21.15.44:0/1955352436 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c6800be70 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c53fff640 1 -- 172.21.15.44:0/1955352436 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7c6803c070 con 0x561bdd3e5790 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 10 client.17487 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 10 client.17487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:32:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 20 client.17487 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4576c0 0x561bdd459ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 4 client.17487 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561bdd03c4e0 con 0x561bdd4576c0 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.079+0000 7f7c73fff640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4576c0 0x561bdd459ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c53fff640 10 client.17487 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c73fff640 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4576c0 0x561bdd459ae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ff8dc9c96d47ed5 server_cookie=cd415ad2c6615b60 in_seq=0 out_seq=0 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c53fff640 1 -- 172.21.15.44:0/1955352436 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561bdd03c4e0 con 0x561bdd4576c0 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c53fff640 10 client.17487 handle_command_reply: tid=0 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 client.17487 shutdown 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 msgr2=0x561bdd3f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3f7600 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7c68004340 tx=0x7f7c680052f0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 shutdown_connections 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561bdd4576c0 0x561bdd459ae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bdd3f0950 0x561bdd3f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561bdd3e60b0 0x561bdd3f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 --2- 172.21.15.44:0/1955352436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bdd3e5790 0x561bdd3f7600 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c53fff640 0 client.17487 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 >> 172.21.15.44:0/1955352436 conn(0x561bdd3e1d00 msgr2=0x561bdd3eba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 shutdown_connections 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 1 -- 172.21.15.44:0/1955352436 wait complete. 2024-09-13T17:32:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.080+0000 7f7c7de2c740 20 client.17487 trim_cache size 0 max 16384 2024-09-13T17:32:46.091 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7c640a1230 msgr2=0x7f7c640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7c640a1230 0x7f7c640a36f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009fb0 tx=0x7f7c6c00d040 comp rx=0 tx=0).stop 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 msgr2=0x7f7c74100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 0x7f7c74100e50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005ab0 tx=0x7f7c60032c20 comp rx=0 tx=0).stop 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 shutdown_connections 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7c640a1230 0x7f7c640a36f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c74101390 0x7f7c741a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74103fd0 0x7f7c74100e50 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 --2- 172.21.15.44:0/2952503346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c740fee80 0x7f7c74100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 >> 172.21.15.44:0/2952503346 conn(0x7f7c740fa930 msgr2=0x7f7c740689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 shutdown_connections 2024-09-13T17:32:46.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.081+0000 7f7c7b108640 1 -- 172.21.15.44:0/2952503346 wait complete. 2024-09-13T17:32:46.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:32:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 -- 172.21.15.44:0/269930493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0fee80 msgr2=0x7f4a9c0ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/269930493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0fee80 0x7f4a9c0ff260 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009930 tx=0x7f4a9802a420 comp rx=0 tx=0).stop 2024-09-13T17:32:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 -- 172.21.15.44:0/269930493 shutdown_connections 2024-09-13T17:32:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/269930493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/269930493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c0ff830 0x7f4a9c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/269930493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0fee80 0x7f4a9c0ff260 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 -- 172.21.15.44:0/269930493 >> 172.21.15.44:0/269930493 conn(0x7f4a9c0fa8d0 msgr2=0x7f4a9c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 -- 172.21.15.44:0/269930493 shutdown_connections 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.196+0000 7f4aa3fce640 1 -- 172.21.15.44:0/269930493 wait complete. 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 Processor -- start 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 -- start start 2024-09-13T17:32:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0fee80 0x7f4a9c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c19ab10 0x7f4a9c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19b500 con 0x7f4a9c103fd0 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19b670 con 0x7f4a9c19ab10 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa3fce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19b7e0 con 0x7f4a9c0fee80 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50470/0 (socket says 172.21.15.44:50470) 2024-09-13T17:32:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 -- 172.21.15.44:0/3339454442 learned_addr learned my addr 172.21.15.44:0/3339454442 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0fee80 0x7f4a9c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa256b640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c19ab10 0x7f4a9c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 -- 172.21.15.44:0/3339454442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0fee80 msgr2=0x7f4a9c1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0fee80 0x7f4a9c1961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 -- 172.21.15.44:0/3339454442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c19ab10 msgr2=0x7f4a9c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c19ab10 0x7f4a9c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa1569640 1 -- 172.21.15.44:0/3339454442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98009560 con 0x7f4a9c103fd0 2024-09-13T17:32:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.197+0000 7f4aa256b640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c19ab10 0x7f4a9c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.198+0000 7f4aa1569640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c196720 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c005fd0 tx=0x7f4a8c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.198+0000 7f4a92ffd640 1 -- 172.21.15.44:0/3339454442 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009c30 con 0x7f4a9c103fd0 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.198+0000 7f4a92ffd640 1 -- 172.21.15.44:0/3339454442 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a8c0044b0 con 0x7f4a9c103fd0 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.198+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c1a2dc0 con 0x7f4a9c103fd0 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.198+0000 7f4a92ffd640 1 -- 172.21.15.44:0/3339454442 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c003da0 con 0x7f4a9c103fd0 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.198+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c1a3260 con 0x7f4a9c103fd0 2024-09-13T17:32:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.200+0000 7f4a92ffd640 1 -- 172.21.15.44:0/3339454442 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4a8c003f00 con 0x7f4a9c103fd0 2024-09-13T17:32:46.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.200+0000 7f4aa6dcd740 1 Processor -- start 2024-09-13T17:32:46.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 -- start start 2024-09-13T17:32:46.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed10da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed11c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed11f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed077cc0 con 0x564bed10d690 2024-09-13T17:32:46.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed075450 con 0x564bed11cd60 2024-09-13T17:32:46.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed070d40 con 0x564bed10dfb0 2024-09-13T17:32:46.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4a92ffd640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4a880a1280 0x7f4a880a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed11c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa256b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed11c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:34942/0 (socket says 172.21.15.44:34942) 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1175025287 learned_addr learned my addr 172.21.15.44:0/1175025287 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa256b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed10da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50478/0 (socket says 172.21.15.44:50478) 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa256b640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4a880a1280 0x7f4a880a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4a92ffd640 1 -- 172.21.15.44:0/3339454442 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f4a8c0ccc50 con 0x7f4a9c103fd0 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1569640 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed11f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1175025287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 msgr2=0x564bed11f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed11f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1175025287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 msgr2=0x564bed10da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed10da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1175025287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bed038620 con 0x564bed10dfb0 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa256b640 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.201+0000 7f4aa1569640 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed11f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:46.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4aa256b640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4a880a1280 0x7f4a880a3740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005c40 tx=0x7f4a94009280 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed11c820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009900 tx=0x7f4a98005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfc054b01924c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4a87fff640 1 -- 172.21.15.44:0/1175025287 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98004230 con 0x564bed10dfb0 2024-09-13T17:32:46.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4a87fff640 1 -- 172.21.15.44:0/1175025287 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a98031df0 con 0x564bed10dfb0 2024-09-13T17:32:46.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4a87fff640 1 -- 172.21.15.44:0/1175025287 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003bb0 con 0x564bed10dfb0 2024-09-13T17:32:46.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1175025287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 msgr2=0x564bed11c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.202+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed11c820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009900 tx=0x7f4a98005c00 comp rx=0 tx=0).stop 2024-09-13T17:32:46.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1175025287 shutdown_connections 2024-09-13T17:32:46.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed11c820 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed11f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1175025287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1175025287 >> 172.21.15.44:0/1175025287 conn(0x564bed109d00 msgr2=0x564bed10a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1175025287 shutdown_connections 2024-09-13T17:32:46.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1175025287 wait complete. 2024-09-13T17:32:46.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- start start 2024-09-13T17:32:46.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 Processor -- start 2024-09-13T17:32:46.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed11c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed119740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed119c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed02f180 con 0x564bed10d690 2024-09-13T17:32:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564becd49430 con 0x564bed11cd60 2024-09-13T17:32:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564becff45e0 con 0x564bed10dfb0 2024-09-13T17:32:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.203+0000 7f4aa6dcd740 10 client.0 _ll_register_callbacks cb 0x564bed060aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed11c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed11c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50480/0 (socket says 172.21.15.44:50480) 2024-09-13T17:32:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/128281541 learned_addr learned my addr 172.21.15.44:0/128281541 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1569640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed119740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa256b640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed119c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 msgr2=0x564bed119740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed119740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/128281541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 msgr2=0x564bed119c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed119c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/128281541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c10c360 con 0x564bed10d690 2024-09-13T17:32:46.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed11c3b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4a980057c0 tx=0x7f4a98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a2fde3a70b886e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4a857fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4a857fa640 1 -- 172.21.15.44:0/128281541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9804b070 con 0x564bed10d690 2024-09-13T17:32:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4a857fa640 1 -- 172.21.15.44:0/128281541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f4a98004030 con 0x564bed10d690 2024-09-13T17:32:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bed09ab80 con 0x564bed10d690 2024-09-13T17:32:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4a857fa640 1 -- 172.21.15.44:0/128281541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a980041f0 con 0x564bed10d690 2024-09-13T17:32:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa6dcd740 10 client.17499 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.204+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564bed06f3c0 con 0x564bed10d690 2024-09-13T17:32:46.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4a857fa640 1 -- 172.21.15.44:0/128281541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564bed09ab80 con 0x564bed10d690 2024-09-13T17:32:46.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4a857fa640 1 -- 172.21.15.44:0/128281541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a98043020 con 0x564bed10d690 2024-09-13T17:32:46.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa6dcd740 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:46.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa6dcd740 10 client.17499 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:32:46.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa6dcd740 20 client.17499 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:46.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed12c4b0 0x564bed12e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa6dcd740 4 client.17499 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bed0631f0 con 0x564bed12c4b0 2024-09-13T17:32:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.205+0000 7f4aa1569640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed12c4b0 0x564bed12e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4a857fa640 10 client.17499 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4aa1569640 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed12c4b0 0x564bed12e8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62759a2ea3e60a2e server_cookie=d8d2da0212857979 in_seq=0 out_seq=0 2024-09-13T17:32:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4a857fa640 1 -- 172.21.15.44:0/128281541 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564bed0631f0 con 0x564bed12c4b0 2024-09-13T17:32:46.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4a857fa640 10 client.17499 handle_command_reply: tid=0 2024-09-13T17:32:46.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4aa6dcd740 1 client.17499 shutdown 2024-09-13T17:32:46.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 msgr2=0x564bed11c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.206+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed11c3b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4a980057c0 tx=0x7f4a98009260 comp rx=0 tx=0).stop 2024-09-13T17:32:46.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 shutdown_connections 2024-09-13T17:32:46.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed12c4b0 0x564bed12e8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed119740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed119c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4a857fa640 0 client.17499 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/128281541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed11c3b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 >> 172.21.15.44:0/128281541 conn(0x564bed109d00 msgr2=0x564bed10fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 shutdown_connections 2024-09-13T17:32:46.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/128281541 wait complete. 2024-09-13T17:32:46.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.207+0000 7f4aa6dcd740 20 client.17499 trim_cache size 0 max 16384 2024-09-13T17:32:46.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 Processor -- start 2024-09-13T17:32:46.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 -- start start 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed10d690 0x564bed114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 0x564bed123a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed11cd60 0x564bed123f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed10ea60 con 0x564bed10dfb0 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed12c2d0 con 0x564bed10d690 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed1043b0 con 0x564bed11cd60 2024-09-13T17:32:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa1569640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 0x564bed123a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa1569640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 0x564bed123a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50492/0 (socket says 172.21.15.44:50492) 2024-09-13T17:32:46.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa1569640 1 -- 172.21.15.44:0/684043274 learned_addr learned my addr 172.21.15.44:0/684043274 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed10d690 0x564bed114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.218+0000 7f4aa256b640 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed11cd60 0x564bed123f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4aa1569640 1 -- 172.21.15.44:0/684043274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed11cd60 msgr2=0x564bed123f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4aa1569640 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed11cd60 0x564bed123f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4aa1569640 1 -- 172.21.15.44:0/684043274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed10d690 msgr2=0x564bed114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4aa1569640 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed10d690 0x564bed114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4aa1569640 1 -- 172.21.15.44:0/684043274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bed011770 con 0x564bed10dfb0 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4aa1569640 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 0x564bed123a50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0ca080 tx=0x7f4a8c095640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe32fcf792362d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4a90ff9640 1 -- 172.21.15.44:0/684043274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0a1e60 con 0x564bed10dfb0 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4a90ff9640 1 -- 172.21.15.44:0/684043274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a8c097b20 con 0x564bed10dfb0 2024-09-13T17:32:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.219+0000 7f4a90ff9640 1 -- 172.21.15.44:0/684043274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0960f0 con 0x564bed10dfb0 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/684043274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 msgr2=0x564bed123a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 0x564bed123a50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0ca080 tx=0x7f4a8c095640 comp rx=0 tx=0).stop 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/684043274 shutdown_connections 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed11cd60 0x564bed123f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10dfb0 0x564bed123a50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/684043274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed10d690 0x564bed114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/684043274 >> 172.21.15.44:0/684043274 conn(0x564bed109d00 msgr2=0x564bed1147c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/684043274 shutdown_connections 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/684043274 wait complete. 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- start start 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 Processor -- start 2024-09-13T17:32:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed120010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed120550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed123380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed1238f0 con 0x564bed10d690 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed123a60 con 0x564bed11cd60 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.220+0000 7f4aa6dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bed123c20 con 0x564bed10dfb0 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa6dcd740 10 client.0 _ll_register_callbacks cb 0x7f4a98005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed120010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed120010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50494/0 (socket says 172.21.15.44:50494) 2024-09-13T17:32:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1863244614 learned_addr learned my addr 172.21.15.44:0/1863244614 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1569640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed120550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa256b640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 msgr2=0x564bed120550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed120550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1863244614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 msgr2=0x564bed123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 -- 172.21.15.44:0/1863244614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c1a3260 con 0x564bed10d690 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa1d6a640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed120010 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009230 tx=0x7f4a9803de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9995fae424f1193 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4a857fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4a857fa640 1 -- 172.21.15.44:0/1863244614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9804b070 con 0x564bed10d690 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4a857fa640 1 -- 172.21.15.44:0/1863244614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f4a98000e40 con 0x564bed10d690 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a98031df0 con 0x564bed10d690 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4a857fa640 1 -- 172.21.15.44:0/1863244614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9803d450 con 0x564bed10d690 2024-09-13T17:32:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa6dcd740 10 client.17505 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.221+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564bed036e80 con 0x564bed10d690 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4a857fa640 1 -- 172.21.15.44:0/1863244614 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a98031df0 con 0x564bed10d690 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4a857fa640 1 -- 172.21.15.44:0/1863244614 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a98043040 con 0x564bed10d690 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa6dcd740 10 client.17505 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa6dcd740 10 client.17505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa6dcd740 20 client.17505 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:46.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed183700 0x564bed185b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa6dcd740 4 client.17505 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564becd644e0 con 0x564bed183700 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa1569640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed183700 0x564bed185b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4a857fa640 10 client.17505 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.222+0000 7f4aa1569640 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed183700 0x564bed185b20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9322b4264a3a2919 server_cookie=d40dc9ad5a337ea8 in_seq=0 out_seq=0 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4a857fa640 1 -- 172.21.15.44:0/1863244614 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564becd644e0 con 0x564bed183700 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4a857fa640 10 client.17505 handle_command_reply: tid=0 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 client.17505 shutdown 2024-09-13T17:32:46.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 msgr2=0x564bed120010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed120010 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009230 tx=0x7f4a9803de80 comp rx=0 tx=0).stop 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 shutdown_connections 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x564bed183700 0x564bed185b20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564bed10dfb0 0x564bed120550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4a857fa640 0 client.17505 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564bed11cd60 0x564bed123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 --2- 172.21.15.44:0/1863244614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bed10d690 0x564bed120010 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 >> 172.21.15.44:0/1863244614 conn(0x564bed109d00 msgr2=0x564bed117ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 shutdown_connections 2024-09-13T17:32:46.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 1 -- 172.21.15.44:0/1863244614 wait complete. 2024-09-13T17:32:46.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa6dcd740 20 client.17505 trim_cache size 0 max 16384 2024-09-13T17:32:46.251 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4a880a1280 msgr2=0x7f4a880a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4a880a1280 0x7f4a880a3740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005c40 tx=0x7f4a94009280 comp rx=0 tx=0).stop 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 msgr2=0x7f4a9c196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c196720 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c005fd0 tx=0x7f4a8c00dde0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 shutdown_connections 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4a880a1280 0x7f4a880a3740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a9c19ab10 0x7f4a9c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c103fd0 0x7f4a9c196720 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 --2- 172.21.15.44:0/3339454442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0fee80 0x7f4a9c1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.223+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 >> 172.21.15.44:0/3339454442 conn(0x7f4a9c0fa8d0 msgr2=0x7f4a9c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.224+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 shutdown_connections 2024-09-13T17:32:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.224+0000 7f4aa3fce640 1 -- 172.21.15.44:0/3339454442 wait complete. 2024-09-13T17:32:46.254 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:45 smithi044 ceph-mon[31175]: pgmap v184: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-13T17:32:46.255 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:45 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/855787439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:32:46.256 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:32:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:45 smithi100 ceph-mon[35055]: pgmap v184: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-13T17:32:46.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:45 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/855787439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:32:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 -- 172.21.15.44:0/3063334245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 msgr2=0x7f010c10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 --2- 172.21.15.44:0/3063334245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10bb20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0104009cd0 tx=0x7f010402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 -- 172.21.15.44:0/3063334245 shutdown_connections 2024-09-13T17:32:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 --2- 172.21.15.44:0/3063334245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10bb20 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 --2- 172.21.15.44:0/3063334245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c0ff830 0x7f010c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 --2- 172.21.15.44:0/3063334245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f010c0fee80 0x7f010c0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 -- 172.21.15.44:0/3063334245 >> 172.21.15.44:0/3063334245 conn(0x7f010c0fa930 msgr2=0x7f010c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 -- 172.21.15.44:0/3063334245 shutdown_connections 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.359+0000 7f0110cb1640 1 -- 172.21.15.44:0/3063334245 wait complete. 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 Processor -- start 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 -- start start 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f010c0fee80 0x7f010c10c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c10ceb0 0x7f010c1a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1a4970 con 0x7f010c103fd0 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1a4ae0 con 0x7f010c10ceb0 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0110cb1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1a4ca0 con 0x7f010c0fee80 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50512/0 (socket says 172.21.15.44:50512) 2024-09-13T17:32:46.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f0103fff640 1 -- 172.21.15.44:0/4021475182 learned_addr learned my addr 172.21.15.44:0/4021475182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f010a59c640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f010c0fee80 0x7f010c10c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.360+0000 7f010ad9d640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c10ceb0 0x7f010c1a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0103fff640 1 -- 172.21.15.44:0/4021475182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f010c0fee80 msgr2=0x7f010c10c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0103fff640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f010c0fee80 0x7f010c10c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0103fff640 1 -- 172.21.15.44:0/4021475182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c10ceb0 msgr2=0x7f010c1a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0103fff640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c10ceb0 0x7f010c1a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0103fff640 1 -- 172.21.15.44:0/4021475182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104009900 con 0x7f010c103fd0 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f010ad9d640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c10ceb0 0x7f010c1a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0103fff640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10c970 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f00f400e9c0 tx=0x7f00f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f01037fe640 1 -- 172.21.15.44:0/4021475182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4009700 con 0x7f010c103fd0 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f01037fe640 1 -- 172.21.15.44:0/4021475182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00f400cd80 con 0x7f010c103fd0 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1a4f80 con 0x7f010c103fd0 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f01037fe640 1 -- 172.21.15.44:0/4021475182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4010430 con 0x7f010c103fd0 2024-09-13T17:32:46.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.361+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1a5450 con 0x7f010c103fd0 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 Processor -- start 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 -- start start 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01037fe640 1 -- 172.21.15.44:0/4021475182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f00f4004eb0 con 0x7f010c103fd0 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb524cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb5259d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb51b6cc0 con 0x564eb524c790 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb51b4450 con 0x564eb524d0b0 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.364+0000 7f01139d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb51afd40 con 0x564eb5257950 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb524cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb524cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50528/0 (socket says 172.21.15.44:50528) 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 -- 172.21.15.44:0/1510191491 learned_addr learned my addr 172.21.15.44:0/1510191491 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010a59c640 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f0103fff640 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb5259d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 -- 172.21.15.44:0/1510191491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 msgr2=0x564eb5259d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb5259d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 -- 172.21.15.44:0/1510191491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 msgr2=0x564eb5257410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 -- 172.21.15.44:0/1510191491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb5177620 con 0x564eb524c790 2024-09-13T17:32:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f0103fff640 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb5259d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010ad9d640 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb524cb70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f01040097b0 tx=0x7f0104031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ef51b2abfe9f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f01037fe640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00f80a12b0 0x7f00f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f0100ff9640 1 -- 172.21.15.44:0/1510191491 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01040307a0 con 0x564eb524c790 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f010a59c640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00f80a12b0 0x7f00f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f01037fe640 1 -- 172.21.15.44:0/4021475182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f00f4014070 con 0x7f010c103fd0 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f0100ff9640 1 -- 172.21.15.44:0/1510191491 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01040315d0 con 0x564eb524c790 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.365+0000 7f0100ff9640 1 -- 172.21.15.44:0/1510191491 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104003e30 con 0x564eb524c790 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 -- 172.21.15.44:0/1510191491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 msgr2=0x564eb524cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f010a59c640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00f80a12b0 0x7f00f80a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x564eb5249b60 tx=0x7f00fc005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb524cb70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f01040097b0 tx=0x7f0104031430 comp rx=0 tx=0).stop 2024-09-13T17:32:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 -- 172.21.15.44:0/1510191491 shutdown_connections 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb5259d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 --2- 172.21.15.44:0/1510191491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb524cb70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 -- 172.21.15.44:0/1510191491 >> 172.21.15.44:0/1510191491 conn(0x564eb5248d00 msgr2=0x564eb52490e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 -- 172.21.15.44:0/1510191491 shutdown_connections 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.366+0000 7f01139d7740 1 -- 172.21.15.44:0/1510191491 wait complete. 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 -- start start 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 Processor -- start 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524c790 0x564eb5256ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb5257950 0x564eb5254440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb516e180 con 0x564eb524d0b0 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4e88430 con 0x564eb5257950 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb51335e0 con 0x564eb524c790 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f01139d7740 10 client.0 _ll_register_callbacks cb 0x564eb519faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50544/0 (socket says 172.21.15.44:50544) 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 -- 172.21.15.44:0/3435159278 learned_addr learned my addr 172.21.15.44:0/3435159278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f010a59c640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524c790 0x564eb5256ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f010ad9d640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb5257950 0x564eb5254440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 -- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524c790 msgr2=0x564eb5256ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524c790 0x564eb5256ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 -- 172.21.15.44:0/3435159278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb5257950 msgr2=0x564eb5254440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb5257950 0x564eb5254440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 -- 172.21.15.44:0/3435159278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c068570 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f0103fff640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 0x564eb5257410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f00f409a160 tx=0x7f00f4099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db406a02d6e19f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.367+0000 7f00f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f00f1ffb640 1 -- 172.21.15.44:0/3435159278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f409a8d0 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f00f1ffb640 1 -- 172.21.15.44:0/3435159278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f00f40b1060 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eb51d9b80 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f00f1ffb640 1 -- 172.21.15.44:0/3435159278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f409a1d0 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 10 client.17517 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564eb51ae3c0 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f00f1ffb640 1 -- 172.21.15.44:0/3435159278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564eb51d9b80 con 0x564eb524d0b0 2024-09-13T17:32:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f00f1ffb640 1 -- 172.21.15.44:0/3435159278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00f407a050 con 0x564eb524d0b0 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 10 client.17517 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 20 client.17517 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb5267200 0x564eb52695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 4 client.17517 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.368+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eb51a21f0 con 0x564eb5267200 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.369+0000 7f010a59c640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb5267200 0x564eb52695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.369+0000 7f010a59c640 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb5267200 0x564eb52695f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df23024dea5009eb server_cookie=ec27003eb869d1ef in_seq=0 out_seq=0 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.369+0000 7f00f1ffb640 10 client.17517 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f00f1ffb640 1 -- 172.21.15.44:0/3435159278 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564eb51a21f0 con 0x564eb5267200 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f00f1ffb640 10 client.17517 handle_command_reply: tid=0 2024-09-13T17:32:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 client.17517 shutdown 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 msgr2=0x564eb5257410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 0x564eb5257410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f00f409a160 tx=0x7f00f4099dd0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 shutdown_connections 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb5267200 0x564eb52695f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb5257950 0x564eb5254440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524d0b0 0x564eb5257410 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f00f1ffb640 0 client.17517 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 --2- 172.21.15.44:0/3435159278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524c790 0x564eb5256ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 >> 172.21.15.44:0/3435159278 conn(0x564eb5248d00 msgr2=0x564eb524ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 shutdown_connections 2024-09-13T17:32:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 1 -- 172.21.15.44:0/3435159278 wait complete. 2024-09-13T17:32:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.370+0000 7f01139d7740 20 client.17517 trim_cache size 0 max 16384 2024-09-13T17:32:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 Processor -- start 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 -- start start 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb5254420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb525e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb525eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb5265b70 con 0x564eb524c790 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb5263fc0 con 0x564eb524d0b0 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f01139d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb5248ad0 con 0x564eb5257950 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f010a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb5254420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f010a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb5254420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50548/0 (socket says 172.21.15.44:50548) 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f010a59c640 1 -- 172.21.15.44:0/4040428523 learned_addr learned my addr 172.21.15.44:0/4040428523 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.381+0000 7f0103fff640 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb525e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010ad9d640 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb525eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010a59c640 1 -- 172.21.15.44:0/4040428523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 msgr2=0x564eb525eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010a59c640 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb525eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010a59c640 1 -- 172.21.15.44:0/4040428523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 msgr2=0x564eb525e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010a59c640 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb525e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010a59c640 1 -- 172.21.15.44:0/4040428523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb5150770 con 0x564eb524c790 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f0103fff640 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb525e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f010a59c640 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb5254420 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f00fc012cb0 tx=0x7f00fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d777bb2890a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f01017fa640 1 -- 172.21.15.44:0/4040428523 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc026040 con 0x564eb524c790 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f01017fa640 1 -- 172.21.15.44:0/4040428523 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00fc004510 con 0x564eb524c790 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f01017fa640 1 -- 172.21.15.44:0/4040428523 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc017070 con 0x564eb524c790 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f01139d7740 1 -- 172.21.15.44:0/4040428523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 msgr2=0x564eb5254420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.382+0000 7f01139d7740 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb5254420 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f00fc012cb0 tx=0x7f00fc017000 comp rx=0 tx=0).stop 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- 172.21.15.44:0/4040428523 shutdown_connections 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb5257950 0x564eb525eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524d0b0 0x564eb525e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 --2- 172.21.15.44:0/4040428523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb524c790 0x564eb5254420 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- 172.21.15.44:0/4040428523 >> 172.21.15.44:0/4040428523 conn(0x564eb5248d00 msgr2=0x564eb526df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- 172.21.15.44:0/4040428523 shutdown_connections 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- 172.21.15.44:0/4040428523 wait complete. 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- start start 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 Processor -- start 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524c790 0x564eb525e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524d0b0 0x564eb525cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 0x564eb525d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb525d6b0 con 0x564eb5257950 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb525d820 con 0x564eb524c790 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb525d990 con 0x564eb524d0b0 2024-09-13T17:32:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f01139d7740 10 client.0 _ll_register_callbacks cb 0x7f00f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f010ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 0x564eb525d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f010ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 0x564eb525d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50560/0 (socket says 172.21.15.44:50560) 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f010ad9d640 1 -- 172.21.15.44:0/1943463969 learned_addr learned my addr 172.21.15.44:0/1943463969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f0103fff640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524d0b0 0x564eb525cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.383+0000 7f010a59c640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524c790 0x564eb525e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f010ad9d640 1 -- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524d0b0 msgr2=0x564eb525cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f010ad9d640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524d0b0 0x564eb525cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f010ad9d640 1 -- 172.21.15.44:0/1943463969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524c790 msgr2=0x564eb525e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f010ad9d640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524c790 0x564eb525e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f010ad9d640 1 -- 172.21.15.44:0/1943463969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc004180 con 0x564eb5257950 2024-09-13T17:32:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f010ad9d640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 0x564eb525d170 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0104009bf0 tx=0x7f0104030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f72bda5ae37da6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f00f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f00f1ffb640 1 -- 172.21.15.44:0/1943463969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010404a070 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f00f1ffb640 1 -- 172.21.15.44:0/1943463969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f0104031980 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01040315d0 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f00f1ffb640 1 -- 172.21.15.44:0/1943463969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0104009540 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 10 client.17523 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564eb5175e80 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f00f1ffb640 1 -- 172.21.15.44:0/1943463969 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01040315d0 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f00f1ffb640 1 -- 172.21.15.44:0/1943463969 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0104055080 con 0x564eb5257950 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 10 client.17523 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 20 client.17523 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb52be8b0 0x564eb52c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 4 client.17523 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.384+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564eb4ea34e0 con 0x564eb52be8b0 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f010a59c640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb52be8b0 0x564eb52c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f00f1ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f010a59c640 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb52be8b0 0x564eb52c0cd0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce15915c9ba720c server_cookie=67d3302794b1d4a9 in_seq=0 out_seq=0 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f00f1ffb640 1 -- 172.21.15.44:0/1943463969 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564eb4ea34e0 con 0x564eb52be8b0 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f00f1ffb640 10 client.17523 handle_command_reply: tid=0 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 client.17523 shutdown 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 msgr2=0x564eb525d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 0x564eb525d170 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0104009bf0 tx=0x7f0104030dd0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 shutdown_connections 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564eb52be8b0 0x564eb52c0cd0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564eb524d0b0 0x564eb525cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f00f1ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564eb5257950 0x564eb525d170 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 --2- 172.21.15.44:0/1943463969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb524c790 0x564eb525e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.385+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 >> 172.21.15.44:0/1943463969 conn(0x564eb5248d00 msgr2=0x564eb5252a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 shutdown_connections 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f01139d7740 1 -- 172.21.15.44:0/1943463969 wait complete. 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f01139d7740 20 client.17523 trim_cache size 0 max 16384 2024-09-13T17:32:46.397 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00f80a12b0 msgr2=0x7f00f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00f80a12b0 0x7f00f80a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x564eb5249b60 tx=0x7f00fc005f90 comp rx=0 tx=0).stop 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 msgr2=0x7f010c10c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10c970 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f00f400e9c0 tx=0x7f00f400ee80 comp rx=0 tx=0).stop 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 shutdown_connections 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00f80a12b0 0x7f00f80a3770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c10ceb0 0x7f010c1a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c103fd0 0x7f010c10c970 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 --2- 172.21.15.44:0/4021475182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f010c0fee80 0x7f010c10c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 >> 172.21.15.44:0/4021475182 conn(0x7f010c0fa930 msgr2=0x7f010c0fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 shutdown_connections 2024-09-13T17:32:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:46.386+0000 7f0110cb1640 1 -- 172.21.15.44:0/4021475182 wait complete. 2024-09-13T17:32:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:45 smithi070 ceph-mon[34502]: pgmap v184: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-13T17:32:46.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:45 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/855787439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:32:47.401 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:32:47.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:32:47.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 -- 172.21.15.44:0/957237622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320100f40 msgr2=0x7f4320101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 --2- 172.21.15.44:0/957237622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320100f40 0x7f4320101320 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4310009900 tx=0x7f431002a410 comp rx=0 tx=0).stop 2024-09-13T17:32:47.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 -- 172.21.15.44:0/957237622 shutdown_connections 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 --2- 172.21.15.44:0/957237622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4320101860 0x7f4320109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 --2- 172.21.15.44:0/957237622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432010a310 0x7f432010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 --2- 172.21.15.44:0/957237622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320100f40 0x7f4320101320 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 -- 172.21.15.44:0/957237622 >> 172.21.15.44:0/957237622 conn(0x7f43200fa930 msgr2=0x7f43200fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 -- 172.21.15.44:0/957237622 shutdown_connections 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.519+0000 7f4325a27640 1 -- 172.21.15.44:0/957237622 wait complete. 2024-09-13T17:32:47.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 Processor -- start 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 -- start start 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4320100f40 0x7f4320196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 0x7f4320196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432010a310 0x7f432019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432019b050 con 0x7f4320101860 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432019b1c0 con 0x7f432010a310 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f4325a27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432019b380 con 0x7f4320100f40 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f431e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 0x7f4320196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f431effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4320100f40 0x7f4320196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f431e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 0x7f4320196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50584/0 (socket says 172.21.15.44:50584) 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f431e7fc640 1 -- 172.21.15.44:0/2121863157 learned_addr learned my addr 172.21.15.44:0/2121863157 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.520+0000 7f431f7fe640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432010a310 0x7f432019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f431e7fc640 1 -- 172.21.15.44:0/2121863157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4320100f40 msgr2=0x7f4320196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f431e7fc640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4320100f40 0x7f4320196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f431e7fc640 1 -- 172.21.15.44:0/2121863157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432010a310 msgr2=0x7f432019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f431e7fc640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432010a310 0x7f432019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f431e7fc640 1 -- 172.21.15.44:0/2121863157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310009560 con 0x7f4320101860 2024-09-13T17:32:47.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f431e7fc640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 0x7f4320196660 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f431400d770 tx=0x7f431400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f4324a25640 1 -- 172.21.15.44:0/2121863157 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314009c30 con 0x7f4320101860 2024-09-13T17:32:47.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432019b660 con 0x7f4320101860 2024-09-13T17:32:47.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f4324a25640 1 -- 172.21.15.44:0/2121863157 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f431400de80 con 0x7f4320101860 2024-09-13T17:32:47.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f4324a25640 1 -- 172.21.15.44:0/2121863157 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314003d40 con 0x7f4320101860 2024-09-13T17:32:47.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.521+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43201a2ee0 con 0x7f4320101860 2024-09-13T17:32:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4324a25640 1 -- 172.21.15.44:0/2121863157 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4314003ea0 con 0x7f4320101860 2024-09-13T17:32:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4328824740 1 Processor -- start 2024-09-13T17:32:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4328824740 1 -- start start 2024-09-13T17:32:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4328824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e1f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4328824740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 0x56412e204440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4328824740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e204980 0x56412e206d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.524+0000 7f4328824740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e164c60 con 0x56412e1fa0e0 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f4328824740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e1623f0 con 0x56412e204980 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f4328824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e15dce0 con 0x56412e1f97c0 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 0x56412e204440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 0x56412e204440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50590/0 (socket says 172.21.15.44:50590) 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 -- 172.21.15.44:0/791869691 learned_addr learned my addr 172.21.15.44:0/791869691 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431f7fe640 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e1f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f4324a25640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43080a1200 0x7f43080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 -- 172.21.15.44:0/791869691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 msgr2=0x56412e1f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431e7fc640 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e204980 0x56412e206d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e1f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 -- 172.21.15.44:0/791869691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e204980 msgr2=0x56412e206d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e204980 0x56412e206d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 -- 172.21.15.44:0/791869691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56412e119970 con 0x56412e1fa0e0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431e7fc640 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e204980 0x56412e206d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431f7fe640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43080a1200 0x7f43080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f4324a25640 1 -- 172.21.15.44:0/2121863157 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f43140cc8e0 con 0x7f4320101860 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431effd640 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 0x56412e204440 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f431002a930 tx=0x7f4310031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eed67098c5b5f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f431f7fe640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43080a1200 0x7f43080a36c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4318007ec0 tx=0x7f4318004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.525+0000 7f4305ffb640 1 -- 172.21.15.44:0/791869691 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310004300 con 0x56412e1fa0e0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4305ffb640 1 -- 172.21.15.44:0/791869691 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4310004460 con 0x56412e1fa0e0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4305ffb640 1 -- 172.21.15.44:0/791869691 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310003c70 con 0x56412e1fa0e0 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 -- 172.21.15.44:0/791869691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 msgr2=0x56412e204440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 0x56412e204440 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f431002a930 tx=0x7f4310031670 comp rx=0 tx=0).stop 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 -- 172.21.15.44:0/791869691 shutdown_connections 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e204980 0x56412e206d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1fa0e0 0x56412e204440 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 --2- 172.21.15.44:0/791869691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e1f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 -- 172.21.15.44:0/791869691 >> 172.21.15.44:0/791869691 conn(0x56412e1f5d50 msgr2=0x56412e1f6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 -- 172.21.15.44:0/791869691 shutdown_connections 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.526+0000 7f4328824740 1 -- 172.21.15.44:0/791869691 wait complete. 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 -- start start 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 Processor -- start 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e204410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e2016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 0x56412e201e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e11bf80 con 0x56412e204980 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e11bc20 con 0x56412e1fa0e0 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e0e8ab0 con 0x56412e1f97c0 2024-09-13T17:32:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f4328824740 10 client.0 _ll_register_callbacks cb 0x56412e157350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f431f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 0x56412e201e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f431f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 0x56412e201e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50606/0 (socket says 172.21.15.44:50606) 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f431f7fe640 1 -- 172.21.15.44:0/1439794524 learned_addr learned my addr 172.21.15.44:0/1439794524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.527+0000 7f431e7fc640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e2016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431effd640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e204410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431f7fe640 1 -- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 msgr2=0x56412e204410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431f7fe640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e204410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431f7fe640 1 -- 172.21.15.44:0/1439794524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 msgr2=0x56412e2016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431f7fe640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e2016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431f7fe640 1 -- 172.21.15.44:0/1439794524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432010d430 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431effd640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e204410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f431f7fe640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 0x56412e201e40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f43180077f0 tx=0x7f4318004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f80c2f0defb04d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f42feffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f42feffd640 1 -- 172.21.15.44:0/1439794524 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318027670 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f42feffd640 1 -- 172.21.15.44:0/1439794524 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f43180277d0 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56412de51590 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f42feffd640 1 -- 172.21.15.44:0/1439794524 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4318007ba0 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f4328824740 10 client.17535 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56412e15c360 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.528+0000 7f42feffd640 1 -- 172.21.15.44:0/1439794524 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56412de51590 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f42feffd640 1 -- 172.21.15.44:0/1439794524 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f431801eb00 con 0x56412e204980 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f4328824740 10 client.17535 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f4328824740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f4328824740 20 client.17535 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f4328824740 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e214720 0x56412e216b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f4328824740 4 client.17535 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56412de62860 con 0x56412e214720 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f431effd640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e214720 0x56412e216b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f431effd640 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e214720 0x56412e216b10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5dda3a39cd15336 server_cookie=22d96bdbd50b26fe in_seq=0 out_seq=0 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.529+0000 7f42feffd640 10 client.17535 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f42feffd640 1 -- 172.21.15.44:0/1439794524 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56412de62860 con 0x56412e214720 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f42feffd640 10 client.17535 handle_command_reply: tid=0 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 client.17535 shutdown 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 msgr2=0x56412e201e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 0x56412e201e40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f43180077f0 tx=0x7f4318004ab0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 shutdown_connections 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e214720 0x56412e216b10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e204980 0x56412e201e40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e2016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 --2- 172.21.15.44:0/1439794524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e1f97c0 0x56412e204410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f42feffd640 0 client.17535 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 >> 172.21.15.44:0/1439794524 conn(0x56412e1f5d50 msgr2=0x56412e1fb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 shutdown_connections 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.530+0000 7f4328824740 1 -- 172.21.15.44:0/1439794524 wait complete. 2024-09-13T17:32:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.531+0000 7f4328824740 20 client.17535 trim_cache size 0 max 16384 2024-09-13T17:32:47.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 Processor -- start 2024-09-13T17:32:47.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 -- start start 2024-09-13T17:32:47.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e1fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e203f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e20b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e212fe0 con 0x56412e1f97c0 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e210560 con 0x56412e1fa0e0 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f4328824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e1f6e70 con 0x56412e204980 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e1fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e1fb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50614/0 (socket says 172.21.15.44:50614) 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 -- 172.21.15.44:0/4155055598 learned_addr learned my addr 172.21.15.44:0/4155055598 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431f7fe640 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e20b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 -- 172.21.15.44:0/4155055598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 msgr2=0x56412e20b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e20b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 -- 172.21.15.44:0/4155055598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 msgr2=0x56412e203f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e203f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 -- 172.21.15.44:0/4155055598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56412e0fe1c0 con 0x56412e1f97c0 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.545+0000 7f431effd640 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e1fb8b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f431002a930 tx=0x7f4310005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cb6d011d985c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f43067fc640 1 -- 172.21.15.44:0/4155055598 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431004c030 con 0x56412e1f97c0 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f43067fc640 1 -- 172.21.15.44:0/4155055598 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f431004d040 con 0x56412e1f97c0 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f43067fc640 1 -- 172.21.15.44:0/4155055598 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310050a90 con 0x56412e1f97c0 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 -- 172.21.15.44:0/4155055598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 msgr2=0x56412e1fb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e1fb8b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f431002a930 tx=0x7f4310005fd0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 -- 172.21.15.44:0/4155055598 shutdown_connections 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e20b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e203f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 --2- 172.21.15.44:0/4155055598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e1fb8b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 -- 172.21.15.44:0/4155055598 >> 172.21.15.44:0/4155055598 conn(0x56412e1f5d50 msgr2=0x56412e1fb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 -- 172.21.15.44:0/4155055598 shutdown_connections 2024-09-13T17:32:47.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.546+0000 7f4328824740 1 -- 172.21.15.44:0/4155055598 wait complete. 2024-09-13T17:32:47.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 -- start start 2024-09-13T17:32:47.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 Processor -- start 2024-09-13T17:32:47.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e20a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e20ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e26ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e26b1d0 con 0x56412e1f97c0 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e26b340 con 0x56412e1fa0e0 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e26b500 con 0x56412e204980 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f4328824740 10 client.0 _ll_register_callbacks cb 0x7f4318012ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e20a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e20a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50622/0 (socket says 172.21.15.44:50622) 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 -- 172.21.15.44:0/3860751670 learned_addr learned my addr 172.21.15.44:0/3860751670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431f7fe640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e26ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 -- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 msgr2=0x56412e26ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e26ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 -- 172.21.15.44:0/3860751670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 msgr2=0x56412e20ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431e7fc640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e20ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.547+0000 7f431effd640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e20ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f431effd640 1 -- 172.21.15.44:0/3860751670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56412e119970 con 0x56412e1f97c0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f431e7fc640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e20ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f431effd640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e20a850 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4310005d90 tx=0x7f4310031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2572949c755e394 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f42feffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f42feffd640 1 -- 172.21.15.44:0/3860751670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431004c050 con 0x56412e1f97c0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f42feffd640 1 -- 172.21.15.44:0/3860751670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7f4310003710 con 0x56412e1f97c0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4310004460 con 0x56412e1f97c0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f42feffd640 1 -- 172.21.15.44:0/3860751670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4310003980 con 0x56412e1f97c0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 10 client.17541 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56412e123990 con 0x56412e1f97c0 2024-09-13T17:32:47.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f42feffd640 1 -- 172.21.15.44:0/3860751670 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4310004460 con 0x56412e1f97c0 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f42feffd640 1 -- 172.21.15.44:0/3860751670 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f431005b460 con 0x56412e1f97c0 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 10 client.17541 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 20 client.17541 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e26c830 0x56412e26ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 4 client.17541 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.548+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56412deb5eb0 con 0x56412e26c830 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.549+0000 7f431e7fc640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e26c830 0x56412e26ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.549+0000 7f431e7fc640 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e26c830 0x56412e26ec50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1663b937809b4929 server_cookie=3b926b5525414ca in_seq=0 out_seq=0 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.549+0000 7f42feffd640 10 client.17541 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f42feffd640 1 -- 172.21.15.44:0/3860751670 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56412deb5eb0 con 0x56412e26c830 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f42feffd640 10 client.17541 handle_command_reply: tid=0 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 client.17541 shutdown 2024-09-13T17:32:47.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 msgr2=0x56412e20a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e20a850 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4310005d90 tx=0x7f4310031c90 comp rx=0 tx=0).stop 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 shutdown_connections 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x56412e26c830 0x56412e26ec50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412e204980 0x56412e26ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56412e1fa0e0 0x56412e20ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 --2- 172.21.15.44:0/3860751670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56412e1f97c0 0x56412e20a850 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f42feffd640 0 client.17541 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.550+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 >> 172.21.15.44:0/3860751670 conn(0x56412e1f5d50 msgr2=0x56412e1f4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 shutdown_connections 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4328824740 1 -- 172.21.15.44:0/3860751670 wait complete. 2024-09-13T17:32:47.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4328824740 20 client.17541 trim_cache size 0 max 16384 2024-09-13T17:32:47.559 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:32:47.559 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:32:47.559 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "09aae1ab-2dd8-465a-8f77-02bce4b85e37", 2024-09-13T17:32:47.559 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:32:47.559 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43080a1200 msgr2=0x7f43080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43080a1200 0x7f43080a36c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4318007ec0 tx=0x7f4318004a40 comp rx=0 tx=0).stop 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 msgr2=0x7f4320196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 0x7f4320196660 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f431400d770 tx=0x7f431400dc30 comp rx=0 tx=0).stop 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 shutdown_connections 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f43080a1200 0x7f43080a36c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432010a310 0x7f432019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320101860 0x7f4320196660 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 --2- 172.21.15.44:0/2121863157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4320100f40 0x7f4320196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 >> 172.21.15.44:0/2121863157 conn(0x7f43200fa930 msgr2=0x7f43200fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 shutdown_connections 2024-09-13T17:32:47.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.551+0000 7f4325a27640 1 -- 172.21.15.44:0/2121863157 wait complete. 2024-09-13T17:32:47.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:32:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 -- 172.21.15.44:0/3095900060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc104c30 msgr2=0x7fc1fc105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 --2- 172.21.15.44:0/3095900060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc104c30 0x7fc1fc105010 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009cd0 tx=0x7fc1ec02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 -- 172.21.15.44:0/3095900060 shutdown_connections 2024-09-13T17:32:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 --2- 172.21.15.44:0/3095900060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1fc0feff0 0x7fc1fc0ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 --2- 172.21.15.44:0/3095900060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 --2- 172.21.15.44:0/3095900060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc104c30 0x7fc1fc105010 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 -- 172.21.15.44:0/2703523155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0ff810 msgr2=0x7fd7ac103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2703523155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0ff810 0x7fd7ac103850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009860 tx=0x7fd79c02a340 comp rx=0 tx=0).stop 2024-09-13T17:32:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 -- 172.21.15.44:0/3095900060 >> 172.21.15.44:0/3095900060 conn(0x7fc1fc074940 msgr2=0x7fc1fc074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 -- 172.21.15.44:0/3095900060 shutdown_connections 2024-09-13T17:32:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fc203d65640 1 -- 172.21.15.44:0/3095900060 wait complete. 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 -- 172.21.15.44:0/2703523155 shutdown_connections 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2703523155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7ac103fb0 0x7fd7ac10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2703523155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0ff810 0x7fd7ac103850 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2703523155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0fee60 0x7fd7ac0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 -- 172.21.15.44:0/2703523155 >> 172.21.15.44:0/2703523155 conn(0x7fd7ac075c60 msgr2=0x7fd7ac0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 -- 172.21.15.44:0/2703523155 shutdown_connections 2024-09-13T17:32:47.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.668+0000 7fd7b4733640 1 -- 172.21.15.44:0/2703523155 wait complete. 2024-09-13T17:32:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 Processor -- start 2024-09-13T17:32:47.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 Processor -- start 2024-09-13T17:32:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 -- start start 2024-09-13T17:32:47.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 -- start start 2024-09-13T17:32:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1fc0feff0 0x7fc1fc198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1fc104c30 0x7fc1fc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc19d270 con 0x7fc1fc0ff970 2024-09-13T17:32:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc19d3e0 con 0x7fc1fc104c30 2024-09-13T17:32:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc203d65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc19d5a0 con 0x7fc1fc0feff0 2024-09-13T17:32:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7ac0fee60 0x7fd7ac1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac103fb0 0x7fd7ac196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 0x7fd7ac19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19b500 con 0x7fd7ac19ab10 2024-09-13T17:32:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19b670 con 0x7fd7ac103fb0 2024-09-13T17:32:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b4733640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19b7e0 con 0x7fd7ac0fee60 2024-09-13T17:32:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc201300640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc201300640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50650/0 (socket says 172.21.15.44:50650) 2024-09-13T17:32:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fc201300640 1 -- 172.21.15.44:0/65896492 learned_addr learned my addr 172.21.15.44:0/65896492 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.669+0000 7fd7b2cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 0x7fd7ac19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 0x7fd7ac19afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50666/0 (socket says 172.21.15.44:50666) 2024-09-13T17:32:47.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/2757045195 learned_addr learned my addr 172.21.15.44:0/2757045195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201b01640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1fc0feff0 0x7fc1fc198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc202302640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1fc104c30 0x7fc1fc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7ac0fee60 0x7fd7ac1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac103fb0 0x7fd7ac196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201300640 1 -- 172.21.15.44:0/65896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1fc0feff0 msgr2=0x7fc1fc198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201300640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1fc0feff0 0x7fc1fc198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201300640 1 -- 172.21.15.44:0/65896492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1fc104c30 msgr2=0x7fc1fc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201300640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1fc104c30 0x7fc1fc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201300640 1 -- 172.21.15.44:0/65896492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec009900 con 0x7fc1fc0ff970 2024-09-13T17:32:47.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/2757045195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7ac0fee60 msgr2=0x7fd7ac1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7ac0fee60 0x7fd7ac1961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/2757045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac103fb0 msgr2=0x7fd7ac196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac103fb0 0x7fd7ac196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/2757045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a8009560 con 0x7fd7ac19ab10 2024-09-13T17:32:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc201300640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc198880 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000e9f0 tx=0x7fc1f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 0x7fd7ac19afc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400ea60 tx=0x7fd7a400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc1faffd640 1 -- 172.21.15.44:0/65896492 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0009720 con 0x7fc1fc0ff970 2024-09-13T17:32:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc1faffd640 1 -- 172.21.15.44:0/65896492 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1f000cd80 con 0x7fc1fc0ff970 2024-09-13T17:32:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc19d880 con 0x7fc1fc0ff970 2024-09-13T17:32:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc1faffd640 1 -- 172.21.15.44:0/65896492 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0010430 con 0x7fc1fc0ff970 2024-09-13T17:32:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc1a5100 con 0x7fc1fc0ff970 2024-09-13T17:32:47.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7a37fe640 1 -- 172.21.15.44:0/2757045195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400ce40 con 0x7fd7ac19ab10 2024-09-13T17:32:47.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7a37fe640 1 -- 172.21.15.44:0/2757045195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a4005020 con 0x7fd7ac19ab10 2024-09-13T17:32:47.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.670+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c009490 con 0x7fd7ac19ab10 2024-09-13T17:32:47.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.671+0000 7fd7a37fe640 1 -- 172.21.15.44:0/2757045195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4010690 con 0x7fd7ac19ab10 2024-09-13T17:32:47.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.671+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac1a3100 con 0x7fd7ac19ab10 2024-09-13T17:32:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.673+0000 7fc1faffd640 1 -- 172.21.15.44:0/65896492 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc1f0004fb0 con 0x7fc1fc0ff970 2024-09-13T17:32:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.673+0000 7fc206a8c740 1 Processor -- start 2024-09-13T17:32:47.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.673+0000 7fd7b7459740 1 Processor -- start 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.673+0000 7fc206a8c740 1 -- start start 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.673+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8330 0x563d3c6a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6b34f0 0x563d3c6b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc206a8c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c613c30 con 0x563d3c6a8c50 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc206a8c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c6113c0 con 0x563d3c6a8330 2024-09-13T17:32:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc206a8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c60ccb0 con 0x563d3c6b34f0 2024-09-13T17:32:47.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7a37fe640 1 -- 172.21.15.44:0/2757045195 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd7a4005190 con 0x7fd7ac19ab10 2024-09-13T17:32:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50678/0 (socket says 172.21.15.44:50678) 2024-09-13T17:32:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 -- 172.21.15.44:0/1984501944 learned_addr learned my addr 172.21.15.44:0/1984501944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc202302640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8330 0x563d3c6a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201300640 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6b34f0 0x563d3c6b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc1faffd640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc1e40a1200 0x7fc1e40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 -- start start 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 -- 172.21.15.44:0/1984501944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6b34f0 msgr2=0x563d3c6b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6b34f0 0x563d3c6b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 -- 172.21.15.44:0/1984501944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8330 msgr2=0x563d3c6a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8330 0x563d3c6a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 -- 172.21.15.44:0/1984501944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d3c5b3910 con 0x563d3c6a8c50 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201300640 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6b34f0 0x563d3c6b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc202302640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc1e40a1200 0x7fc1e40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069be00 0x55626069c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc1faffd640 1 -- 172.21.15.44:0/65896492 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc1f0014070 con 0x7fc1fc0ff970 2024-09-13T17:32:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 0x5562606a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556260606f20 con 0x5562606a6fc0 2024-09-13T17:32:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562606046b0 con 0x55626069c720 2024-09-13T17:32:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fd7b7459740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562605fffa0 con 0x55626069be00 2024-09-13T17:32:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.674+0000 7fc201b01640 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b2fb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec02ae00 tx=0x7fc1ec005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86ed24be5a867ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fc202302640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc1e40a1200 0x7fc1e40a36c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fd0 tx=0x7fc1f40046c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b1cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 0x5562606a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 0x5562606a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:41650/0 (socket says 172.21.15.44:41650) 2024-09-13T17:32:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 -- 172.21.15.44:0/637388052 learned_addr learned my addr 172.21.15.44:0/637388052 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fc1e3fff640 1 -- 172.21.15.44:0/1984501944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0307a0 con 0x563d3c6a8c50 2024-09-13T17:32:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fc1e3fff640 1 -- 172.21.15.44:0/1984501944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1ec040070 con 0x563d3c6a8c50 2024-09-13T17:32:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069be00 0x55626069c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fc1e3fff640 1 -- 172.21.15.44:0/1984501944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec003bd0 con 0x563d3c6a8c50 2024-09-13T17:32:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 -- 172.21.15.44:0/637388052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069be00 msgr2=0x55626069c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069be00 0x55626069c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 -- 172.21.15.44:0/637388052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 msgr2=0x5562606a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 -- 172.21.15.44:0/637388052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562605ba840 con 0x55626069c720 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7a37fe640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7980a1280 0x7fd7980a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 0x5562606a6a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009e60 tx=0x7fd7a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3ed8a289062412 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7980a1280 0x7fd7980a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7a37fe640 1 -- 172.21.15.44:0/2757045195 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd7a4014070 con 0x7fd7ac19ab10 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7a0ff9640 1 -- 172.21.15.44:0/637388052 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8005020 con 0x55626069c720 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7a0ff9640 1 -- 172.21.15.44:0/637388052 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a8005610 con 0x55626069c720 2024-09-13T17:32:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.675+0000 7fd7a0ff9640 1 -- 172.21.15.44:0/637388052 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8019670 con 0x55626069c720 2024-09-13T17:32:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 -- 172.21.15.44:0/1984501944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 msgr2=0x563d3c6b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b2fb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec02ae00 tx=0x7fc1ec005fa0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 -- 172.21.15.44:0/1984501944 shutdown_connections 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6b34f0 0x563d3c6b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b2fb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1984501944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8330 0x563d3c6a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 -- 172.21.15.44:0/1984501944 >> 172.21.15.44:0/1984501944 conn(0x563d3c6a48c0 msgr2=0x563d3c6a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 -- 172.21.15.44:0/1984501944 shutdown_connections 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 -- 172.21.15.44:0/1984501944 wait complete. 2024-09-13T17:32:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 -- start start 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7980a1280 0x7fd7980a3740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009830 tx=0x7fd79c02c040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 -- 172.21.15.44:0/637388052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 msgr2=0x5562606a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 0x5562606a6a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009e60 tx=0x7fd7a8011810 comp rx=0 tx=0).stop 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 -- 172.21.15.44:0/637388052 shutdown_connections 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069c720 0x5562606a6a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 --2- 172.21.15.44:0/637388052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069be00 0x55626069c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 -- 172.21.15.44:0/637388052 >> 172.21.15.44:0/637388052 conn(0x556260698370 msgr2=0x556260698750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fc206a8c740 1 Processor -- start 2024-09-13T17:32:47.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 -- 172.21.15.44:0/637388052 shutdown_connections 2024-09-13T17:32:47.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.676+0000 7fd7b7459740 1 -- 172.21.15.44:0/637388052 wait complete. 2024-09-13T17:32:47.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 -- start start 2024-09-13T17:32:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 Processor -- start 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6b34f0 0x563d3c6b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c5cae80 con 0x563d3c6a8c50 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c5cab20 con 0x563d3c6b34f0 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c590260 con 0x563d3c6a8330 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc206a8c740 10 client.0 _ll_register_callbacks cb 0x563d3c5c5df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc202302640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6b34f0 0x563d3c6b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50698/0 (socket says 172.21.15.44:50698) 2024-09-13T17:32:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 -- 172.21.15.44:0/1033221481 learned_addr learned my addr 172.21.15.44:0/1033221481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 0x5562606a6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201b01640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 -- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 msgr2=0x563d3c6b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 -- 172.21.15.44:0/1033221481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6b34f0 msgr2=0x563d3c6b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6b34f0 0x563d3c6b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562605a12a0 con 0x5562606a6fc0 2024-09-13T17:32:47.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562602d8430 con 0x55626069be00 2024-09-13T17:32:47.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556260583680 con 0x55626069c720 2024-09-13T17:32:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc201300640 1 -- 172.21.15.44:0/1033221481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc10c340 con 0x563d3c6a8c50 2024-09-13T17:32:47.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b7459740 10 client.0 _ll_register_callbacks cb 0x5562605f9610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:47.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b2cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 0x5562606a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fc202302640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6b34f0 0x563d3c6b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 0x5562606a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:41680/0 (socket says 172.21.15.44:41680) 2024-09-13T17:32:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b2cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50700/0 (socket says 172.21.15.44:50700) 2024-09-13T17:32:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b24cf640 1 -- 172.21.15.44:0/1801033061 learned_addr learned my addr 172.21.15.44:0/1801033061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.677+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc201300640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0140 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00ca7e0 tx=0x7fc1f0099fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45ef00ad84a1c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc1e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc1e17fa640 1 -- 172.21.15.44:0/1033221481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0099830 con 0x563d3c6a8c50 2024-09-13T17:32:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 msgr2=0x5562606a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/1801033061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 msgr2=0x5562606a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 0x5562606a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d3c635680 con 0x563d3c6a8c50 2024-09-13T17:32:47.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b2cd0640 1 -- 172.21.15.44:0/1801033061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a8009a90 con 0x5562606a6fc0 2024-09-13T17:32:47.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 0x5562606a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc1e17fa640 1 -- 172.21.15.44:0/1033221481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc1f009a840 con 0x563d3c6a8c50 2024-09-13T17:32:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc1e17fa640 1 -- 172.21.15.44:0/1033221481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f0005470 con 0x563d3c6a8c50 2024-09-13T17:32:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 10 client.17562 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d3c60b330 con 0x563d3c6a8c50 2024-09-13T17:32:47.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a4000 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40ca980 tx=0x7fd7a4005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5db4de3ca43fe3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:47.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd795ffb640 1 -- 172.21.15.44:0/1801033061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40cd070 con 0x5562606a6fc0 2024-09-13T17:32:47.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd795ffb640 1 -- 172.21.15.44:0/1801033061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd7a409a340 con 0x5562606a6fc0 2024-09-13T17:32:47.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562602f34e0 con 0x5562606a6fc0 2024-09-13T17:32:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc1e17fa640 1 -- 172.21.15.44:0/1033221481 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d3c635680 con 0x563d3c6a8c50 2024-09-13T17:32:47.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd795ffb640 1 -- 172.21.15.44:0/1801033061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a40cd3d0 con 0x5562606a6fc0 2024-09-13T17:32:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc1e17fa640 1 -- 172.21.15.44:0/1033221481 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1f001f060 con 0x563d3c6a8c50 2024-09-13T17:32:47.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b7459740 10 client.17565 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5562605fe620 con 0x5562606a6fc0 2024-09-13T17:32:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 10 client.17562 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 20 client.17562 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.678+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6c2e70 0x563d3c6c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fc206a8c740 4 client.17562 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d3c6005d0 con 0x563d3c6c2e70 2024-09-13T17:32:47.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd795ffb640 1 -- 172.21.15.44:0/1801033061 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5562602f34e0 con 0x5562606a6fc0 2024-09-13T17:32:47.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd795ffb640 1 -- 172.21.15.44:0/1801033061 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7a401f060 con 0x5562606a6fc0 2024-09-13T17:32:47.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b7459740 10 client.17565 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:47.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b7459740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:47.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b7459740 20 client.17565 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:47.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b7459740 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x5562606b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b7459740 4 client.17565 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:32:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fc201b01640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6c2e70 0x563d3c6c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562603047e0 con 0x5562606b6d10 2024-09-13T17:32:47.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x5562606b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fc1e17fa640 10 client.17562 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fc201b01640 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6c2e70 0x563d3c6c5260 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b2d56da0ea148d server_cookie=cc8276976069308f in_seq=0 out_seq=0 2024-09-13T17:32:47.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x5562606b9130 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5d423be1dba021 server_cookie=f912582ae26f4f0d in_seq=0 out_seq=0 2024-09-13T17:32:47.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.679+0000 7fd795ffb640 10 client.17565 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc1e17fa640 1 -- 172.21.15.44:0/1033221481 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563d3c6005d0 con 0x563d3c6c2e70 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc1e17fa640 10 client.17562 handle_command_reply: tid=0 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 client.17562 shutdown 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 msgr2=0x563d3c6b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0140 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00ca7e0 tx=0x7fc1f0099fd0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 shutdown_connections 2024-09-13T17:32:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6c2e70 0x563d3c6c5260 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6b34f0 0x563d3c6b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0140 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc1e17fa640 0 client.17562 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 --2- 172.21.15.44:0/1033221481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 >> 172.21.15.44:0/1033221481 conn(0x563d3c6a48c0 msgr2=0x563d3c6a3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd795ffb640 1 -- 172.21.15.44:0/1801033061 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5562603047e0 con 0x5562606b6d10 2024-09-13T17:32:47.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd795ffb640 10 client.17565 handle_command_reply: tid=0 2024-09-13T17:32:47.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd7b7459740 1 client.17565 shutdown 2024-09-13T17:32:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 shutdown_connections 2024-09-13T17:32:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 1 -- 172.21.15.44:0/1033221481 wait complete. 2024-09-13T17:32:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fc206a8c740 20 client.17562 trim_cache size 0 max 16384 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 msgr2=0x5562606a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd7b7459740 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a4000 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40ca980 tx=0x7fd7a4005470 comp rx=0 tx=0).stop 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 shutdown_connections 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.680+0000 7fd7b7459740 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x5562606b9130 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd795ffb640 0 client.17565 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562606a6fc0 0x5562606a4000 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 1 --2- 172.21.15.44:0/1801033061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55626069be00 0x5562606a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 >> 172.21.15.44:0/1801033061 conn(0x556260698370 msgr2=0x55626069e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 shutdown_connections 2024-09-13T17:32:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 1 -- 172.21.15.44:0/1801033061 wait complete. 2024-09-13T17:32:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.681+0000 7fd7b7459740 20 client.17565 trim_cache size 0 max 16384 2024-09-13T17:32:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 Processor -- start 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 -- start start 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c6a5a50 con 0x563d3c6a48c0 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c6bfc00 con 0x563d3c6a8c50 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc206a8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c5c5d10 con 0x563d3c6a8330 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50710/0 (socket says 172.21.15.44:50710) 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 -- 172.21.15.44:0/3256020018 learned_addr learned my addr 172.21.15.44:0/3256020018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201300640 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc202302640 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 -- 172.21.15.44:0/3256020018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 msgr2=0x563d3c6ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 -- 172.21.15.44:0/3256020018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 msgr2=0x563d3c6b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 -- 172.21.15.44:0/3256020018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d3c5ad300 con 0x563d3c6a48c0 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.693+0000 7fc201b01640 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ab410 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec02aed0 tx=0x7fc1ec031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22487d3b0144826 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc1f8ff9640 1 -- 172.21.15.44:0/3256020018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec04b040 con 0x563d3c6a48c0 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc1f8ff9640 1 -- 172.21.15.44:0/3256020018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1ec0315b0 con 0x563d3c6a48c0 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc1f8ff9640 1 -- 172.21.15.44:0/3256020018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec053590 con 0x563d3c6a48c0 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 -- 172.21.15.44:0/3256020018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 msgr2=0x563d3c6ab410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ab410 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec02aed0 tx=0x7fc1ec031c70 comp rx=0 tx=0).stop 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 -- 172.21.15.44:0/3256020018 shutdown_connections 2024-09-13T17:32:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 --2- 172.21.15.44:0/3256020018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ab410 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 -- 172.21.15.44:0/3256020018 >> 172.21.15.44:0/3256020018 conn(0x563d3c6b3020 msgr2=0x563d3c6bec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 -- 172.21.15.44:0/3256020018 shutdown_connections 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.694+0000 7fc206a8c740 1 -- 172.21.15.44:0/3256020018 wait complete. 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 -- start start 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 Processor -- start 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c6ad6d0 con 0x563d3c6a48c0 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c6ad840 con 0x563d3c6a8c50 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3c721b50 con 0x563d3c6a8330 2024-09-13T17:32:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc206a8c740 10 client.0 _ll_register_callbacks cb 0x7fc1ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc201b01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fd7b7459740 1 Processor -- start 2024-09-13T17:32:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc201b01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ba480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50720/0 (socket says 172.21.15.44:50720) 2024-09-13T17:32:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc201b01640 1 -- 172.21.15.44:0/993660750 learned_addr learned my addr 172.21.15.44:0/993660750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fd7b7459740 1 -- start start 2024-09-13T17:32:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc201300640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fc202302640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.695+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x55626069ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x55626069f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x5562606a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc201b01640 1 -- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 msgr2=0x563d3c6aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b7459740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556260693ee0 con 0x55626069be00 2024-09-13T17:32:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b7459740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562606b5550 con 0x5562606a6fc0 2024-09-13T17:32:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b7459740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562606a4370 con 0x55626069c720 2024-09-13T17:32:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc201b01640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc201b01640 1 -- 172.21.15.44:0/993660750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 msgr2=0x563d3c6acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc201b01640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc201b01640 1 -- 172.21.15.44:0/993660750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d3c5b3910 con 0x563d3c6a48c0 2024-09-13T17:32:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc202302640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x55626069ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x55626069ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50724/0 (socket says 172.21.15.44:50724) 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 -- 172.21.15.44:0/2258239424 learned_addr learned my addr 172.21.15.44:0/2258239424 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x55626069f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x5562606a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 -- 172.21.15.44:0/2258239424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 msgr2=0x55626069f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x55626069f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 -- 172.21.15.44:0/2258239424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 msgr2=0x5562606a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x5562606a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 -- 172.21.15.44:0/2258239424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562605a0790 con 0x55626069be00 2024-09-13T17:32:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc201b01640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ba480 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009430 tx=0x7fc1ec0314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98843dd241defe56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc1e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:47.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x55626069f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:47.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x5562606a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:32:47.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x55626069ede0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd7a801b040 tx=0x7fd7a8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49befb8a0beda622 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc1e17fa640 1 -- 172.21.15.44:0/993660750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec04b040 con 0x563d3c6a48c0 2024-09-13T17:32:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc1e17fa640 1 -- 172.21.15.44:0/993660750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fc1ec042070 con 0x563d3c6a48c0 2024-09-13T17:32:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1ec040070 con 0x563d3c6a48c0 2024-09-13T17:32:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7a17fa640 1 -- 172.21.15.44:0/2258239424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8028040 con 0x55626069be00 2024-09-13T17:32:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7a17fa640 1 -- 172.21.15.44:0/2258239424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a8004b30 con 0x55626069be00 2024-09-13T17:32:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc1e17fa640 1 -- 172.21.15.44:0/993660750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1ec0308c0 con 0x563d3c6a48c0 2024-09-13T17:32:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fd7a17fa640 1 -- 172.21.15.44:0/2258239424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004220 con 0x55626069be00 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc206a8c740 10 client.17571 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.696+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d3c5d28e0 con 0x563d3c6a48c0 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc1e17fa640 1 -- 172.21.15.44:0/993660750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1ec040070 con 0x563d3c6a48c0 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc1e17fa640 1 -- 172.21.15.44:0/993660750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1ec047480 con 0x563d3c6a48c0 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc206a8c740 10 client.17571 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc206a8c740 10 client.17571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc206a8c740 20 client.17571 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc206a8c740 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6b34f0 0x563d3c724d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc206a8c740 4 client.17571 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:32:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d3c300300 con 0x563d3c6b34f0 2024-09-13T17:32:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc201300640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6b34f0 0x563d3c724d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 -- 172.21.15.44:0/2258239424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 msgr2=0x55626069ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x55626069ede0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd7a801b040 tx=0x7fd7a8005780 comp rx=0 tx=0).stop 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 -- 172.21.15.44:0/2258239424 shutdown_connections 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x55626069f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x5562606a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 --2- 172.21.15.44:0/2258239424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x55626069ede0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 -- 172.21.15.44:0/2258239424 >> 172.21.15.44:0/2258239424 conn(0x556260698370 msgr2=0x556260698750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 -- 172.21.15.44:0/2258239424 shutdown_connections 2024-09-13T17:32:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fd7b7459740 1 -- 172.21.15.44:0/2258239424 wait complete. 2024-09-13T17:32:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc1e17fa640 10 client.17571 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.697+0000 7fc201300640 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6b34f0 0x563d3c724d40 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac070a91d47481da server_cookie=96ae8049da19597f in_seq=0 out_seq=0 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 -- start start 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 Processor -- start 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x5562606ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x55626070c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562606ae1c0 con 0x55626069be00 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626070c750 con 0x5562606a6fc0 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626070c8c0 con 0x55626069c720 2024-09-13T17:32:47.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b7459740 10 client.0 _ll_register_callbacks cb 0x7fd7a4099e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x5562606ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x5562606ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50740/0 (socket says 172.21.15.44:50740) 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 -- 172.21.15.44:0/115611099 learned_addr learned my addr 172.21.15.44:0/115611099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b2cd0640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x55626070c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 -- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 msgr2=0x5562606adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 -- 172.21.15.44:0/115611099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 msgr2=0x55626070c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x55626070c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.698+0000 7fd7b24cf640 1 -- 172.21.15.44:0/115611099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562605ba840 con 0x55626069be00 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b24cf640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x5562606ad560 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8011e70 tx=0x7fd7a8004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1c75537c6cf87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd795ffb640 1 -- 172.21.15.44:0/115611099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8028040 con 0x55626069be00 2024-09-13T17:32:47.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd795ffb640 1 -- 172.21.15.44:0/115611099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fd7a8005270 con 0x55626069be00 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a8005610 con 0x55626069be00 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd795ffb640 1 -- 172.21.15.44:0/115611099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a8005430 con 0x55626069be00 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 10 client.17577 fetch_fsmap learned FSMap version 25 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5562605c5ea0 con 0x55626069be00 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd795ffb640 1 -- 172.21.15.44:0/115611099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a8005610 con 0x55626069be00 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd795ffb640 1 -- 172.21.15.44:0/115611099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7a80035f0 con 0x55626069be00 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 10 client.17577 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 20 client.17577 populate_metadata read hostname 'smithi044' 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x55626070fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 4 client.17577 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.699+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556260357e30 con 0x5562606b6d10 2024-09-13T17:32:47.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.700+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x55626070fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:32:47.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.700+0000 7fd795ffb640 10 client.17577 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.700+0000 7fd7b1cce640 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x55626070fdc0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcef33508469d5c2 server_cookie=2aac01d1daa7abc6 in_seq=0 out_seq=0 2024-09-13T17:32:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.710+0000 7fc1e17fa640 1 -- 172.21.15.44:0/993660750 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d3c300300 con 0x563d3c6b34f0 2024-09-13T17:32:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.710+0000 7fc1e17fa640 10 client.17571 handle_command_reply: tid=0 2024-09-13T17:32:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.710+0000 7fc206a8c740 1 client.17571 shutdown 2024-09-13T17:32:47.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.710+0000 7fd795ffb640 1 -- 172.21.15.44:0/115611099 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556260357e30 con 0x5562606b6d10 2024-09-13T17:32:47.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.710+0000 7fd795ffb640 10 client.17577 handle_command_reply: tid=0 2024-09-13T17:32:47.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.710+0000 7fd7b7459740 1 client.17577 shutdown 2024-09-13T17:32:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 msgr2=0x563d3c6ba480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ba480 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009430 tx=0x7fc1ec0314b0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 msgr2=0x5562606ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x5562606ad560 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8011e70 tx=0x7fd7a8004e40 comp rx=0 tx=0).stop 2024-09-13T17:32:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 shutdown_connections 2024-09-13T17:32:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5562606b6d10 0x55626070fdc0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 shutdown_connections 2024-09-13T17:32:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563d3c6b34f0 0x563d3c724d40 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d3c6a8330 0x563d3c6aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d3c6a8c50 0x563d3c6acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc1e17fa640 0 client.17571 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 --2- 172.21.15.44:0/993660750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d3c6a48c0 0x563d3c6ba480 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55626069c720 0x5562606adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd795ffb640 0 client.17577 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:32:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562606a6fc0 0x55626070c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 --2- 172.21.15.44:0/115611099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55626069be00 0x5562606ad560 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 >> 172.21.15.44:0/115611099 conn(0x556260698370 msgr2=0x556260698750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 >> 172.21.15.44:0/993660750 conn(0x563d3c6b3020 msgr2=0x563d3c6bec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 shutdown_connections 2024-09-13T17:32:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 1 -- 172.21.15.44:0/115611099 wait complete. 2024-09-13T17:32:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 shutdown_connections 2024-09-13T17:32:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 1 -- 172.21.15.44:0/993660750 wait complete. 2024-09-13T17:32:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b7459740 20 client.17577 trim_cache size 0 max 16384 2024-09-13T17:32:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc206a8c740 20 client.17571 trim_cache size 0 max 16384 2024-09-13T17:32:47.747 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:32:47.747 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:32:47.747 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout: "09aae1ab-2dd8-465a-8f77-02bce4b85e37": { 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "09aae1ab-2dd8-465a-8f77-02bce4b85e37", 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:32:47.748 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:32:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:32:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:32:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:32:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:32:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc1e40a1200 msgr2=0x7fc1e40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc1e40a1200 0x7fc1e40a36c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fd0 tx=0x7fc1f40046c0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7980a1280 msgr2=0x7fd7980a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7980a1280 0x7fd7980a3740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009830 tx=0x7fd79c02c040 comp rx=0 tx=0).stop 2024-09-13T17:32:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 msgr2=0x7fd7ac19afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 0x7fd7ac19afc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400ea60 tx=0x7fd7a400ef20 comp rx=0 tx=0).stop 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 msgr2=0x7fc1fc198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc198880 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000e9f0 tx=0x7fc1f000edf0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 shutdown_connections 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc1e40a1200 0x7fc1e40a36c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1fc104c30 0x7fc1fc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1fc0ff970 0x7fc1fc198880 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.711+0000 7fc203d65640 1 --2- 172.21.15.44:0/65896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1fc0feff0 0x7fc1fc198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 >> 172.21.15.44:0/65896492 conn(0x7fc1fc074940 msgr2=0x7fc1fc067930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 shutdown_connections 2024-09-13T17:32:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd7980a1280 0x7fd7980a3740 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac19ab10 0x7fd7ac19afc0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac103fb0 0x7fd7ac196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 --2- 172.21.15.44:0/2757045195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7ac0fee60 0x7fd7ac1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:32:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 >> 172.21.15.44:0/2757045195 conn(0x7fd7ac075c60 msgr2=0x7fd7ac0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:32:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 shutdown_connections 2024-09-13T17:32:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fc203d65640 1 -- 172.21.15.44:0/65896492 wait complete. 2024-09-13T17:32:47.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 shutdown_connections 2024-09-13T17:32:47.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:32:47.712+0000 7fd7b4733640 1 -- 172.21.15.44:0/2757045195 wait complete. 2024-09-13T17:32:47.753 INFO:tasks.cephfs.filesystem:scrub status for tag:09aae1ab-2dd8-465a-8f77-02bce4b85e37 - {'path': '/', 'tag': '09aae1ab-2dd8-465a-8f77-02bce4b85e37', 'options': 'recursive,force'} 2024-09-13T17:32:48.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:47 smithi044 ceph-mon[31175]: pgmap v185: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:32:48.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:47 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:32:48.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:47 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:48.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:47 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:32:48.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:47 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:48.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:47 smithi100 ceph-mon[35055]: pgmap v185: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:32:48.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:47 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:32:48.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:47 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:48.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:47 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:32:48.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:47 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:47 smithi070 ceph-mon[34502]: pgmap v185: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:32:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:47 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:32:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:47 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:47 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:32:48.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:47 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:32:49.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:48 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:32:49.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:48 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:32:49.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:48 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:32:50.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:49 smithi100 ceph-mon[35055]: pgmap v186: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-09-13T17:32:50.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:49 smithi044 ceph-mon[31175]: pgmap v186: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-09-13T17:32:50.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:49 smithi070 ceph-mon[34502]: pgmap v186: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-09-13T17:32:52.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:51 smithi100 ceph-mon[35055]: pgmap v187: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-13T17:32:52.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:51 smithi044 ceph-mon[31175]: pgmap v187: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-13T17:32:52.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:51 smithi070 ceph-mon[34502]: pgmap v187: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-13T17:32:54.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:53 smithi100 ceph-mon[35055]: pgmap v188: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-13T17:32:54.358 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:53 smithi044 ceph-mon[31175]: pgmap v188: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-13T17:32:54.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:53 smithi070 ceph-mon[34502]: pgmap v188: 129 pgs: 129 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-13T17:32:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:32:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:32:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:54 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:32:55.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:54 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:32:55.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:32:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:32:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:54 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:32:55.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:54 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:32:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:32:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:32:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:54 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:32:55.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:54 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:32:56.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:55 smithi100 ceph-mon[35055]: pgmap v189: 129 pgs: 129 active+clean; 1.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-13T17:32:56.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:55 smithi044 ceph-mon[31175]: pgmap v189: 129 pgs: 129 active+clean; 1.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-13T17:32:56.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:55 smithi070 ceph-mon[34502]: pgmap v189: 129 pgs: 129 active+clean; 1.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-13T17:32:58.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:32:58 smithi100 ceph-mon[35055]: pgmap v190: 129 pgs: 129 active+clean; 1.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-13T17:32:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:32:58 smithi044 ceph-mon[31175]: pgmap v190: 129 pgs: 129 active+clean; 1.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-13T17:32:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:32:58 smithi070 ceph-mon[34502]: pgmap v190: 129 pgs: 129 active+clean; 1.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-13T17:33:00.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:00 smithi100 ceph-mon[35055]: pgmap v191: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-09-13T17:33:00.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:00 smithi044 ceph-mon[31175]: pgmap v191: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-09-13T17:33:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:00 smithi070 ceph-mon[34502]: pgmap v191: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-09-13T17:33:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 -- 172.21.15.44:0/1326017996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 msgr2=0x7fbd3c0ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 --2- 172.21.15.44:0/1326017996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c0ff430 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009930 tx=0x7fbd3002a420 comp rx=0 tx=0).stop 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 -- 172.21.15.44:0/1326017996 shutdown_connections 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 --2- 172.21.15.44:0/1326017996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c0ff970 0x7fbd3c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 --2- 172.21.15.44:0/1326017996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c0ff430 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 --2- 172.21.15.44:0/1326017996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c104c30 0x7fbd3c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 -- 172.21.15.44:0/1326017996 >> 172.21.15.44:0/1326017996 conn(0x7fbd3c074940 msgr2=0x7fbd3c074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 -- 172.21.15.44:0/1326017996 shutdown_connections 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.806+0000 7fbd41176640 1 -- 172.21.15.44:0/1326017996 wait complete. 2024-09-13T17:33:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 Processor -- start 2024-09-13T17:33:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 -- start start 2024-09-13T17:33:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c0ff970 0x7fbd3c1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c104c30 0x7fbd3c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19b0a0 con 0x7fbd3c0feff0 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19b210 con 0x7fbd3c104c30 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd41176640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19b3d0 con 0x7fbd3c0ff970 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40942/0 (socket says 172.21.15.44:40942) 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.807+0000 7fbd3ad9d640 1 -- 172.21.15.44:0/1797822456 learned_addr learned my addr 172.21.15.44:0/1797822456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c0ff970 0x7fbd3c1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3ad9d640 1 -- 172.21.15.44:0/1797822456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c0ff970 msgr2=0x7fbd3c1966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c104c30 0x7fbd3c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3ad9d640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c0ff970 0x7fbd3c1966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3ad9d640 1 -- 172.21.15.44:0/1797822456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c104c30 msgr2=0x7fbd3c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3ad9d640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c104c30 0x7fbd3c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3ad9d640 1 -- 172.21.15.44:0/1797822456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30009560 con 0x7fbd3c0feff0 2024-09-13T17:33:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd3ad9d640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c196170 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00d950 tx=0x7fbd2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd23fff640 1 -- 172.21.15.44:0/1797822456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c009a30 con 0x7fbd3c0feff0 2024-09-13T17:33:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd23fff640 1 -- 172.21.15.44:0/1797822456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd2c0044b0 con 0x7fbd3c0feff0 2024-09-13T17:33:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c19b6b0 con 0x7fbd3c0feff0 2024-09-13T17:33:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd23fff640 1 -- 172.21.15.44:0/1797822456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0047a0 con 0x7fbd3c0feff0 2024-09-13T17:33:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.808+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c1a2f30 con 0x7fbd3c0feff0 2024-09-13T17:33:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.810+0000 7fbd23fff640 1 -- 172.21.15.44:0/1797822456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbd2c004020 con 0x7fbd3c0feff0 2024-09-13T17:33:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 Processor -- start 2024-09-13T17:33:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 -- start start 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6330 0x55af03fe6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd23fff640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbd240a1200 0x7fbd240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6c50 0x55af03ff0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03f51c30 con 0x55af03ff14f0 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03f4f3c0 con 0x55af03fe6c50 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd43e96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03f4acb0 con 0x55af03fe6330 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd3ad9d640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbd240a1200 0x7fbd240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40948/0 (socket says 172.21.15.44:40948) 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd3b59e640 1 -- 172.21.15.44:0/459338170 learned_addr learned my addr 172.21.15.44:0/459338170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6330 0x55af03fe6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd23fff640 1 -- 172.21.15.44:0/1797822456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbd2c0cca90 con 0x7fbd3c0feff0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.811+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6c50 0x55af03ff0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3b59e640 1 -- 172.21.15.44:0/459338170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6330 msgr2=0x55af03fe6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6330 0x55af03fe6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3b59e640 1 -- 172.21.15.44:0/459338170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6c50 msgr2=0x55af03ff0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6c50 0x55af03ff0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3b59e640 1 -- 172.21.15.44:0/459338170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af03ef1910 con 0x55af03ff14f0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3ad9d640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbd240a1200 0x7fbd240a36c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0c9f70 tx=0x7fbd2c06b960 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff38e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbd34014a80 tx=0x7fbd34009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3728d12b54d674f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd217fa640 1 -- 172.21.15.44:0/459338170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34014270 con 0x55af03ff14f0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd217fa640 1 -- 172.21.15.44:0/459338170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd3401a030 con 0x55af03ff14f0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.812+0000 7fbd217fa640 1 -- 172.21.15.44:0/459338170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34005280 con 0x55af03ff14f0 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 -- 172.21.15.44:0/459338170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 msgr2=0x55af03ff38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff38e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbd34014a80 tx=0x7fbd34009680 comp rx=0 tx=0).stop 2024-09-13T17:33:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 -- 172.21.15.44:0/459338170 shutdown_connections 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff38e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6c50 0x55af03ff0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 --2- 172.21.15.44:0/459338170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6330 0x55af03fe6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 -- 172.21.15.44:0/459338170 >> 172.21.15.44:0/459338170 conn(0x55af03fe28c0 msgr2=0x55af03fe2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 -- 172.21.15.44:0/459338170 shutdown_connections 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 -- 172.21.15.44:0/459338170 wait complete. 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 -- start start 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.813+0000 7fbd43e96740 1 Processor -- start 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 0x55af03fee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03ff14f0 0x55af03fee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03f08e80 con 0x55af03fe6c50 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03f08b20 con 0x55af03fe6330 2024-09-13T17:33:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03ece260 con 0x55af03ff14f0 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd43e96740 10 client.0 _ll_register_callbacks cb 0x55af03f03df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 0x55af03fee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 0x55af03fee040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40962/0 (socket says 172.21.15.44:40962) 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 -- 172.21.15.44:0/2922408360 learned_addr learned my addr 172.21.15.44:0/2922408360 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03ff14f0 0x55af03fee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 -- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03ff14f0 msgr2=0x55af03fee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03ff14f0 0x55af03fee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 -- 172.21.15.44:0/2922408360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 msgr2=0x55af03ff0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 -- 172.21.15.44:0/2922408360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd34014730 con 0x55af03fe6c50 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 0x55af03fee040 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009340 tx=0x7fbd300318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9bfa6aca3cdda2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.814+0000 7fbd1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd1eff5640 1 -- 172.21.15.44:0/2922408360 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30004480 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd1eff5640 1 -- 172.21.15.44:0/2922408360 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbd30031960 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af03f73680 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd1eff5640 1 -- 172.21.15.44:0/2922408360 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd30031b20 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 10 client.17589 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55af03f49330 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd1eff5640 1 -- 172.21.15.44:0/2922408360 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af03f73680 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd1eff5640 1 -- 172.21.15.44:0/2922408360 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd30045470 con 0x55af03fe6c50 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 10 client.17589 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 20 client.17589 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04000dd0 0x55af040031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 4 client.17589 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.815+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af03f3e5d0 con 0x55af04000dd0 2024-09-13T17:33:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.816+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04000dd0 0x55af040031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.816+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04000dd0 0x55af040031f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93cba285ed09efe1 server_cookie=71606cde9d97105b in_seq=0 out_seq=0 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.816+0000 7fbd1eff5640 10 client.17589 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd1eff5640 1 -- 172.21.15.44:0/2922408360 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55af03f3e5d0 con 0x55af04000dd0 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd1eff5640 10 client.17589 handle_command_reply: tid=0 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 client.17589 shutdown 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 msgr2=0x55af03fee040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 0x55af03fee040 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009340 tx=0x7fbd300318a0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 shutdown_connections 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04000dd0 0x55af040031f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03ff14f0 0x55af03fee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6c50 0x55af03fee040 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd1eff5640 0 client.17589 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 --2- 172.21.15.44:0/2922408360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 >> 172.21.15.44:0/2922408360 conn(0x55af03fe28c0 msgr2=0x55af03fe1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 shutdown_connections 2024-09-13T17:33:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 1 -- 172.21.15.44:0/2922408360 wait complete. 2024-09-13T17:33:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.817+0000 7fbd43e96740 20 client.17589 trim_cache size 0 max 16384 2024-09-13T17:33:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.829+0000 7fbd43e96740 1 Processor -- start 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.829+0000 7fbd43e96740 1 -- start start 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 0x55af03fde120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03feb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03ff14f0 0x55af03feb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd43e96740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03ffd9d0 con 0x55af03fe6330 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd43e96740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03ff3fa0 con 0x55af03ff14f0 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd43e96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03fec570 con 0x55af03fe6c50 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 0x55af03fde120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 0x55af03fde120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40978/0 (socket says 172.21.15.44:40978) 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 -- 172.21.15.44:0/4052742811 learned_addr learned my addr 172.21.15.44:0/4052742811 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03feb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 -- 172.21.15.44:0/4052742811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 msgr2=0x55af03feb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03feb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 -- 172.21.15.44:0/4052742811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03ff14f0 msgr2=0x55af03feb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03ff14f0 0x55af03feb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 -- 172.21.15.44:0/4052742811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af03eeb300 con 0x55af03fe6330 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03feb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 0x55af03fde120 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbd34005fd0 tx=0x7fbd3400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59158820c4e39e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd21ffb640 1 -- 172.21.15.44:0/4052742811 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400fa60 con 0x55af03fe6330 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd21ffb640 1 -- 172.21.15.44:0/4052742811 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd3402e8d0 con 0x55af03fe6330 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.830+0000 7fbd21ffb640 1 -- 172.21.15.44:0/4052742811 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34005250 con 0x55af03fe6330 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 -- 172.21.15.44:0/4052742811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 msgr2=0x55af03fde120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 0x55af03fde120 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbd34005fd0 tx=0x7fbd3400fc90 comp rx=0 tx=0).stop 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 -- 172.21.15.44:0/4052742811 shutdown_connections 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03feb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03ff14f0 0x55af03feb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 --2- 172.21.15.44:0/4052742811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03fe6330 0x55af03fde120 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 -- 172.21.15.44:0/4052742811 >> 172.21.15.44:0/4052742811 conn(0x55af03fe28c0 msgr2=0x55af03fdda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 -- 172.21.15.44:0/4052742811 shutdown_connections 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 -- 172.21.15.44:0/4052742811 wait complete. 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 -- start start 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.831+0000 7fbd43e96740 1 Processor -- start 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03ff6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03ff77f0 con 0x55af03ff14f0 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af03ff7960 con 0x55af03fe6330 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af040572e0 con 0x55af03fe6c50 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 10 client.0 _ll_register_callbacks cb 0x7fbd34009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40986/0 (socket says 172.21.15.44:40986) 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 -- 172.21.15.44:0/932903576 learned_addr learned my addr 172.21.15.44:0/932903576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03ff6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 -- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 msgr2=0x55af03ff6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03ff6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 -- 172.21.15.44:0/932903576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 msgr2=0x55af03ff8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 -- 172.21.15.44:0/932903576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd34029040 con 0x55af03ff14f0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd3b59e640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff70f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x55af03ff6370 tx=0x7fbd340093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5dc55e6f1e3dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd1eff5640 1 -- 172.21.15.44:0/932903576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34014160 con 0x55af03ff14f0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd1eff5640 1 -- 172.21.15.44:0/932903576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9315+0+0 (secure 0 0 0) 0x7fbd34026070 con 0x55af03ff14f0 2024-09-13T17:33:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.832+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd3401a030 con 0x55af03ff14f0 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd1eff5640 1 -- 172.21.15.44:0/932903576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd340142c0 con 0x55af03ff14f0 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 10 client.17595 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55af03f108e0 con 0x55af03ff14f0 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd1eff5640 1 -- 172.21.15.44:0/932903576 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd3401a030 con 0x55af03ff14f0 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd1eff5640 1 -- 172.21.15.44:0/932903576 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd3403c070 con 0x55af03ff14f0 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 10 client.17595 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 10 client.17595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 20 client.17595 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04058710 0x55af0405ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 4 client.17595 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af03c3e300 con 0x55af04058710 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.833+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04058710 0x55af0405ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.834+0000 7fbd1eff5640 10 client.17595 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.834+0000 7fbd3a59c640 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04058710 0x55af0405ab30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529a8a14f5560466 server_cookie=1844648e033a1597 in_seq=0 out_seq=0 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.847+0000 7fbd1eff5640 1 -- 172.21.15.44:0/932903576 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af03c3e300 con 0x55af04058710 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.847+0000 7fbd1eff5640 10 client.17595 handle_command_reply: tid=0 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.847+0000 7fbd43e96740 1 client.17595 shutdown 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 msgr2=0x55af03ff70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff70f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x55af03ff6370 tx=0x7fbd340093c0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 shutdown_connections 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55af04058710 0x55af0405ab30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af03fe6c50 0x55af03ff6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af03ff14f0 0x55af03ff70f0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd1eff5640 0 client.17595 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 --2- 172.21.15.44:0/932903576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af03fe6330 0x55af03ff8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 >> 172.21.15.44:0/932903576 conn(0x55af03fe28c0 msgr2=0x55af03fec8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 shutdown_connections 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 1 -- 172.21.15.44:0/932903576 wait complete. 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd43e96740 20 client.17595 trim_cache size 0 max 16384 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:33:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:33:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbd240a1200 msgr2=0x7fbd240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbd240a1200 0x7fbd240a36c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0c9f70 tx=0x7fbd2c06b960 comp rx=0 tx=0).stop 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 msgr2=0x7fbd3c196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c196170 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00d950 tx=0x7fbd2c00de10 comp rx=0 tx=0).stop 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 shutdown_connections 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbd240a1200 0x7fbd240a36c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c0ff970 0x7fbd3c1966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c104c30 0x7fbd3c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 --2- 172.21.15.44:0/1797822456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd3c0feff0 0x7fbd3c196170 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.848+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 >> 172.21.15.44:0/1797822456 conn(0x7fbd3c074940 msgr2=0x7fbd3c067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.849+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 shutdown_connections 2024-09-13T17:33:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:00.849+0000 7fbd41176640 1 -- 172.21.15.44:0/1797822456 wait complete. 2024-09-13T17:33:02.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:02 smithi100 ceph-mon[35055]: pgmap v192: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:02.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:02 smithi044 ceph-mon[31175]: pgmap v192: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:02 smithi070 ceph-mon[34502]: pgmap v192: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:04 smithi100 ceph-mon[35055]: pgmap v193: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:04.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:04 smithi044 ceph-mon[31175]: pgmap v193: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:04.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:04 smithi070 ceph-mon[34502]: pgmap v193: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:05.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:05 smithi100 ceph-mon[35055]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T17:33:05.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:05 smithi044 ceph-mon[31175]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T17:33:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:05 smithi070 ceph-mon[34502]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T17:33:06.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:06 smithi100 ceph-mon[35055]: pgmap v195: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-13T17:33:06.598 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:06 smithi070 ceph-mon[34502]: pgmap v195: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-13T17:33:06.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:06 smithi044 ceph-mon[31175]: pgmap v195: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-13T17:33:08.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:08 smithi044 ceph-mon[31175]: pgmap v196: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-13T17:33:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:08 smithi070 ceph-mon[34502]: pgmap v196: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-13T17:33:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:08 smithi100 ceph-mon[35055]: pgmap v196: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-13T17:33:10.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:10 smithi044 ceph-mon[31175]: pgmap v197: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-13T17:33:10.610 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.610 INFO:tasks.workunit.client.0.smithi044.stdout:251/600: open fd = 3 2024-09-13T17:33:10.611 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.611 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.611 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.611 INFO:tasks.workunit.client.0.smithi044.stdout:252/600: open fd = 3 2024-09-13T17:33:10.611 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:253/600: open fd = 3 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.612 INFO:tasks.workunit.client.0.smithi044.stdout:254/600: open fd = 3 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:255/600: open fd = 3 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.613 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.614 INFO:tasks.workunit.client.0.smithi044.stdout:256/600: open fd = 3 2024-09-13T17:33:10.614 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.614 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.614 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.615 INFO:tasks.workunit.client.0.smithi044.stdout:257/600: open fd = 3 2024-09-13T17:33:10.615 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.615 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.615 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.615 INFO:tasks.workunit.client.0.smithi044.stdout:258/600: open fd = 3 2024-09-13T17:33:10.616 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.616 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.616 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.616 INFO:tasks.workunit.client.0.smithi044.stdout:259/600: open fd = 3 2024-09-13T17:33:10.616 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.616 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.617 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.617 INFO:tasks.workunit.client.0.smithi044.stdout:260/600: open fd = 3 2024-09-13T17:33:10.617 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.617 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.617 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.618 INFO:tasks.workunit.client.0.smithi044.stdout:261/600: open fd = 3 2024-09-13T17:33:10.618 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.618 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.619 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.619 INFO:tasks.workunit.client.0.smithi044.stdout:262/600: open fd = 3 2024-09-13T17:33:10.619 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:263/600: open fd = 3 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:264/600: open fd = 3 2024-09-13T17:33:10.620 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:265/600: open fd = 3 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:266/600: open fd = 3 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:267/600: open fd = 3 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.621 INFO:tasks.workunit.client.0.smithi044.stdout:268/600: open fd = 3 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:269/600: open fd = 3 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:270/600: open fd = 3 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:271/600: open fd = 3 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.622 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:272/600: open fd = 3 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:273/600: open fd = 3 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:274/600: open fd = 3 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.623 INFO:tasks.workunit.client.0.smithi044.stdout:275/600: open fd = 3 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:276/600: open fd = 3 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:277/600: open fd = 3 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:278/600: open fd = 3 2024-09-13T17:33:10.624 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:279/600: open fd = 3 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:280/600: open fd = 3 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:281/600: open fd = 3 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.625 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:282/600: open fd = 3 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:283/600: open fd = 3 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:284/600: open fd = 3 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:285/600: open fd = 3 2024-09-13T17:33:10.626 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:286/600: open fd = 3 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:287/600: open fd = 3 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:288/600: open fd = 3 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.627 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:289/600: open fd = 3 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:290/600: open fd = 3 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:291/600: open fd = 3 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:292/600: open fd = 3 2024-09-13T17:33:10.628 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:293/600: open fd = 3 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:294/600: open fd = 3 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.629 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:295/600: open fd = 3 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:296/600: open fd = 3 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:297/600: open fd = 3 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.630 INFO:tasks.workunit.client.0.smithi044.stdout:298/600: open fd = 3 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:299/600: open fd = 3 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:300/600: open fd = 3 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:301/600: open fd = 3 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.631 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:302/600: open fd = 3 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:303/600: open fd = 3 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:304/600: open fd = 3 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:305/600: open fd = 3 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.632 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:306/600: open fd = 3 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:307/600: open fd = 3 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:308/600: open fd = 3 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:309/600: open fd = 3 2024-09-13T17:33:10.633 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:310/600: open fd = 3 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:311/600: open fd = 3 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:312/600: open fd = 3 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.634 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:33:10.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:10 smithi070 ceph-mon[34502]: pgmap v197: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-13T17:33:10.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:10 smithi100 ceph-mon[35055]: pgmap v197: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-13T17:33:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:12 smithi100 ceph-mon[35055]: pgmap v198: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-13T17:33:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:12 smithi100 ceph-mon[35055]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T17:33:12.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:12 smithi044 ceph-mon[31175]: pgmap v198: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-13T17:33:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:12 smithi044 ceph-mon[31175]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T17:33:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:12 smithi070 ceph-mon[34502]: pgmap v198: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-13T17:33:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:12 smithi070 ceph-mon[34502]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T17:33:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/781542422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a940ff7d0 msgr2=0x7f6a94103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/781542422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a940ff7d0 0x7f6a94103810 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009930 tx=0x7f6a8002a420 comp rx=0 tx=0).stop 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/781542422 shutdown_connections 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/781542422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94103f70 0x7f6a9410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/781542422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a940ff7d0 0x7f6a94103810 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/781542422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a940fee20 0x7f6a940ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/781542422 >> 172.21.15.44:0/781542422 conn(0x7f6a940fa8f0 msgr2=0x7f6a940fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/781542422 shutdown_connections 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/781542422 wait complete. 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 Processor -- start 2024-09-13T17:33:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.946+0000 7f6a9ab5e640 1 -- start start 2024-09-13T17:33:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a940fee20 0x7f6a941008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94103f70 0x7f6a94100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 0x7f6a941a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941a4920 con 0x7f6a94101330 2024-09-13T17:33:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941a4a90 con 0x7f6a940fee20 2024-09-13T17:33:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941a4c50 con 0x7f6a94103f70 2024-09-13T17:33:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 0x7f6a941a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 0x7f6a941a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34124/0 (socket says 172.21.15.44:34124) 2024-09-13T17:33:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 -- 172.21.15.44:0/2160201293 learned_addr learned my addr 172.21.15.44:0/2160201293 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a988fa640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a940fee20 0x7f6a941008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a93fff640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94103f70 0x7f6a94100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 -- 172.21.15.44:0/2160201293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94103f70 msgr2=0x7f6a94100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94103f70 0x7f6a94100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 -- 172.21.15.44:0/2160201293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a940fee20 msgr2=0x7f6a941008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a940fee20 0x7f6a941008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 -- 172.21.15.44:0/2160201293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009560 con 0x7f6a94101330 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a990fb640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 0x7f6a941a4350 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00eb10 tx=0x7f6a8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a91ffb640 1 -- 172.21.15.44:0/2160201293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004b20 con 0x7f6a94101330 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a91ffb640 1 -- 172.21.15.44:0/2160201293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a8c004c80 con 0x7f6a94101330 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a941a4f30 con 0x7f6a94101330 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.947+0000 7f6a91ffb640 1 -- 172.21.15.44:0/2160201293 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c005a10 con 0x7f6a94101330 2024-09-13T17:33:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.948+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a941a5400 con 0x7f6a94101330 2024-09-13T17:33:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.950+0000 7f6a91ffb640 1 -- 172.21.15.44:0/2160201293 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6a8c005020 con 0x7f6a94101330 2024-09-13T17:33:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.950+0000 7f6a9d885740 1 Processor -- start 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 -- start start 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 0x564cb5751710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a91ffb640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a840a12b0 0x7f6a840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb575bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb575c4f0 0x564cb575e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb56bcc30 con 0x564cb5751330 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb56ba3c0 con 0x564cb575c4f0 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a9d885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb56b5cb0 con 0x564cb5751c50 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 0x564cb5751710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 0x564cb5751710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34134/0 (socket says 172.21.15.44:34134) 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a988fa640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a840a12b0 0x7f6a840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a93fff640 1 -- 172.21.15.44:0/3521046463 learned_addr learned my addr 172.21.15.44:0/3521046463 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a990fb640 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb575c4f0 0x564cb575e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a91ffb640 1 -- 172.21.15.44:0/2160201293 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6a8c014070 con 0x7f6a94101330 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a93fff640 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb575bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.951+0000 7f6a93fff640 1 -- 172.21.15.44:0/3521046463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 msgr2=0x564cb575bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a93fff640 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb575bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a93fff640 1 -- 172.21.15.44:0/3521046463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb575c4f0 msgr2=0x564cb575e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a93fff640 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb575c4f0 0x564cb575e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a93fff640 1 -- 172.21.15.44:0/3521046463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb565c910 con 0x564cb5751330 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a990fb640 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb575c4f0 0x564cb575e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a988fa640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a840a12b0 0x7f6a840a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009bd0 tx=0x7f6a88005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a93fff640 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 0x564cb5751710 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6a80005940 tx=0x7f6a80005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750104c6d7bef6c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a7effd640 1 -- 172.21.15.44:0/3521046463 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80044070 con 0x564cb5751330 2024-09-13T17:33:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a7effd640 1 -- 172.21.15.44:0/3521046463 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a80005d50 con 0x564cb5751330 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.952+0000 7f6a7effd640 1 -- 172.21.15.44:0/3521046463 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80003a10 con 0x564cb5751330 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 -- 172.21.15.44:0/3521046463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 msgr2=0x564cb5751710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 0x564cb5751710 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6a80005940 tx=0x7f6a80005720 comp rx=0 tx=0).stop 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 -- 172.21.15.44:0/3521046463 shutdown_connections 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb575bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb575c4f0 0x564cb575e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3521046463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb5751330 0x564cb5751710 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 -- 172.21.15.44:0/3521046463 >> 172.21.15.44:0/3521046463 conn(0x564cb574d8c0 msgr2=0x564cb574dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 -- 172.21.15.44:0/3521046463 shutdown_connections 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 -- 172.21.15.44:0/3521046463 wait complete. 2024-09-13T17:33:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 -- start start 2024-09-13T17:33:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.953+0000 7f6a9d885740 1 Processor -- start 2024-09-13T17:33:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb575bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5759240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb57599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb5673e80 con 0x564cb575c4f0 2024-09-13T17:33:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb5673b20 con 0x564cb5751330 2024-09-13T17:33:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb5639260 con 0x564cb5751c50 2024-09-13T17:33:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a9d885740 10 client.0 _ll_register_callbacks cb 0x564cb566edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb57599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a988fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb575bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb57599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34148/0 (socket says 172.21.15.44:34148) 2024-09-13T17:33:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 -- 172.21.15.44:0/3875600908 learned_addr learned my addr 172.21.15.44:0/3875600908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a93fff640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5759240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 -- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 msgr2=0x564cb5759240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5759240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 -- 172.21.15.44:0/3875600908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 msgr2=0x564cb575bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb575bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 -- 172.21.15.44:0/3875600908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9410c300 con 0x564cb575c4f0 2024-09-13T17:33:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a988fa640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb575bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a990fb640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb57599d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000f80 tx=0x7f6a8c099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc0fc6e0ef5f614 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a73fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a73fff640 1 -- 172.21.15.44:0/3875600908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0b2de0 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.954+0000 7f6a73fff640 1 -- 172.21.15.44:0/3875600908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6a8c004df0 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cb56de680 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a73fff640 1 -- 172.21.15.44:0/3875600908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c063d20 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 10 client.17607 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564cb56b4330 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a73fff640 1 -- 172.21.15.44:0/3875600908 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564cb56de680 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a73fff640 1 -- 172.21.15.44:0/3875600908 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a8c01f060 con 0x564cb575c4f0 2024-09-13T17:33:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 10 client.17607 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 20 client.17607 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb576c000 0x564cb576e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 4 client.17607 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cb56a95d0 con 0x564cb576c000 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.955+0000 7f6a988fa640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb576c000 0x564cb576e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.956+0000 7f6a73fff640 10 client.17607 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.956+0000 7f6a988fa640 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb576c000 0x564cb576e420 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46999f59052dde5 server_cookie=69bc98bf12af98ea in_seq=0 out_seq=0 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.956+0000 7f6a73fff640 1 -- 172.21.15.44:0/3875600908 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564cb56a95d0 con 0x564cb576c000 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.956+0000 7f6a73fff640 10 client.17607 handle_command_reply: tid=0 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.956+0000 7f6a9d885740 1 client.17607 shutdown 2024-09-13T17:33:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 msgr2=0x564cb57599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb57599d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000f80 tx=0x7f6a8c099d70 comp rx=0 tx=0).stop 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 shutdown_connections 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb576c000 0x564cb576e420 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5759240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb57599d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a73fff640 0 client.17607 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 --2- 172.21.15.44:0/3875600908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb575bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 >> 172.21.15.44:0/3875600908 conn(0x564cb574d8c0 msgr2=0x564cb574cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 shutdown_connections 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 1 -- 172.21.15.44:0/3875600908 wait complete. 2024-09-13T17:33:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.957+0000 7f6a9d885740 20 client.17607 trim_cache size 0 max 16384 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.969+0000 7f6a9d885740 1 Processor -- start 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.969+0000 7f6a9d885740 1 -- start start 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb574d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5754550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5754a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a9d885740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb576a870 con 0x564cb575c4f0 2024-09-13T17:33:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a9d885740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb576be20 con 0x564cb5751330 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a9d885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb566ed10 con 0x564cb5751c50 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5754a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5754a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34160/0 (socket says 172.21.15.44:34160) 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 -- 172.21.15.44:0/2743394658 learned_addr learned my addr 172.21.15.44:0/2743394658 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a988fa640 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb574d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a93fff640 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5754550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 -- 172.21.15.44:0/2743394658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 msgr2=0x564cb5754550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5754550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 -- 172.21.15.44:0/2743394658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 msgr2=0x564cb574d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb574d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 -- 172.21.15.44:0/2743394658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb5656300 con 0x564cb575c4f0 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a988fa640 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb574d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a990fb640 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5754a90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c096730 tx=0x7f6a8c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beca8295c75f5241 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a7f7fe640 1 -- 172.21.15.44:0/2743394658 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c096570 con 0x564cb575c4f0 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a7f7fe640 1 -- 172.21.15.44:0/2743394658 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a8c099b90 con 0x564cb575c4f0 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.970+0000 7f6a7f7fe640 1 -- 172.21.15.44:0/2743394658 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0a26b0 con 0x564cb575c4f0 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 -- 172.21.15.44:0/2743394658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 msgr2=0x564cb5754a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5754a90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c096730 tx=0x7f6a8c00efd0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 -- 172.21.15.44:0/2743394658 shutdown_connections 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5754550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5754a90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 --2- 172.21.15.44:0/2743394658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb574d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 -- 172.21.15.44:0/2743394658 >> 172.21.15.44:0/2743394658 conn(0x564cb574d8c0 msgr2=0x564cb5772d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 -- 172.21.15.44:0/2743394658 shutdown_connections 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 -- 172.21.15.44:0/2743394658 wait complete. 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 -- start start 2024-09-13T17:33:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.971+0000 7f6a9d885740 1 Processor -- start 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb57631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5763700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5755ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb57563e0 con 0x564cb575c4f0 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb5756550 con 0x564cb5751330 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb5756710 con 0x564cb5751c50 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a9d885740 10 client.0 _ll_register_callbacks cb 0x7f6a88005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5755ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5755ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34172/0 (socket says 172.21.15.44:34172) 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a988fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb57631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 -- 172.21.15.44:0/4102058350 learned_addr learned my addr 172.21.15.44:0/4102058350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a93fff640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5763700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 -- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 msgr2=0x564cb5763700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5763700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 -- 172.21.15.44:0/4102058350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 msgr2=0x564cb57631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb57631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a990fb640 1 -- 172.21.15.44:0/4102058350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cb56b4330 con 0x564cb575c4f0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.972+0000 7f6a988fa640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb57631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a990fb640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5755ce0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c085040 tx=0x7f6a8c099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccc73bd32542ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a73fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a73fff640 1 -- 172.21.15.44:0/4102058350 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004df0 con 0x564cb575c4f0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a73fff640 1 -- 172.21.15.44:0/4102058350 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6a8c094e40 con 0x564cb575c4f0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a80005d50 con 0x564cb575c4f0 2024-09-13T17:33:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a73fff640 1 -- 172.21.15.44:0/4102058350 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c0b2de0 con 0x564cb575c4f0 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 10 client.17613 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564cb567b8e0 con 0x564cb575c4f0 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a73fff640 1 -- 172.21.15.44:0/4102058350 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a8c0ab080 con 0x564cb575c4f0 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a73fff640 1 -- 172.21.15.44:0/4102058350 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a8c0052d0 con 0x564cb575c4f0 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 10 client.17613 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 20 client.17613 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb57c3470 0x564cb57c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 4 client.17613 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.973+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564cb53a9300 con 0x564cb57c3470 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.974+0000 7f6a988fa640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb57c3470 0x564cb57c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.974+0000 7f6a73fff640 10 client.17613 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.974+0000 7f6a988fa640 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb57c3470 0x564cb57c5860 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17618a1dabe0655 server_cookie=a9b72afd49f49f68 in_seq=0 out_seq=0 2024-09-13T17:33:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a73fff640 1 -- 172.21.15.44:0/4102058350 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564cb53a9300 con 0x564cb57c3470 2024-09-13T17:33:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a73fff640 10 client.17613 handle_command_reply: tid=0 2024-09-13T17:33:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 client.17613 shutdown 2024-09-13T17:33:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 msgr2=0x564cb5755ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5755ce0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c085040 tx=0x7f6a8c099d70 comp rx=0 tx=0).stop 2024-09-13T17:33:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 shutdown_connections 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x564cb57c3470 0x564cb57c5860 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cb5751c50 0x564cb5763700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb575c4f0 0x564cb5755ce0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 --2- 172.21.15.44:0/4102058350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564cb5751330 0x564cb57631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a73fff640 0 client.17613 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 >> 172.21.15.44:0/4102058350 conn(0x564cb574d8c0 msgr2=0x564cb5754550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 shutdown_connections 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 1 -- 172.21.15.44:0/4102058350 wait complete. 2024-09-13T17:33:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.990+0000 7f6a9d885740 20 client.17613 trim_cache size 0 max 16384 2024-09-13T17:33:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:33:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:33:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:33:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a840a12b0 msgr2=0x7f6a840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a840a12b0 0x7f6a840a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009bd0 tx=0x7f6a88005f90 comp rx=0 tx=0).stop 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 msgr2=0x7f6a941a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 0x7f6a941a4350 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00eb10 tx=0x7f6a8c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 shutdown_connections 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6a840a12b0 0x7f6a840a3770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94103f70 0x7f6a94100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a94101330 0x7f6a941a4350 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 --2- 172.21.15.44:0/2160201293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a940fee20 0x7f6a941008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 >> 172.21.15.44:0/2160201293 conn(0x7f6a940fa8f0 msgr2=0x7f6a94068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 shutdown_connections 2024-09-13T17:33:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:13.991+0000 7f6a9ab5e640 1 -- 172.21.15.44:0/2160201293 wait complete. 2024-09-13T17:33:14.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:14 smithi100 ceph-mon[35055]: pgmap v200: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 7 op/s 2024-09-13T17:33:14.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:14 smithi044 ceph-mon[31175]: pgmap v200: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 7 op/s 2024-09-13T17:33:14.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:14 smithi070 ceph-mon[34502]: pgmap v200: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 7 op/s 2024-09-13T17:33:16.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:16 smithi100 ceph-mon[35055]: pgmap v201: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-13T17:33:16.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:16 smithi044 ceph-mon[31175]: pgmap v201: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-13T17:33:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:16 smithi070 ceph-mon[34502]: pgmap v201: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-13T17:33:17.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:33:17.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 -- 172.21.15.44:0/741798762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 msgr2=0x7fb4c00fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 --2- 172.21.15.44:0/741798762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c00fe8d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009930 tx=0x7fb4bc02a420 comp rx=0 tx=0).stop 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 -- 172.21.15.44:0/741798762 shutdown_connections 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 --2- 172.21.15.44:0/741798762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105100 0x7fb4c010cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 --2- 172.21.15.44:0/741798762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4c00feea0 0x7fb4c01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 --2- 172.21.15.44:0/741798762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c00fe8d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 -- 172.21.15.44:0/741798762 >> 172.21.15.44:0/741798762 conn(0x7fb4c00753c0 msgr2=0x7fb4c00fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 -- 172.21.15.44:0/741798762 shutdown_connections 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.872+0000 7fb4c7908640 1 -- 172.21.15.44:0/741798762 wait complete. 2024-09-13T17:33:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 Processor -- start 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 -- start start 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105100 0x7fb4c01967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4c019abb0 0x7fb4c019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c019b5a0 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c019b710 con 0x7fb4c019abb0 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c7908640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c019b880 con 0x7fb4c0105100 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c0196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34192/0 (socket says 172.21.15.44:34192) 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3043277570 learned_addr learned my addr 172.21.15.44:0/3043277570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4c019abb0 0x7fb4c019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.873+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105100 0x7fb4c01967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3043277570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105100 msgr2=0x7fb4c01967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105100 0x7fb4c01967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3043277570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4c019abb0 msgr2=0x7fb4c019b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4c019abb0 0x7fb4c019b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3043277570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009560 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c0196280 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005ab0 tx=0x7fb4bc032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4b67fc640 1 -- 172.21.15.44:0/3043277570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc032e10 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4b67fc640 1 -- 172.21.15.44:0/3043277570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4bc0046c0 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c01a2e60 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4b67fc640 1 -- 172.21.15.44:0/3043277570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc003850 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.874+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c01a3280 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.876+0000 7fb4b67fc640 1 -- 172.21.15.44:0/3043277570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4bc004830 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 Processor -- start 2024-09-13T17:33:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 -- start start 2024-09-13T17:33:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a525f20 con 0x55718a5bb730 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5236b0 con 0x55718a5c5fd0 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a51efa0 con 0x55718a5bae10 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4b67fc640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4ac0a1280 0x7fb4ac0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4c56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4c4ea3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4ac0a1280 0x7fb4ac0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4c56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34206/0 (socket says 172.21.15.44:34206) 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3025614836 learned_addr learned my addr 172.21.15.44:0/3025614836 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.877+0000 7fb4c4ea3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:49156/0 (socket says 172.21.15.44:49156) 2024-09-13T17:33:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4b67fc640 1 -- 172.21.15.44:0/3043277570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb4bc0edca0 con 0x7fb4c00fe4f0 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3025614836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 msgr2=0x55718a5bb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5bb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3025614836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 msgr2=0x55718a5c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3025614836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718a4d9840 con 0x55718a5c5fd0 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4ac0a1280 0x7fb4ac0a3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80098b0 tx=0x7fb4b8009d90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c83c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0005fd0 tx=0x7fb4b000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e82e5584d32f608 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4ab7fe640 1 -- 172.21.15.44:0/3025614836 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000dea0 con 0x55718a5c5fd0 2024-09-13T17:33:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4ab7fe640 1 -- 172.21.15.44:0/3025614836 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4b00054b0 con 0x55718a5c5fd0 2024-09-13T17:33:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.878+0000 7fb4ab7fe640 1 -- 172.21.15.44:0/3025614836 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0004e70 con 0x55718a5c5fd0 2024-09-13T17:33:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3025614836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 msgr2=0x55718a5c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c83c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0005fd0 tx=0x7fb4b000f730 comp rx=0 tx=0).stop 2024-09-13T17:33:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3025614836 shutdown_connections 2024-09-13T17:33:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c83c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3025614836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5bb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3025614836 >> 172.21.15.44:0/3025614836 conn(0x55718a5b7380 msgr2=0x55718a5b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3025614836 shutdown_connections 2024-09-13T17:33:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3025614836 wait complete. 2024-09-13T17:33:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.879+0000 7fb4ca62d740 1 -- start start 2024-09-13T17:33:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 Processor -- start 2024-09-13T17:33:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 0x55718a5c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bb730 0x55718a5c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a4c02a0 con 0x55718a5bae10 2024-09-13T17:33:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a1f7430 con 0x55718a5bb730 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a4a2680 con 0x55718a5c5fd0 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4ca62d740 10 client.0 _ll_register_callbacks cb 0x55718a518610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 0x55718a5c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 0x55718a5c58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34220/0 (socket says 172.21.15.44:34220) 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3660633631 learned_addr learned my addr 172.21.15.44:0/3660633631 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bb730 0x55718a5c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 msgr2=0x55718a5c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3660633631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bb730 msgr2=0x55718a5c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bb730 0x55718a5c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c56a4640 1 -- 172.21.15.44:0/3660633631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b000d840 con 0x55718a5bae10 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.880+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 0x55718a5c58f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0b6e90 tx=0x7fb4bc08a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1763688221371538 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3660633631 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0ba9e0 con 0x55718a5bae10 2024-09-13T17:33:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3660633631 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb4bc0be810 con 0x55718a5bae10 2024-09-13T17:33:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55718a2124e0 con 0x55718a5bae10 2024-09-13T17:33:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3660633631 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4bc0c2e60 con 0x55718a5bae10 2024-09-13T17:33:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 10 client.17625 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55718a51d620 con 0x55718a5bae10 2024-09-13T17:33:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3660633631 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4bc0d2070 con 0x55718a5bae10 2024-09-13T17:33:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3660633631 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4bc0bac40 con 0x55718a5bae10 2024-09-13T17:33:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 10 client.17625 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 20 client.17625 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a5d5a70 0x55718a5d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 4 client.17625 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.881+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55718a2237e0 con 0x55718a5d5a70 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.882+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a5d5a70 0x55718a5d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.882+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a5d5a70 0x55718a5d7e60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0553bf59655be1f server_cookie=8b1c09238eca28ff in_seq=0 out_seq=0 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.882+0000 7fb4a8ff9640 10 client.17625 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3660633631 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55718a2237e0 con 0x55718a5d5a70 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4a8ff9640 10 client.17625 handle_command_reply: tid=0 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 client.17625 shutdown 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 msgr2=0x55718a5c58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 0x55718a5c58f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0b6e90 tx=0x7fb4bc08a960 comp rx=0 tx=0).stop 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 shutdown_connections 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a5d5a70 0x55718a5d7e60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bb730 0x55718a5c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4a8ff9640 0 client.17625 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3660633631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bae10 0x55718a5c58f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 >> 172.21.15.44:0/3660633631 conn(0x55718a5b7380 msgr2=0x55718a5bd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 shutdown_connections 2024-09-13T17:33:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3660633631 wait complete. 2024-09-13T17:33:17.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.883+0000 7fb4ca62d740 20 client.17625 trim_cache size 0 max 16384 2024-09-13T17:33:17.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 Processor -- start 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 -- start start 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bae10 0x55718a5d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5c34e0 con 0x55718a5bb730 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5bd9e0 con 0x55718a5bae10 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5c1160 con 0x55718a5c5fd0 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34232/0 (socket says 172.21.15.44:34232) 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/214843327 learned_addr learned my addr 172.21.15.44:0/214843327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/214843327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 msgr2=0x55718a5cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bae10 0x55718a5d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/214843327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bae10 msgr2=0x55718a5d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bae10 0x55718a5d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/214843327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718a4bf790 con 0x55718a5bb730 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bae10 0x55718a5d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5ccd60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0004a20 tx=0x7fb4b001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82c1df9b01a1050 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.898+0000 7fb4abfff640 1 -- 172.21.15.44:0/214843327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0022070 con 0x55718a5bb730 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4abfff640 1 -- 172.21.15.44:0/214843327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4b0023040 con 0x55718a5bb730 2024-09-13T17:33:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4abfff640 1 -- 172.21.15.44:0/214843327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0029690 con 0x55718a5bb730 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 -- 172.21.15.44:0/214843327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 msgr2=0x55718a5ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5ccd60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0004a20 tx=0x7fb4b001f040 comp rx=0 tx=0).stop 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 -- 172.21.15.44:0/214843327 shutdown_connections 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5c5fd0 0x55718a5cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5ccd60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/214843327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5bae10 0x55718a5d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 -- 172.21.15.44:0/214843327 >> 172.21.15.44:0/214843327 conn(0x55718a5b7380 msgr2=0x55718a5dc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 -- 172.21.15.44:0/214843327 shutdown_connections 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 -- 172.21.15.44:0/214843327 wait complete. 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.899+0000 7fb4ca62d740 1 -- start start 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 Processor -- start 2024-09-13T17:33:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5c9e20 con 0x55718a5bb730 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5c9f90 con 0x55718a5c5fd0 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5ca150 con 0x55718a5bae10 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4ca62d740 10 client.0 _ll_register_callbacks cb 0x7fb4bc0eafc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34248/0 (socket says 172.21.15.44:34248) 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3834666959 learned_addr learned my addr 172.21.15.44:0/3834666959 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c5ea5640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 msgr2=0x55718a5ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3834666959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 msgr2=0x55718a5c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.900+0000 7fb4c4ea3640 1 -- 172.21.15.44:0/3834666959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55718a4d9840 con 0x55718a5bb730 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4c4ea3640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c9320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002d000 tx=0x7fb4b000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c4f51abe5dbef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3834666959 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0022040 con 0x55718a5bb730 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3834666959 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb4b0004230 con 0x55718a5bb730 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4b00054b0 con 0x55718a5bb730 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3834666959 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4b000dca0 con 0x55718a5bb730 2024-09-13T17:33:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 10 client.17631 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55718a4e4ea0 con 0x55718a5bb730 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3834666959 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4b00054b0 con 0x55718a5bb730 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3834666959 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4b0004e20 con 0x55718a5bb730 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 10 client.17631 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 20 client.17631 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a62d060 0x55718a62f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 4 client.17631 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.901+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55718a276e30 con 0x55718a62d060 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a62d060 0x55718a62f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4c56a4640 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a62d060 0x55718a62f480 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead6c66c49a2623d server_cookie=d55b73cf78b613d2 in_seq=0 out_seq=0 2024-09-13T17:33:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4a8ff9640 10 client.17631 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4a8ff9640 1 -- 172.21.15.44:0/3834666959 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55718a276e30 con 0x55718a62d060 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4a8ff9640 10 client.17631 handle_command_reply: tid=0 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 client.17631 shutdown 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 msgr2=0x55718a5c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c9320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002d000 tx=0x7fb4b000ff30 comp rx=0 tx=0).stop 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 shutdown_connections 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55718a62d060 0x55718a62f480 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55718a5c5fd0 0x55718a5c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4a8ff9640 0 client.17631 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55718a5bb730 0x55718a5c9320 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 --2- 172.21.15.44:0/3834666959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718a5bae10 0x55718a5ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.902+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 >> 172.21.15.44:0/3834666959 conn(0x55718a5b7380 msgr2=0x55718a5bdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 shutdown_connections 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4ca62d740 1 -- 172.21.15.44:0/3834666959 wait complete. 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4ca62d740 20 client.17631 trim_cache size 0 max 16384 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:33:17.913 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:33:17.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4ac0a1280 msgr2=0x7fb4ac0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4ac0a1280 0x7fb4ac0a3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80098b0 tx=0x7fb4b8009d90 comp rx=0 tx=0).stop 2024-09-13T17:33:17.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 msgr2=0x7fb4c0196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:17.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c0196280 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005ab0 tx=0x7fb4bc032c20 comp rx=0 tx=0).stop 2024-09-13T17:33:17.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 shutdown_connections 2024-09-13T17:33:17.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4ac0a1280 0x7fb4ac0a3740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105100 0x7fb4c01967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4c019abb0 0x7fb4c019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 --2- 172.21.15.44:0/3043277570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4c00fe4f0 0x7fb4c0196280 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 >> 172.21.15.44:0/3043277570 conn(0x7fb4c00753c0 msgr2=0x7fb4c01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 shutdown_connections 2024-09-13T17:33:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:17.903+0000 7fb4c7908640 1 -- 172.21.15.44:0/3043277570 wait complete. 2024-09-13T17:33:17.916 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:33:17.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 -- 172.21.15.44:0/1099913494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c740fef00 msgr2=0x7f6c740ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 --2- 172.21.15.44:0/1099913494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c740fef00 0x7f6c740ff2e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402a410 comp rx=0 tx=0).stop 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 -- 172.21.15.44:0/1099913494 shutdown_connections 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 --2- 172.21.15.44:0/1099913494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74104050 0x7f6c7410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 --2- 172.21.15.44:0/1099913494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c740ff8b0 0x7f6c741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 --2- 172.21.15.44:0/1099913494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c740fef00 0x7f6c740ff2e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 -- 172.21.15.44:0/1099913494 >> 172.21.15.44:0/1099913494 conn(0x7f6c740fa950 msgr2=0x7f6c740fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 -- 172.21.15.44:0/1099913494 shutdown_connections 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.018+0000 7f6c79412640 1 -- 172.21.15.44:0/1099913494 wait complete. 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 Processor -- start 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 -- start start 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c740fef00 0x7f6c74196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 0x7f6c74196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7419ab30 0x7f6c7419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419b520 con 0x7f6c74104050 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419b690 con 0x7f6c7419ab30 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c79412640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419b800 con 0x7f6c740fef00 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 0x7f6c74196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 0x7f6c74196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34272/0 (socket says 172.21.15.44:34272) 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 -- 172.21.15.44:0/1522300062 learned_addr learned my addr 172.21.15.44:0/1522300062 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c737fe640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7419ab30 0x7f6c7419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c72ffd640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c740fef00 0x7f6c74196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 -- 172.21.15.44:0/1522300062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c740fef00 msgr2=0x7f6c74196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c740fef00 0x7f6c74196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 -- 172.21.15.44:0/1522300062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7419ab30 msgr2=0x7f6c7419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7419ab30 0x7f6c7419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.019+0000 7f6c727fc640 1 -- 172.21.15.44:0/1522300062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009560 con 0x7f6c74104050 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.020+0000 7f6c727fc640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 0x7f6c74196740 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6c68005fd0 tx=0x7f6c6800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.020+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68009c30 con 0x7f6c74104050 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.020+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c741a2de0 con 0x7f6c74104050 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.020+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c680044b0 con 0x7f6c74104050 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.020+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68003da0 con 0x7f6c74104050 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.020+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c741a3230 con 0x7f6c74104050 2024-09-13T17:33:18.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.021+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c741003d0 con 0x7f6c74104050 2024-09-13T17:33:18.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.022+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6c68003f00 con 0x7f6c74104050 2024-09-13T17:33:18.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.023+0000 7f6c5bfff640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c5c0a1200 0x7f6c5c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.023+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6c680cd480 con 0x7f6c74104050 2024-09-13T17:33:18.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.023+0000 7f6c72ffd640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c5c0a1200 0x7f6c5c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.023+0000 7f6c72ffd640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c5c0a1200 0x7f6c5c0a36c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c64005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.026+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6c68096560 con 0x7f6c74104050 2024-09-13T17:33:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.234+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c740ff2e0 con 0x7f6c74104050 2024-09-13T17:33:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.235+0000 7f6c5bfff640 1 -- 172.21.15.44:0/1522300062 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f6c68017020 con 0x7f6c74104050 2024-09-13T17:33:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:33:18.236 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:33:18.236 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:33:18.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c5c0a1200 msgr2=0x7f6c5c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c5c0a1200 0x7f6c5c0a36c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c64005a20 comp rx=0 tx=0).stop 2024-09-13T17:33:18.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 msgr2=0x7f6c74196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 0x7f6c74196740 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6c68005fd0 tx=0x7f6c6800dde0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 shutdown_connections 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6c5c0a1200 0x7f6c5c0a36c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7419ab30 0x7f6c7419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74104050 0x7f6c74196740 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 --2- 172.21.15.44:0/1522300062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c740fef00 0x7f6c74196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 >> 172.21.15.44:0/1522300062 conn(0x7f6c740fa950 msgr2=0x7f6c740fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 shutdown_connections 2024-09-13T17:33:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.237+0000 7f6c79412640 1 -- 172.21.15.44:0/1522300062 wait complete. 2024-09-13T17:33:18.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:33:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 -- 172.21.15.44:0/2263140609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 msgr2=0x7f1f180ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/2263140609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f180ff260 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009930 tx=0x7f1f0802a420 comp rx=0 tx=0).stop 2024-09-13T17:33:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 -- 172.21.15.44:0/2263140609 shutdown_connections 2024-09-13T17:33:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/2263140609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f18103fd0 0x7f1f1810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/2263140609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f180ff830 0x7f1f18103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/2263140609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f180ff260 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 -- 172.21.15.44:0/2263140609 >> 172.21.15.44:0/2263140609 conn(0x7f1f180fa930 msgr2=0x7f1f180fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 -- 172.21.15.44:0/2263140609 shutdown_connections 2024-09-13T17:33:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.352+0000 7f1f1cd26640 1 -- 172.21.15.44:0/2263140609 wait complete. 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 Processor -- start 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 -- start start 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f18196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f18103fd0 0x7f1f18197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f18197a70 0x7f1f181a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a2710 con 0x7f1f180fee80 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a2880 con 0x7f1f18197a70 2024-09-13T17:33:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1cd26640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a2a40 con 0x7f1f18103fd0 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f18196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f18196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34300/0 (socket says 172.21.15.44:34300) 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 -- 172.21.15.44:0/1507595545 learned_addr learned my addr 172.21.15.44:0/1507595545 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f18103fd0 0x7f1f18197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f18197a70 0x7f1f181a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 -- 172.21.15.44:0/1507595545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f18103fd0 msgr2=0x7f1f18197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f18103fd0 0x7f1f18197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 -- 172.21.15.44:0/1507595545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f18197a70 msgr2=0x7f1f181a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f18197a70 0x7f1f181a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.353+0000 7f1f1659c640 1 -- 172.21.15.44:0/1507595545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08009560 con 0x7f1f180fee80 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.354+0000 7f1f1659c640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f18196f50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802aa10 tx=0x7f1f08032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.354+0000 7f1f0f7fe640 1 -- 172.21.15.44:0/1507595545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08005210 con 0x7f1f180fee80 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.354+0000 7f1f0f7fe640 1 -- 172.21.15.44:0/1507595545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f08005370 con 0x7f1f180fee80 2024-09-13T17:33:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.354+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f181a2cc0 con 0x7f1f180fee80 2024-09-13T17:33:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.354+0000 7f1f0f7fe640 1 -- 172.21.15.44:0/1507595545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080038e0 con 0x7f1f180fee80 2024-09-13T17:33:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.354+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f181a31c0 con 0x7f1f180fee80 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.356+0000 7f1f0f7fe640 1 -- 172.21.15.44:0/1507595545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1f08003a40 con 0x7f1f180fee80 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.356+0000 7f1f1fa48740 1 Processor -- start 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 -- start start 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f54feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f5509950 0x55e7f550bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f5468cc0 con 0x55e7f54ff0b0 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f5466450 con 0x55e7f54fe790 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f5461d40 con 0x55e7f5509950 2024-09-13T17:33:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34310/0 (socket says 172.21.15.44:34310) 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 -- 172.21.15.44:0/3709285958 learned_addr learned my addr 172.21.15.44:0/3709285958 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f5509950 0x55e7f550bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f54feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f0f7fe640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1f040a1280 0x7f1f040a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 -- 172.21.15.44:0/3709285958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f5509950 msgr2=0x55e7f550bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f5509950 0x55e7f550bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 -- 172.21.15.44:0/3709285958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 msgr2=0x55e7f54feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f54feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f1659c640 1 -- 172.21.15.44:0/3709285958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f5429620 con 0x55e7f54ff0b0 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.357+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1f040a1280 0x7f1f040a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.358+0000 7f1f0f7fe640 1 -- 172.21.15.44:0/1507595545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f1f080edd80 con 0x7f1f180fee80 2024-09-13T17:33:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.358+0000 7f1f1659c640 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509410 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1f080eb0d0 tx=0x7f1f08085fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb06c2caa5a54ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.358+0000 7f1f0cff9640 1 -- 172.21.15.44:0/3709285958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080ba8f0 con 0x55e7f54ff0b0 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.358+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1f040a1280 0x7f1f040a3740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007f30 tx=0x7f1f10004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.358+0000 7f1f0cff9640 1 -- 172.21.15.44:0/3709285958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f080ba660 con 0x55e7f54ff0b0 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.358+0000 7f1f0cff9640 1 -- 172.21.15.44:0/3709285958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080ba480 con 0x55e7f54ff0b0 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3709285958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 msgr2=0x55e7f5509410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509410 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1f080eb0d0 tx=0x7f1f08085fb0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3709285958 shutdown_connections 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f5509950 0x55e7f550bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509410 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3709285958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f54feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3709285958 >> 172.21.15.44:0/3709285958 conn(0x55e7f54fad00 msgr2=0x55e7f54fb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3709285958 shutdown_connections 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3709285958 wait complete. 2024-09-13T17:33:18.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- start start 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 Processor -- start 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f5508e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 0x55e7f55062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f5420180 con 0x55e7f5509950 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f513a430 con 0x55e7f54fe790 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f53e55e0 con 0x55e7f54ff0b0 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.359+0000 7f1f1fa48740 10 client.0 _ll_register_callbacks cb 0x55e7f5451aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 0x55e7f55062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 0x55e7f55062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34316/0 (socket says 172.21.15.44:34316) 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 -- 172.21.15.44:0/3977245513 learned_addr learned my addr 172.21.15.44:0/3977245513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f1659c640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f5508e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 -- 172.21.15.44:0/3977245513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 msgr2=0x55e7f5509350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 -- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 msgr2=0x55e7f5508e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f5508e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 -- 172.21.15.44:0/3977245513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f181a31c0 con 0x55e7f5509950 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 0x55e7f55062a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1f100049d0 tx=0x7f1f10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74942e8306a43ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1efdffb640 1 -- 172.21.15.44:0/3977245513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10010db0 con 0x55e7f5509950 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1efdffb640 1 -- 172.21.15.44:0/3977245513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f1f1001f430 con 0x55e7f5509950 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7f548bb80 con 0x55e7f5509950 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1efdffb640 1 -- 172.21.15.44:0/3977245513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1001f5f0 con 0x55e7f5509950 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f1fa48740 10 client.17649 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.360+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e7f54603c0 con 0x55e7f5509950 2024-09-13T17:33:18.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1efdffb640 1 -- 172.21.15.44:0/3977245513 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e7f548bb80 con 0x55e7f5509950 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1efdffb640 1 -- 172.21.15.44:0/3977245513 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f1001a070 con 0x55e7f5509950 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1fa48740 10 client.17649 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1fa48740 10 client.17649 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1fa48740 20 client.17649 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5518fe0 0x55e7f551b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1fa48740 4 client.17649 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7f54541f0 con 0x55e7f5518fe0 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.361+0000 7f1f1659c640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5518fe0 0x55e7f551b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1659c640 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5518fe0 0x55e7f551b400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7e3f0ecac337db server_cookie=92d769cead833c in_seq=0 out_seq=0 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1efdffb640 10 client.17649 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1efdffb640 1 -- 172.21.15.44:0/3977245513 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e7f54541f0 con 0x55e7f5518fe0 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1efdffb640 10 client.17649 handle_command_reply: tid=0 2024-09-13T17:33:18.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 client.17649 shutdown 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 msgr2=0x55e7f55062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 0x55e7f55062a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1f100049d0 tx=0x7f1f10004ab0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 shutdown_connections 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5518fe0 0x55e7f551b400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f5509350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1efdffb640 0 client.17649 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f5509950 0x55e7f55062a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/3977245513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f54fe790 0x55e7f5508e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 >> 172.21.15.44:0/3977245513 conn(0x55e7f54fad00 msgr2=0x55e7f5500a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.362+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 shutdown_connections 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.363+0000 7f1f1fa48740 1 -- 172.21.15.44:0/3977245513 wait complete. 2024-09-13T17:33:18.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.363+0000 7f1f1fa48740 20 client.17649 trim_cache size 0 max 16384 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 Processor -- start 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 -- start start 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54fe790 0x55e7f55103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f55108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f5501360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f5515610 con 0x55e7f54ff0b0 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f5520630 con 0x55e7f5509950 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f54ff620 con 0x55e7f54fe790 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f55108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f55108e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34320/0 (socket says 172.21.15.44:34320) 2024-09-13T17:33:18.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 -- 172.21.15.44:0/2216845759 learned_addr learned my addr 172.21.15.44:0/2216845759 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f1659c640 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54fe790 0x55e7f55103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f5501360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 -- 172.21.15.44:0/2216845759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54fe790 msgr2=0x55e7f55103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54fe790 0x55e7f55103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 -- 172.21.15.44:0/2216845759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 msgr2=0x55e7f5501360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f5501360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.374+0000 7f1f15d9b640 1 -- 172.21.15.44:0/2216845759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f5402770 con 0x55e7f54ff0b0 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f55108e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1f18069730 tx=0x7f1f00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa6fc7107762c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f0d7fa640 1 -- 172.21.15.44:0/2216845759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00016070 con 0x55e7f54ff0b0 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f0d7fa640 1 -- 172.21.15.44:0/2216845759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f000044b0 con 0x55e7f54ff0b0 2024-09-13T17:33:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f0d7fa640 1 -- 172.21.15.44:0/2216845759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00004a00 con 0x55e7f54ff0b0 2024-09-13T17:33:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 -- 172.21.15.44:0/2216845759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 msgr2=0x55e7f55108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f55108e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1f18069730 tx=0x7f1f00011810 comp rx=0 tx=0).stop 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 -- 172.21.15.44:0/2216845759 shutdown_connections 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f5501360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54ff0b0 0x55e7f55108e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/2216845759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54fe790 0x55e7f55103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 -- 172.21.15.44:0/2216845759 >> 172.21.15.44:0/2216845759 conn(0x55e7f54fad00 msgr2=0x55e7f5504a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 -- 172.21.15.44:0/2216845759 shutdown_connections 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.375+0000 7f1f1fa48740 1 -- 172.21.15.44:0/2216845759 wait complete. 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 -- start start 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 Processor -- start 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 0x55e7f55107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f550ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f550f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f550f860 con 0x55e7f54fe790 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f550f9d0 con 0x55e7f5509950 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f556f510 con 0x55e7f54ff0b0 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1fa48740 10 client.0 _ll_register_callbacks cb 0x7f1f080f1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 0x55e7f55107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 0x55e7f55107c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34332/0 (socket says 172.21.15.44:34332) 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 -- 172.21.15.44:0/1496046041 learned_addr learned my addr 172.21.15.44:0/1496046041 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f16d9d640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f550f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f550ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 -- 172.21.15.44:0/1496046041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 msgr2=0x55e7f550ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f550ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 -- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 msgr2=0x55e7f550f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f550f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f1659c640 1 -- 172.21.15.44:0/1496046041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0000c930 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.376+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f550ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1659c640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 0x55e7f55107c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1f080b8e70 tx=0x7f1f08009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8249c8ea6b2622d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1efdffb640 1 -- 172.21.15.44:0/1496046041 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080ba480 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1efdffb640 1 -- 172.21.15.44:0/1496046041 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f1f080bb220 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f080ba660 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1efdffb640 1 -- 172.21.15.44:0/1496046041 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f080bab20 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 10 client.17655 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e7f5427e80 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1efdffb640 1 -- 172.21.15.44:0/1496046041 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f080d2030 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1efdffb640 1 -- 172.21.15.44:0/1496046041 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f080eb3e0 con 0x55e7f54fe790 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 10 client.17655 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 10 client.17655 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 20 client.17655 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5570940 0x55e7f5572d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 4 client.17655 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.377+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7f51554e0 con 0x55e7f5570940 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5570940 0x55e7f5572d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f15d9b640 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5570940 0x55e7f5572d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8aa551c02ca13b42 server_cookie=2268c51e8f283ce9 in_seq=0 out_seq=0 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1efdffb640 10 client.17655 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1efdffb640 1 -- 172.21.15.44:0/1496046041 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7f51554e0 con 0x55e7f5570940 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1efdffb640 10 client.17655 handle_command_reply: tid=0 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 client.17655 shutdown 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 msgr2=0x55e7f55107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 0x55e7f55107c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1f080b8e70 tx=0x7f1f08009260 comp rx=0 tx=0).stop 2024-09-13T17:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 shutdown_connections 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55e7f5570940 0x55e7f5572d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7f54ff0b0 0x55e7f550ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1efdffb640 0 client.17655 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7f5509950 0x55e7f550f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 --2- 172.21.15.44:0/1496046041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7f54fe790 0x55e7f55107c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 >> 172.21.15.44:0/1496046041 conn(0x55e7f54fad00 msgr2=0x55e7f55048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 shutdown_connections 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.378+0000 7f1f1fa48740 1 -- 172.21.15.44:0/1496046041 wait complete. 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1fa48740 20 client.17655 trim_cache size 0 max 16384 2024-09-13T17:33:18.386 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:33:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1f040a1280 msgr2=0x7f1f040a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1f040a1280 0x7f1f040a3740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007f30 tx=0x7f1f10004a40 comp rx=0 tx=0).stop 2024-09-13T17:33:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 msgr2=0x7f1f18196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f18196f50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802aa10 tx=0x7f1f08032c20 comp rx=0 tx=0).stop 2024-09-13T17:33:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 shutdown_connections 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f1f040a1280 0x7f1f040a3740 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f18103fd0 0x7f1f18197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f18197a70 0x7f1f181a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 --2- 172.21.15.44:0/1507595545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180fee80 0x7f1f18196f50 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 >> 172.21.15.44:0/1507595545 conn(0x7f1f180fa930 msgr2=0x7f1f180fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 shutdown_connections 2024-09-13T17:33:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.379+0000 7f1f1cd26640 1 -- 172.21.15.44:0/1507595545 wait complete. 2024-09-13T17:33:18.391 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 -- 172.21.15.44:0/1900112481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd854072880 msgr2=0x7fd854072c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 --2- 172.21.15.44:0/1900112481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd854072880 0x7fd854072c60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd850009900 tx=0x7fd85002a480 comp rx=0 tx=0).stop 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 -- 172.21.15.44:0/1900112481 shutdown_connections 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 --2- 172.21.15.44:0/1900112481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd85410c580 0x7fd85410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 --2- 172.21.15.44:0/1900112481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8540731a0 0x7fd85410c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 --2- 172.21.15.44:0/1900112481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd854072880 0x7fd854072c60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 -- 172.21.15.44:0/1900112481 >> 172.21.15.44:0/1900112481 conn(0x7fd8540fb9e0 msgr2=0x7fd8540fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 -- 172.21.15.44:0/1900112481 shutdown_connections 2024-09-13T17:33:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.494+0000 7fd85c42b640 1 -- 172.21.15.44:0/1900112481 wait complete. 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 Processor -- start 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 -- start start 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd854072880 0x7fd854198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8540731a0 0x7fd854198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 0x7fd85419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85419d340 con 0x7fd85410c580 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85419d4b0 con 0x7fd854072880 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85c42b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85419d670 con 0x7fd8540731a0 2024-09-13T17:33:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 0x7fd85419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 0x7fd85419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34340/0 (socket says 172.21.15.44:34340) 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 -- 172.21.15.44:0/313517549 learned_addr learned my addr 172.21.15.44:0/313517549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd8599c6640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8540731a0 0x7fd854198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd854072880 0x7fd854198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 -- 172.21.15.44:0/313517549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8540731a0 msgr2=0x7fd854198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8540731a0 0x7fd854198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 -- 172.21.15.44:0/313517549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd854072880 msgr2=0x7fd854198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd854072880 0x7fd854198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.495+0000 7fd85a9c8640 1 -- 172.21.15.44:0/313517549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850009560 con 0x7fd85410c580 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.496+0000 7fd85a9c8640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 0x7fd85419cd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00eb10 tx=0x7fd84c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.496+0000 7fd84b7fe640 1 -- 172.21.15.44:0/313517549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00ce80 con 0x7fd85410c580 2024-09-13T17:33:18.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.496+0000 7fd84b7fe640 1 -- 172.21.15.44:0/313517549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd84c005020 con 0x7fd85410c580 2024-09-13T17:33:18.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.496+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85419d950 con 0x7fd85410c580 2024-09-13T17:33:18.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.496+0000 7fd84b7fe640 1 -- 172.21.15.44:0/313517549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c010690 con 0x7fd85410c580 2024-09-13T17:33:18.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.496+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8541a51d0 con 0x7fd85410c580 2024-09-13T17:33:18.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 Processor -- start 2024-09-13T17:33:18.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd84b7fe640 1 -- 172.21.15.44:0/313517549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd84c004a40 con 0x7fd85410c580 2024-09-13T17:33:18.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 -- start start 2024-09-13T17:33:18.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f426b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f431410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f433d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f390cc0 con 0x562e2f431950 2024-09-13T17:33:18.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f38e450 con 0x562e2f426790 2024-09-13T17:33:18.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.499+0000 7fd85f22a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f389d40 con 0x562e2f4270b0 2024-09-13T17:33:18.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f433d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f433d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34354/0 (socket says 172.21.15.44:34354) 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 -- 172.21.15.44:0/3612782937 learned_addr learned my addr 172.21.15.44:0/3612782937 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd84b7fe640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd8400a1280 0x7fd8400a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f426b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f431410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd85a9c8640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd8400a1280 0x7fd8400a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 -- 172.21.15.44:0/3612782937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 msgr2=0x562e2f431410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f431410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 -- 172.21.15.44:0/3612782937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 msgr2=0x562e2f426b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f426b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 -- 172.21.15.44:0/3612782937 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2f351620 con 0x562e2f431950 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd84b7fe640 1 -- 172.21.15.44:0/313517549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd84c014070 con 0x7fd85410c580 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f433d40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd844013630 tx=0x7fd84400c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45563505a80e71cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd848e05640 1 -- 172.21.15.44:0/3612782937 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844005020 con 0x562e2f431950 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd85a9c8640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd8400a1280 0x7fd8400a3740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd84c09a200 tx=0x7fd84c005f40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd848e05640 1 -- 172.21.15.44:0/3612782937 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd844005610 con 0x562e2f431950 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.500+0000 7fd848e05640 1 -- 172.21.15.44:0/3612782937 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844011800 con 0x562e2f431950 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 -- 172.21.15.44:0/3612782937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 msgr2=0x562e2f433d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f433d40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd844013630 tx=0x7fd84400c690 comp rx=0 tx=0).stop 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 -- 172.21.15.44:0/3612782937 shutdown_connections 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f431410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f433d40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3612782937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f426b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 -- 172.21.15.44:0/3612782937 >> 172.21.15.44:0/3612782937 conn(0x562e2f422d00 msgr2=0x562e2f4230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 -- 172.21.15.44:0/3612782937 shutdown_connections 2024-09-13T17:33:18.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 -- 172.21.15.44:0/3612782937 wait complete. 2024-09-13T17:33:18.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.501+0000 7fd85f22a740 1 -- start start 2024-09-13T17:33:18.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 Processor -- start 2024-09-13T17:33:18.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f430f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f42e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f348180 con 0x562e2f431950 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f062430 con 0x562e2f426790 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f30d5e0 con 0x562e2f4270b0 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85f22a740 10 client.0 _ll_register_callbacks cb 0x562e2f379aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:18.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f42e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f42e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34360/0 (socket says 172.21.15.44:34360) 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 -- 172.21.15.44:0/1334430854 learned_addr learned my addr 172.21.15.44:0/1334430854 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f430f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 -- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 msgr2=0x562e2f42e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 -- 172.21.15.44:0/1334430854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 msgr2=0x562e2f430f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f430f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 -- 172.21.15.44:0/1334430854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84400cda0 con 0x562e2f431950 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd8599c6640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f42e8a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd844004f70 tx=0x7fd844015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd62fdc7a95d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.502+0000 7fd83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd83dffb640 1 -- 172.21.15.44:0/1334430854 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844005320 con 0x562e2f431950 2024-09-13T17:33:18.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd83dffb640 1 -- 172.21.15.44:0/1334430854 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd8440115a0 con 0x562e2f431950 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e2f3b3b80 con 0x562e2f431950 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd83dffb640 1 -- 172.21.15.44:0/1334430854 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd844005480 con 0x562e2f431950 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 10 client.17667 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562e2f3883c0 con 0x562e2f431950 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd83dffb640 1 -- 172.21.15.44:0/1334430854 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562e2f3b3b80 con 0x562e2f431950 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd83dffb640 1 -- 172.21.15.44:0/1334430854 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd84401e660 con 0x562e2f431950 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 10 client.17667 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 10 client.17667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 20 client.17667 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f4411e0 0x562e2f4435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 4 client.17667 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.503+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e2f37c1f0 con 0x562e2f4411e0 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.504+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f4411e0 0x562e2f4435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.504+0000 7fd83dffb640 10 client.17667 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.504+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f4411e0 0x562e2f4435d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f17867dde9f869ff server_cookie=820807a1b23c2a1b in_seq=0 out_seq=0 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd83dffb640 1 -- 172.21.15.44:0/1334430854 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562e2f37c1f0 con 0x562e2f4411e0 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd83dffb640 10 client.17667 handle_command_reply: tid=0 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 client.17667 shutdown 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 msgr2=0x562e2f42e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f42e8a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd844004f70 tx=0x7fd844015040 comp rx=0 tx=0).stop 2024-09-13T17:33:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 shutdown_connections 2024-09-13T17:33:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f4411e0 0x562e2f4435d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f42e8a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 --2- 172.21.15.44:0/1334430854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f426790 0x562e2f430f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd83dffb640 0 client.17667 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 >> 172.21.15.44:0/1334430854 conn(0x562e2f422d00 msgr2=0x562e2f428a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 shutdown_connections 2024-09-13T17:33:18.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 1 -- 172.21.15.44:0/1334430854 wait complete. 2024-09-13T17:33:18.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.505+0000 7fd85f22a740 20 client.17667 trim_cache size 0 max 16384 2024-09-13T17:33:18.531 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:18 smithi100 ceph-mon[35055]: pgmap v202: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-13T17:33:18.531 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:18 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1522300062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:33:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 Processor -- start 2024-09-13T17:33:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 -- start start 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 0x562e2f42e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f431950 0x562e2f42cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f41d3d0 con 0x562e2f426790 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f431560 con 0x562e2f431950 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85f22a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f434400 con 0x562e2f4270b0 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 0x562e2f42e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 0x562e2f42e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34362/0 (socket says 172.21.15.44:34362) 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 -- 172.21.15.44:0/3924762862 learned_addr learned my addr 172.21.15.44:0/3924762862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 -- 172.21.15.44:0/3924762862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 msgr2=0x562e2f42e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 -- 172.21.15.44:0/3924762862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f431950 msgr2=0x562e2f42cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f431950 0x562e2f42cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd8599c6640 1 -- 172.21.15.44:0/3924762862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2f32a770 con 0x562e2f426790 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.516+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 0x562e2f42e320 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd8440114b0 tx=0x7fd84400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7507e49c5e45eb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd849606640 1 -- 172.21.15.44:0/3924762862 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400c3b0 con 0x562e2f426790 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd849606640 1 -- 172.21.15.44:0/3924762862 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd844026070 con 0x562e2f426790 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd849606640 1 -- 172.21.15.44:0/3924762862 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844005230 con 0x562e2f426790 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 -- 172.21.15.44:0/3924762862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 msgr2=0x562e2f42e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 0x562e2f42e320 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd8440114b0 tx=0x7fd84400cf30 comp rx=0 tx=0).stop 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 -- 172.21.15.44:0/3924762862 shutdown_connections 2024-09-13T17:33:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f4270b0 0x562e2f42e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f431950 0x562e2f42cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3924762862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f426790 0x562e2f42e320 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 -- 172.21.15.44:0/3924762862 >> 172.21.15.44:0/3924762862 conn(0x562e2f422d00 msgr2=0x562e2f43d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 -- 172.21.15.44:0/3924762862 shutdown_connections 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.517+0000 7fd85f22a740 1 -- 172.21.15.44:0/3924762862 wait complete. 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 -- start start 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 Processor -- start 2024-09-13T17:33:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f426790 0x562e2f435690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f4270b0 0x562e2f435bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f497680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f436280 con 0x562e2f431950 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f497bc0 con 0x562e2f4270b0 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2f497d00 con 0x562e2f426790 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85f22a740 10 client.0 _ll_register_callbacks cb 0x7fd84400c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f497680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f497680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34364/0 (socket says 172.21.15.44:34364) 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 -- 172.21.15.44:0/3894132742 learned_addr learned my addr 172.21.15.44:0/3894132742 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f426790 0x562e2f435690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f4270b0 0x562e2f435bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 -- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f426790 msgr2=0x562e2f435690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f426790 0x562e2f435690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 -- 172.21.15.44:0/3894132742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f4270b0 msgr2=0x562e2f435bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f4270b0 0x562e2f435bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.518+0000 7fd8599c6640 1 -- 172.21.15.44:0/3894132742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e2f3883c0 con 0x562e2f431950 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd8599c6640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f497680 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd844004de0 tx=0x7fd8440049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9076ac65fc886736 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd83dffb640 1 -- 172.21.15.44:0/3894132742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844005440 con 0x562e2f431950 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd83dffb640 1 -- 172.21.15.44:0/3894132742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd84401e460 con 0x562e2f431950 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd844005610 con 0x562e2f431950 2024-09-13T17:33:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd83dffb640 1 -- 172.21.15.44:0/3894132742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84401e620 con 0x562e2f431950 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 10 client.17673 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562e2f34fe80 con 0x562e2f431950 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd83dffb640 1 -- 172.21.15.44:0/3894132742 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd844050080 con 0x562e2f431950 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd83dffb640 1 -- 172.21.15.44:0/3894132742 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd84401d550 con 0x562e2f431950 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 10 client.17673 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 10 client.17673 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 20 client.17673 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f498fd0 0x562e2f49b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 4 client.17673 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:33:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.519+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e2f07d4e0 con 0x562e2f498fd0 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.520+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f498fd0 0x562e2f49b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.520+0000 7fd83dffb640 10 client.17673 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.520+0000 7fd85a1c7640 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f498fd0 0x562e2f49b3c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34bd29d40b164cad server_cookie=1724f3ab195f49d2 in_seq=0 out_seq=0 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.520+0000 7fd83dffb640 1 -- 172.21.15.44:0/3894132742 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e2f07d4e0 con 0x562e2f498fd0 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.520+0000 7fd83dffb640 10 client.17673 handle_command_reply: tid=0 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.520+0000 7fd85f22a740 1 client.17673 shutdown 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 msgr2=0x562e2f497680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f497680 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd844004de0 tx=0x7fd8440049d0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 shutdown_connections 2024-09-13T17:33:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562e2f498fd0 0x562e2f49b3c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e2f431950 0x562e2f497680 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd83dffb640 0 client.17673 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562e2f4270b0 0x562e2f435bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 --2- 172.21.15.44:0/3894132742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e2f426790 0x562e2f435690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 >> 172.21.15.44:0/3894132742 conn(0x562e2f422d00 msgr2=0x562e2f42cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 shutdown_connections 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 1 -- 172.21.15.44:0/3894132742 wait complete. 2024-09-13T17:33:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85f22a740 20 client.17673 trim_cache size 0 max 16384 2024-09-13T17:33:18.540 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:33:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd8400a1280 msgr2=0x7fd8400a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd8400a1280 0x7fd8400a3740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd84c09a200 tx=0x7fd84c005f40 comp rx=0 tx=0).stop 2024-09-13T17:33:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 msgr2=0x7fd85419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 0x7fd85419cd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00eb10 tx=0x7fd84c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 shutdown_connections 2024-09-13T17:33:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd8400a1280 0x7fd8400a3740 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8540731a0 0x7fd854198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd85410c580 0x7fd85419cd40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 --2- 172.21.15.44:0/313517549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd854072880 0x7fd854198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 >> 172.21.15.44:0/313517549 conn(0x7fd8540fb9e0 msgr2=0x7fd8540fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 shutdown_connections 2024-09-13T17:33:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.521+0000 7fd85c42b640 1 -- 172.21.15.44:0/313517549 wait complete. 2024-09-13T17:33:18.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:18 smithi044 ceph-mon[31175]: pgmap v202: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-13T17:33:18.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:18 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1522300062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:33:18.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:33:18.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 -- 172.21.15.44:0/2822993049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 msgr2=0x7f117c10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 --2- 172.21.15.44:0/2822993049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c10cb70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ca0 tx=0x7f117402a8c0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 -- 172.21.15.44:0/2822993049 shutdown_connections 2024-09-13T17:33:18.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 --2- 172.21.15.44:0/2822993049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c10cb70 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 --2- 172.21.15.44:0/2822993049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117c0fe220 0x7f117c0fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 --2- 172.21.15.44:0/2822993049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f117c104d00 0x7f117c1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 -- 172.21.15.44:0/2822993049 >> 172.21.15.44:0/2822993049 conn(0x7f117c0753f0 msgr2=0x7f117c0fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 -- 172.21.15.44:0/2822993049 shutdown_connections 2024-09-13T17:33:18.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.646+0000 7f11811ae640 1 -- 172.21.15.44:0/2822993049 wait complete. 2024-09-13T17:33:18.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 Processor -- start 2024-09-13T17:33:18.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 -- start start 2024-09-13T17:33:18.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117c0fe220 0x7f117c196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f117c104d00 0x7f117c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c19b0c0 con 0x7f117c0feba0 2024-09-13T17:33:18.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c19b230 con 0x7f117c0fe220 2024-09-13T17:33:18.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f11811ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c19b3f0 con 0x7f117c104d00 2024-09-13T17:33:18.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f1173fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f1173fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34372/0 (socket says 172.21.15.44:34372) 2024-09-13T17:33:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f1173fff640 1 -- 172.21.15.44:0/3887701949 learned_addr learned my addr 172.21.15.44:0/3887701949 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117c0fe220 0x7f117c196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.647+0000 7f117b59e640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f117c104d00 0x7f117c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1173fff640 1 -- 172.21.15.44:0/3887701949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f117c104d00 msgr2=0x7f117c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1173fff640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f117c104d00 0x7f117c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1173fff640 1 -- 172.21.15.44:0/3887701949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117c0fe220 msgr2=0x7f117c196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1173fff640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117c0fe220 0x7f117c196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1173fff640 1 -- 172.21.15.44:0/3887701949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174009900 con 0x7f117c0feba0 2024-09-13T17:33:18.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1173fff640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c1966d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f116400c920 tx=0x7f116400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1178d99640 1 -- 172.21.15.44:0/3887701949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164007c70 con 0x7f117c0feba0 2024-09-13T17:33:18.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1178d99640 1 -- 172.21.15.44:0/3887701949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1164007dd0 con 0x7f117c0feba0 2024-09-13T17:33:18.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c19b6d0 con 0x7f117c0feba0 2024-09-13T17:33:18.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f1178d99640 1 -- 172.21.15.44:0/3887701949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164018730 con 0x7f117c0feba0 2024-09-13T17:33:18.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.648+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c1a2f50 con 0x7f117c0feba0 2024-09-13T17:33:18.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.651+0000 7f1183ed3740 1 Processor -- start 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.651+0000 7f1178d99640 1 -- 172.21.15.44:0/3887701949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f116400f450 con 0x7f117c0feba0 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.651+0000 7f1183ed3740 1 -- start start 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.651+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1183ed3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaecbccc0 con 0x562aaed52790 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1183ed3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaecba450 con 0x562aaed530b0 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1183ed3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaecb5d40 con 0x562aaed5d950 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed52b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34378/0 (socket says 172.21.15.44:34378) 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 -- 172.21.15.44:0/3574787908 learned_addr learned my addr 172.21.15.44:0/3574787908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1173fff640 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 -- 172.21.15.44:0/3574787908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 msgr2=0x562aaed5fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 -- 172.21.15.44:0/3574787908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 msgr2=0x562aaed5d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 -- 172.21.15.44:0/3574787908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562aaec7d620 con 0x562aaed52790 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1173fff640 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f117b59e640 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed52b70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ca0 tx=0x7f1174005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5056520d64f1f968 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.652+0000 7f1178d99640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11680a1200 0x7f11680a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f11717fa640 1 -- 172.21.15.44:0/3574787908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174031cb0 con 0x562aaed52790 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11680a1200 0x7f11680a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1178d99640 1 -- 172.21.15.44:0/3887701949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f11640cd100 con 0x7f117c0feba0 2024-09-13T17:33:18.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f11717fa640 1 -- 172.21.15.44:0/3574787908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11740318e0 con 0x562aaed52790 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f11717fa640 1 -- 172.21.15.44:0/3574787908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740308b0 con 0x562aaed52790 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 -- 172.21.15.44:0/3574787908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 msgr2=0x562aaed52b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11680a1200 0x7f11680a36c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x562aaed4fb60 tx=0x7f116c005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed52b70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ca0 tx=0x7f1174005f70 comp rx=0 tx=0).stop 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 -- 172.21.15.44:0/3574787908 shutdown_connections 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3574787908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed52b70 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 -- 172.21.15.44:0/3574787908 >> 172.21.15.44:0/3574787908 conn(0x562aaed4ed00 msgr2=0x562aaed4f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 -- 172.21.15.44:0/3574787908 shutdown_connections 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.653+0000 7f1183ed3740 1 -- 172.21.15.44:0/3574787908 wait complete. 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 -- start start 2024-09-13T17:33:18.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 Processor -- start 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaec74180 con 0x562aaed52790 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aae98e430 con 0x562aaed530b0 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaec395e0 con 0x562aaed5d950 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1183ed3740 10 client.0 _ll_register_callbacks cb 0x562aaeca5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f117ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f117ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed5ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34388/0 (socket says 172.21.15.44:34388) 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f117ad9d640 1 -- 172.21.15.44:0/3632662416 learned_addr learned my addr 172.21.15.44:0/3632662416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f1173fff640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.654+0000 7f117b59e640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117ad9d640 1 -- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 msgr2=0x562aaed5a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117ad9d640 1 -- 172.21.15.44:0/3632662416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 msgr2=0x562aaed5d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117ad9d640 1 -- 172.21.15.44:0/3632662416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c10d520 con 0x562aaed52790 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117b59e640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f117ad9d640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed5ced0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f116c0045e0 tx=0x7f116c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a727fb1fc7126e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f11627fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:18.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f11627fc640 1 -- 172.21.15.44:0/3632662416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c017070 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f11627fc640 1 -- 172.21.15.44:0/3632662416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f116c009e00 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562aaecdfb80 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f11627fc640 1 -- 172.21.15.44:0/3632662416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116c01e680 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f1183ed3740 10 client.17685 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562aaecb43c0 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f11627fc640 1 -- 172.21.15.44:0/3632662416 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562aaecdfb80 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f11627fc640 1 -- 172.21.15.44:0/3632662416 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f116c0298b0 con 0x562aaed52790 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f1183ed3740 10 client.17685 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.655+0000 7f1183ed3740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f1183ed3740 20 client.17685 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaed6d300 0x562aaed6f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f1183ed3740 4 client.17685 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562aaeca81f0 con 0x562aaed6d300 2024-09-13T17:33:18.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f1173fff640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaed6d300 0x562aaed6f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f11627fc640 10 client.17685 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.656+0000 7f1173fff640 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaed6d300 0x562aaed6f720 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e357c709a27c37 server_cookie=2a7634dc43388a72 in_seq=0 out_seq=0 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f11627fc640 1 -- 172.21.15.44:0/3632662416 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562aaeca81f0 con 0x562aaed6d300 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f11627fc640 10 client.17685 handle_command_reply: tid=0 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 client.17685 shutdown 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 msgr2=0x562aaed5ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed5ced0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f116c0045e0 tx=0x7f116c004730 comp rx=0 tx=0).stop 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 shutdown_connections 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaed6d300 0x562aaed6f720 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed5d950 0x562aaed5a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed530b0 0x562aaed5d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f11627fc640 0 client.17685 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 --2- 172.21.15.44:0/3632662416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed52790 0x562aaed5ced0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 >> 172.21.15.44:0/3632662416 conn(0x562aaed4ed00 msgr2=0x562aaed54a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 shutdown_connections 2024-09-13T17:33:18.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.657+0000 7f1183ed3740 1 -- 172.21.15.44:0/3632662416 wait complete. 2024-09-13T17:33:18.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.658+0000 7f1183ed3740 20 client.17685 trim_cache size 0 max 16384 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.668+0000 7f1183ed3740 1 Processor -- start 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.668+0000 7f1183ed3740 1 -- start start 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed69610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed5ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1183ed3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaeca8220 con 0x562aaed5d950 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1183ed3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaed6bb60 con 0x562aaed52790 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1183ed3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaed5b530 con 0x562aaed530b0 2024-09-13T17:33:18.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed5ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed5ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34390/0 (socket says 172.21.15.44:34390) 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 -- 172.21.15.44:0/1763408386 learned_addr learned my addr 172.21.15.44:0/1763408386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f1173fff640 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed69610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117ad9d640 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 -- 172.21.15.44:0/1763408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 msgr2=0x562aaed69610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed69610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 -- 172.21.15.44:0/1763408386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 msgr2=0x562aaed690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 -- 172.21.15.44:0/1763408386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562aaec56770 con 0x562aaed5d950 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117ad9d640 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.669+0000 7f117b59e640 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed5ed40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1174005cc0 tx=0x7f1174000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ef36b398260b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1171ffb640 1 -- 172.21.15.44:0/1763408386 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174031450 con 0x562aaed5d950 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1171ffb640 1 -- 172.21.15.44:0/1763408386 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1174003e00 con 0x562aaed5d950 2024-09-13T17:33:18.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1171ffb640 1 -- 172.21.15.44:0/1763408386 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174030b70 con 0x562aaed5d950 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 -- 172.21.15.44:0/1763408386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 msgr2=0x562aaed5ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed5ed40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1174005cc0 tx=0x7f1174000f40 comp rx=0 tx=0).stop 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 -- 172.21.15.44:0/1763408386 shutdown_connections 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed69610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed5ed40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 --2- 172.21.15.44:0/1763408386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 -- 172.21.15.44:0/1763408386 >> 172.21.15.44:0/1763408386 conn(0x562aaed4ed00 msgr2=0x562aaed4f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 -- 172.21.15.44:0/1763408386 shutdown_connections 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.670+0000 7f1183ed3740 1 -- 172.21.15.44:0/1763408386 wait complete. 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 -- start start 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 Processor -- start 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed57000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed57540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed57a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaedc7c30 con 0x562aaed5d950 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaedc7d70 con 0x562aaed52790 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aaedc7f00 con 0x562aaed530b0 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1183ed3740 10 client.0 _ll_register_callbacks cb 0x7f1174031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed57a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed57a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34404/0 (socket says 172.21.15.44:34404) 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 -- 172.21.15.44:0/2381345264 learned_addr learned my addr 172.21.15.44:0/2381345264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f1173fff640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed57540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117ad9d640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed57000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 -- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 msgr2=0x562aaed57540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed57540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 -- 172.21.15.44:0/2381345264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 msgr2=0x562aaed57000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed57000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 -- 172.21.15.44:0/2381345264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562aaec7d620 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117ad9d640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed57000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f117b59e640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed57a80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1174031a50 tx=0x7f1174031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c871b54920a62032 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.671+0000 7f11627fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f11627fc640 1 -- 172.21.15.44:0/2381345264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174030870 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f11627fc640 1 -- 172.21.15.44:0/2381345264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f11740309d0 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11740318e0 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f11627fc640 1 -- 172.21.15.44:0/2381345264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1174030b90 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562aaec7be80 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f11627fc640 1 -- 172.21.15.44:0/2381345264 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11740318e0 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f11627fc640 1 -- 172.21.15.44:0/2381345264 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1174004000 con 0x562aaed5d950 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 20 client.17691 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaedc9140 0x562aaedcb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 4 client.17691 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.672+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562aae9a94e0 con 0x562aaedc9140 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f117ad9d640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaedc9140 0x562aaedcb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f117ad9d640 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaedc9140 0x562aaedcb530 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24e144e5ef982d0 server_cookie=33c7e14fc013c98a in_seq=0 out_seq=0 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f11627fc640 10 client.17691 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f11627fc640 1 -- 172.21.15.44:0/2381345264 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562aae9a94e0 con 0x562aaedc9140 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f11627fc640 10 client.17691 handle_command_reply: tid=0 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 client.17691 shutdown 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 msgr2=0x562aaed57a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed57a80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1174031a50 tx=0x7f1174031700 comp rx=0 tx=0).stop 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 shutdown_connections 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562aaedc9140 0x562aaedcb530 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562aaed530b0 0x562aaed57540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aaed5d950 0x562aaed57a80 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 --2- 172.21.15.44:0/2381345264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aaed52790 0x562aaed57000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f11627fc640 0 client.17691 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 >> 172.21.15.44:0/2381345264 conn(0x562aaed4ed00 msgr2=0x562aaed690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 shutdown_connections 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.673+0000 7f1183ed3740 1 -- 172.21.15.44:0/2381345264 wait complete. 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f1183ed3740 20 client.17691 trim_cache size 0 max 16384 2024-09-13T17:33:18.684 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:33:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11680a1200 msgr2=0x7f11680a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11680a1200 0x7f11680a36c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x562aaed4fb60 tx=0x7f116c005f90 comp rx=0 tx=0).stop 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 msgr2=0x7f117c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c1966d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f116400c920 tx=0x7f116400cde0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 shutdown_connections 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11680a1200 0x7f11680a36c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f117c104d00 0x7f117c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f117c0feba0 0x7f117c1966d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 --2- 172.21.15.44:0/3887701949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117c0fe220 0x7f117c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 >> 172.21.15.44:0/3887701949 conn(0x7f117c0753f0 msgr2=0x7f117c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 shutdown_connections 2024-09-13T17:33:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:18.674+0000 7f11811ae640 1 -- 172.21.15.44:0/3887701949 wait complete. 2024-09-13T17:33:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:18 smithi070 ceph-mon[34502]: pgmap v202: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-13T17:33:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:18 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1522300062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:33:19.688 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:33:19.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:33:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 -- 172.21.15.44:0/3122283321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626810a330 msgr2=0x7f626810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 --2- 172.21.15.44:0/3122283321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626810a330 0x7f626810c7b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6260009ca0 tx=0x7f626002a950 comp rx=0 tx=0).stop 2024-09-13T17:33:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 -- 172.21.15.44:0/3122283321 shutdown_connections 2024-09-13T17:33:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 --2- 172.21.15.44:0/3122283321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626810a330 0x7f626810c7b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 --2- 172.21.15.44:0/3122283321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6268101800 0x7f6268109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 --2- 172.21.15.44:0/3122283321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268100ee0 0x7f62681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 -- 172.21.15.44:0/3122283321 >> 172.21.15.44:0/3122283321 conn(0x7f62680fa8d0 msgr2=0x7f62680fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 -- 172.21.15.44:0/3122283321 shutdown_connections 2024-09-13T17:33:19.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 -- 172.21.15.44:0/3122283321 wait complete. 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.806+0000 7f6270776640 1 Processor -- start 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 -- start start 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268100ee0 0x7f6268198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 0x7f6268198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f626810a330 0x7f626819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626819d280 con 0x7f6268101800 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626819d3f0 con 0x7f626810a330 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f6270776640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626819d5b0 con 0x7f6268100ee0 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 0x7f6268198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 0x7f6268198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37994/0 (socket says 172.21.15.44:37994) 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 -- 172.21.15.44:0/3755866124 learned_addr learned my addr 172.21.15.44:0/3755866124 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626e512640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268100ee0 0x7f6268198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626ed13640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f626810a330 0x7f626819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 -- 172.21.15.44:0/3755866124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268100ee0 msgr2=0x7f6268198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268100ee0 0x7f6268198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 -- 172.21.15.44:0/3755866124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f626810a330 msgr2=0x7f626819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f626810a330 0x7f626819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 -- 172.21.15.44:0/3755866124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260009900 con 0x7f6268101800 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.807+0000 7f626dd11640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 0x7f6268198890 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f625800d950 tx=0x7f625800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.808+0000 7f625f7fe640 1 -- 172.21.15.44:0/3755866124 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258009a30 con 0x7f6268101800 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.808+0000 7f625f7fe640 1 -- 172.21.15.44:0/3755866124 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62580044b0 con 0x7f6268101800 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.808+0000 7f625f7fe640 1 -- 172.21.15.44:0/3755866124 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258005690 con 0x7f6268101800 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.808+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626819d890 con 0x7f6268101800 2024-09-13T17:33:19.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.808+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62681a5110 con 0x7f6268101800 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.810+0000 7f625f7fe640 1 -- 172.21.15.44:0/3755866124 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6258004020 con 0x7f6268101800 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.810+0000 7f627349b740 1 Processor -- start 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 -- start start 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b05cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b067440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9afc7c60 con 0x55ab9b067980 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9afc53f0 con 0x55ab9b05c7c0 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f627349b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9afc0ce0 con 0x55ab9b05d0e0 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b05cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626e512640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b067440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b05cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:44656/0 (socket says 172.21.15.44:44656) 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 -- 172.21.15.44:0/143786953 learned_addr learned my addr 172.21.15.44:0/143786953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f625f7fe640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62540a1200 0x7f62540a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 -- 172.21.15.44:0/143786953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 msgr2=0x55ab9b067440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b067440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 -- 172.21.15.44:0/143786953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 msgr2=0x55ab9b069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.811+0000 7f626ed13640 1 -- 172.21.15.44:0/143786953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab9af7c970 con 0x55ab9b05c7c0 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.812+0000 7f625f7fe640 1 -- 172.21.15.44:0/3755866124 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f62580cc890 con 0x7f6268101800 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.812+0000 7f626ed13640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62540a1200 0x7f62540a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.812+0000 7f626ed13640 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b05cba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6260009ca0 tx=0x7f626002f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9641083eb60704e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.812+0000 7f625cff9640 1 -- 172.21.15.44:0/143786953 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260040040 con 0x55ab9b05c7c0 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.812+0000 7f625cff9640 1 -- 172.21.15.44:0/143786953 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6260003cb0 con 0x55ab9b05c7c0 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.812+0000 7f625cff9640 1 -- 172.21.15.44:0/143786953 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260004380 con 0x55ab9b05c7c0 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 -- 172.21.15.44:0/143786953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 msgr2=0x55ab9b05cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b05cba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6260009ca0 tx=0x7f626002f450 comp rx=0 tx=0).stop 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f626ed13640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62540a1200 0x7f62540a36c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f626002f8e0 tx=0x7f6260050000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 -- 172.21.15.44:0/143786953 shutdown_connections 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b067440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 --2- 172.21.15.44:0/143786953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b05cba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 -- 172.21.15.44:0/143786953 >> 172.21.15.44:0/143786953 conn(0x55ab9b058d50 msgr2=0x55ab9b059130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 -- 172.21.15.44:0/143786953 shutdown_connections 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 -- 172.21.15.44:0/143786953 wait complete. 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 -- start start 2024-09-13T17:33:19.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 Processor -- start 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.813+0000 7f627349b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05c7c0 0x55ab9b0673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b064650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b067980 0x55ab9b064de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9af7ef80 con 0x55ab9b05d0e0 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9af7ec20 con 0x55ab9b067980 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9af4bab0 con 0x55ab9b05c7c0 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 10 client.0 _ll_register_callbacks cb 0x55ab9afba350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b064650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05c7c0 0x55ab9b0673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b064650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38022/0 (socket says 172.21.15.44:38022) 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626dd11640 1 -- 172.21.15.44:0/892011386 learned_addr learned my addr 172.21.15.44:0/892011386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626ed13640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b067980 0x55ab9b064de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 -- 172.21.15.44:0/892011386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b067980 msgr2=0x55ab9b064de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b067980 0x55ab9b064de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 -- 172.21.15.44:0/892011386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 msgr2=0x55ab9b064650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b064650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 -- 172.21.15.44:0/892011386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626810d430 con 0x55ab9b05c7c0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626dd11640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b064650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f626e512640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05c7c0 0x55ab9b0673b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62640058c0 tx=0x7f6264010af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae6edfcb28b0872 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f6251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f6251ffb640 1 -- 172.21.15.44:0/892011386 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264015070 con 0x55ab9b05c7c0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab9acb4590 con 0x55ab9b05c7c0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.814+0000 7f6251ffb640 1 -- 172.21.15.44:0/892011386 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f626400f970 con 0x55ab9b05c7c0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f6251ffb640 1 -- 172.21.15.44:0/892011386 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f626400fb70 con 0x55ab9b05c7c0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 10 client.27049 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ab9afbf360 con 0x55ab9b05c7c0 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f6251ffb640 1 -- 172.21.15.44:0/892011386 <== mon.1 v2:172.21.15.100:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ab9acb4590 con 0x55ab9b05c7c0 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f6251ffb640 1 -- 172.21.15.44:0/892011386 <== mon.1 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f62640053c0 con 0x55ab9b05c7c0 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 10 client.27049 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 10 client.27049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 20 client.27049 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b077790 0x55ab9b079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 4 client.27049 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab9acc5860 con 0x55ab9b077790 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.815+0000 7f626dd11640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b077790 0x55ab9b079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f626dd11640 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b077790 0x55ab9b079bb0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990128b9d22f5c41 server_cookie=ada89e907be48697 in_seq=0 out_seq=0 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f6251ffb640 10 client.27049 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f6251ffb640 1 -- 172.21.15.44:0/892011386 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ab9acc5860 con 0x55ab9b077790 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f6251ffb640 10 client.27049 handle_command_reply: tid=0 2024-09-13T17:33:19.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 client.27049 shutdown 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05c7c0 msgr2=0x55ab9b0673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05c7c0 0x55ab9b0673b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62640058c0 tx=0x7f6264010af0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 shutdown_connections 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b077790 0x55ab9b079bb0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b067980 0x55ab9b064de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b064650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 --2- 172.21.15.44:0/892011386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05c7c0 0x55ab9b0673b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f6251ffb640 0 client.27049 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.816+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 >> 172.21.15.44:0/892011386 conn(0x55ab9b058d50 msgr2=0x55ab9b05e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.817+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 shutdown_connections 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.817+0000 7f627349b740 1 -- 172.21.15.44:0/892011386 wait complete. 2024-09-13T17:33:19.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.817+0000 7f627349b740 20 client.27049 trim_cache size 0 max 16384 2024-09-13T17:33:19.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 Processor -- start 2024-09-13T17:33:19.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 -- start start 2024-09-13T17:33:19.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b07e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b067980 0x55ab9b06eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9b05c610 con 0x55ab9b05d0e0 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9b04aef0 con 0x55ab9b05c7c0 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f627349b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9b0546e0 con 0x55ab9b067980 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38026/0 (socket says 172.21.15.44:38026) 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 -- 172.21.15.44:0/4122567851 learned_addr learned my addr 172.21.15.44:0/4122567851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626e512640 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b07e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626ed13640 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b067980 0x55ab9b06eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 -- 172.21.15.44:0/4122567851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b067980 msgr2=0x55ab9b06eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b067980 0x55ab9b06eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 -- 172.21.15.44:0/4122567851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 msgr2=0x55ab9b07e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b07e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626dd11640 1 -- 172.21.15.44:0/4122567851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab9af611c0 con 0x55ab9b05d0e0 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.831+0000 7f626ed13640 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b067980 0x55ab9b06eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f626dd11640 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06e650 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6258000c00 tx=0x7f6258084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf724e90d5444b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f625d7fa640 1 -- 172.21.15.44:0/4122567851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258099110 con 0x55ab9b05d0e0 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f625d7fa640 1 -- 172.21.15.44:0/4122567851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6258098f30 con 0x55ab9b05d0e0 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f625d7fa640 1 -- 172.21.15.44:0/4122567851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258099cd0 con 0x55ab9b05d0e0 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 -- 172.21.15.44:0/4122567851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 msgr2=0x55ab9b06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06e650 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6258000c00 tx=0x7f6258084040 comp rx=0 tx=0).stop 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 -- 172.21.15.44:0/4122567851 shutdown_connections 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b067980 0x55ab9b06eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06e650 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 --2- 172.21.15.44:0/4122567851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b07e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 -- 172.21.15.44:0/4122567851 >> 172.21.15.44:0/4122567851 conn(0x55ab9b058d50 msgr2=0x55ab9b062f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 -- 172.21.15.44:0/4122567851 shutdown_connections 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.832+0000 7f627349b740 1 -- 172.21.15.44:0/4122567851 wait complete. 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 -- start start 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 Processor -- start 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b06e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9b06d9a0 con 0x55ab9b067980 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9b06db10 con 0x55ab9b05c7c0 2024-09-13T17:33:19.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9b0cdca0 con 0x55ab9b05d0e0 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f627349b740 10 client.0 _ll_register_callbacks cb 0x7f626400a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f626ed13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f626ed13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b06d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38038/0 (socket says 172.21.15.44:38038) 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f626ed13640 1 -- 172.21.15.44:0/387181335 learned_addr learned my addr 172.21.15.44:0/387181335 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f626e512640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.833+0000 7f626dd11640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f626ed13640 1 -- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 msgr2=0x55ab9b06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f626ed13640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f626ed13640 1 -- 172.21.15.44:0/387181335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 msgr2=0x55ab9b06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f626ed13640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b06e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f626ed13640 1 -- 172.21.15.44:0/387181335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62681a5110 con 0x55ab9b067980 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f626ed13640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b06d2a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f626002fef0 tx=0x7f6260005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4428cd3914d0c9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f6251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f6251ffb640 1 -- 172.21.15.44:0/387181335 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260004870 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f6251ffb640 1 -- 172.21.15.44:0/387181335 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6260003e20 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6260003cb0 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f6251ffb640 1 -- 172.21.15.44:0/387181335 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6260003fe0 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f627349b740 10 client.17709 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ab9af86990 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f6251ffb640 1 -- 172.21.15.44:0/387181335 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6260003cb0 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f6251ffb640 1 -- 172.21.15.44:0/387181335 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f626004f020 con 0x55ab9b067980 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f627349b740 10 client.17709 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f627349b740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.834+0000 7f627349b740 20 client.17709 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.835+0000 7f627349b740 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b0cf020 0x55ab9b0d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.835+0000 7f627349b740 4 client.17709 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:33:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.835+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ab9ad18eb0 con 0x55ab9b0cf020 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.835+0000 7f626e512640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b0cf020 0x55ab9b0d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.835+0000 7f6251ffb640 10 client.17709 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.835+0000 7f626e512640 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b0cf020 0x55ab9b0d1410 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbfb9b8a3755f59d server_cookie=8dd95a603a159ccb in_seq=0 out_seq=0 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f6251ffb640 1 -- 172.21.15.44:0/387181335 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ab9ad18eb0 con 0x55ab9b0cf020 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f6251ffb640 10 client.17709 handle_command_reply: tid=0 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 client.17709 shutdown 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 msgr2=0x55ab9b06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b06d2a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f626002fef0 tx=0x7f6260005c70 comp rx=0 tx=0).stop 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 shutdown_connections 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ab9b0cf020 0x55ab9b0d1410 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab9b05d0e0 0x55ab9b06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab9b067980 0x55ab9b06d2a0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 --2- 172.21.15.44:0/387181335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab9b05c7c0 0x55ab9b06e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f6251ffb640 0 client.17709 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 >> 172.21.15.44:0/387181335 conn(0x55ab9b058d50 msgr2=0x55ab9b07e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 shutdown_connections 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.836+0000 7f627349b740 1 -- 172.21.15.44:0/387181335 wait complete. 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f627349b740 20 client.17709 trim_cache size 0 max 16384 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "0f04d115-e0f3-4a11-bb07-2031af733979", 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:33:19.846 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:33:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62540a1200 msgr2=0x7f62540a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62540a1200 0x7f62540a36c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f626002f8e0 tx=0x7f6260050000 comp rx=0 tx=0).stop 2024-09-13T17:33:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 msgr2=0x7f6268198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 0x7f6268198890 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f625800d950 tx=0x7f625800de10 comp rx=0 tx=0).stop 2024-09-13T17:33:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 shutdown_connections 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f62540a1200 0x7f62540a36c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f626810a330 0x7f626819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6268101800 0x7f6268198890 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 --2- 172.21.15.44:0/3755866124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268100ee0 0x7f6268198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 >> 172.21.15.44:0/3755866124 conn(0x7f62680fa8d0 msgr2=0x7f62680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 shutdown_connections 2024-09-13T17:33:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.837+0000 7f6270776640 1 -- 172.21.15.44:0/3755866124 wait complete. 2024-09-13T17:33:19.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:33:19.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 -- 172.21.15.44:0/3419094127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe180100940 msgr2=0x7fe180104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 --2- 172.21.15.44:0/3419094127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe180100940 0x7fe180104980 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe168009930 tx=0x7fe16802a420 comp rx=0 tx=0).stop 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 -- 172.21.15.44:0/3419094127 shutdown_connections 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 --2- 172.21.15.44:0/3419094127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 0x7fe18010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 --2- 172.21.15.44:0/3419094127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe180100940 0x7fe180104980 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 --2- 172.21.15.44:0/3419094127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 0x7fe180100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 -- 172.21.15.44:0/3419094127 >> 172.21.15.44:0/3419094127 conn(0x7fe1800fba20 msgr2=0x7fe1800fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.952+0000 7fe184ee9640 1 -- 172.21.15.44:0/3419094127 shutdown_connections 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 -- 172.21.15.44:0/3419094127 wait complete. 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 Processor -- start 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 -- start start 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 0x7fe1801961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 0x7fe180196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 0x7fe18019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.953+0000 7fe184ee9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019b500 con 0x7fe18019ab10 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe184ee9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019b670 con 0x7fe1800fff90 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe184ee9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019b7e0 con 0x7fe1801050e0 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 0x7fe18019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 0x7fe18019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38068/0 (socket says 172.21.15.44:38068) 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 -- 172.21.15.44:0/2438086114 learned_addr learned my addr 172.21.15.44:0/2438086114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 0x7fe180196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17e59c640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 0x7fe1801961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 -- 172.21.15.44:0/2438086114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 msgr2=0x7fe180196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 0x7fe180196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 -- 172.21.15.44:0/2438086114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 msgr2=0x7fe1801961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 0x7fe1801961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 -- 172.21.15.44:0/2438086114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168009560 con 0x7fe18019ab10 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17e59c640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 0x7fe1801961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 0x7fe180196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe17ed9d640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 0x7fe18019afc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe17000eaa0 tx=0x7fe17000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe1777fe640 1 -- 172.21.15.44:0/2438086114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000cdb0 con 0x7fe18019ab10 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe1777fe640 1 -- 172.21.15.44:0/2438086114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe170005020 con 0x7fe18019ab10 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1801a2dc0 con 0x7fe18019ab10 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe1777fe640 1 -- 172.21.15.44:0/2438086114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700116d0 con 0x7fe18019ab10 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.954+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1801a3240 con 0x7fe18019ab10 2024-09-13T17:33:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe1777fe640 1 -- 172.21.15.44:0/2438086114 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe170005190 con 0x7fe18019ab10 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 Processor -- start 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 -- start start 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 0x5591529bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x5591529ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559152927f20 con 0x5591529bce10 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591529256b0 con 0x5591529bd730 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.957+0000 7fe187c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559152920fa0 con 0x5591529c7fd0 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 0x5591529bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 0x5591529bd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38076/0 (socket says 172.21.15.44:38076) 2024-09-13T17:33:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 -- 172.21.15.44:0/1129319962 learned_addr learned my addr 172.21.15.44:0/1129319962 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe1777fe640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe16c0a1280 0x7fe16c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17e59c640 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x5591529ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 -- 172.21.15.44:0/1129319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 msgr2=0x5591529ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x5591529ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 -- 172.21.15.44:0/1129319962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 msgr2=0x5591529c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 -- 172.21.15.44:0/1129319962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591528db840 con 0x5591529bce10 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17ed9d640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe16c0a1280 0x7fe16c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe1777fe640 1 -- 172.21.15.44:0/2438086114 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe1700cd220 con 0x7fe18019ab10 2024-09-13T17:33:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 0x5591529bd1f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe168005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a502e40c2da1570 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe174ff9640 1 -- 172.21.15.44:0/1129319962 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168044070 con 0x5591529bce10 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe174ff9640 1 -- 172.21.15.44:0/1129319962 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe168031df0 con 0x5591529bce10 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.958+0000 7fe174ff9640 1 -- 172.21.15.44:0/1129319962 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168003b60 con 0x5591529bce10 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 -- 172.21.15.44:0/1129319962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 msgr2=0x5591529bd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe17ed9d640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe16c0a1280 0x7fe16c0a3740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1700964c0 tx=0x7fe170009270 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 0x5591529bd1f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe168005d50 comp rx=0 tx=0).stop 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 -- 172.21.15.44:0/1129319962 shutdown_connections 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x5591529ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 --2- 172.21.15.44:0/1129319962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bce10 0x5591529bd1f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 -- 172.21.15.44:0/1129319962 >> 172.21.15.44:0/1129319962 conn(0x5591529b9380 msgr2=0x5591529b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 -- 172.21.15.44:0/1129319962 shutdown_connections 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 -- 172.21.15.44:0/1129319962 wait complete. 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.959+0000 7fe187c10740 1 -- start start 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 Processor -- start 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bce10 0x5591529c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591528c22a0 con 0x5591529c7fd0 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591525f9430 con 0x5591529bd730 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591528a4680 con 0x5591529bce10 2024-09-13T17:33:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe187c10740 10 client.0 _ll_register_callbacks cb 0x55915291a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38090/0 (socket says 172.21.15.44:38090) 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 -- 172.21.15.44:0/4231114176 learned_addr learned my addr 172.21.15.44:0/4231114176 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17e59c640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bce10 0x5591529c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 -- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bce10 msgr2=0x5591529c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bce10 0x5591529c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 -- 172.21.15.44:0/4231114176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 msgr2=0x5591529c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 -- 172.21.15.44:0/4231114176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18010d470 con 0x5591529c7fd0 2024-09-13T17:33:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529c5270 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe16802a940 tx=0x7fe168040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e26de8fbb56d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.960+0000 7fe165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe165ffb640 1 -- 172.21.15.44:0/4231114176 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168044020 con 0x5591529c7fd0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe165ffb640 1 -- 172.21.15.44:0/4231114176 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe168003900 con 0x5591529c7fd0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591526144e0 con 0x5591529c7fd0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe165ffb640 1 -- 172.21.15.44:0/4231114176 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe168003ac0 con 0x5591529c7fd0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 10 client.17721 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55915291f620 con 0x5591529c7fd0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe165ffb640 1 -- 172.21.15.44:0/4231114176 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5591526144e0 con 0x5591529c7fd0 2024-09-13T17:33:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe165ffb640 1 -- 172.21.15.44:0/4231114176 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe168049b60 con 0x5591529c7fd0 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 10 client.17721 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 20 client.17721 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x5591529d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 4 client.17721 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.961+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591526257e0 con 0x5591529d7ad0 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.962+0000 7fe17e59c640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x5591529d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.962+0000 7fe17e59c640 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x5591529d9ec0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe0be8a707b5527 server_cookie=9e478869db7e1a8a in_seq=0 out_seq=0 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.962+0000 7fe165ffb640 10 client.17721 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.962+0000 7fe165ffb640 1 -- 172.21.15.44:0/4231114176 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5591526257e0 con 0x5591529d7ad0 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.962+0000 7fe165ffb640 10 client.17721 handle_command_reply: tid=0 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.962+0000 7fe187c10740 1 client.17721 shutdown 2024-09-13T17:33:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 msgr2=0x5591529c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529c5270 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe16802a940 tx=0x7fe168040040 comp rx=0 tx=0).stop 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 shutdown_connections 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x5591529d9ec0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529c5270 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bd730 0x5591529c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe165ffb640 0 client.17721 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 --2- 172.21.15.44:0/4231114176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bce10 0x5591529c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 >> 172.21.15.44:0/4231114176 conn(0x5591529b9380 msgr2=0x5591529bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 shutdown_connections 2024-09-13T17:33:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 1 -- 172.21.15.44:0/4231114176 wait complete. 2024-09-13T17:33:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.963+0000 7fe187c10740 20 client.17721 trim_cache size 0 max 16384 2024-09-13T17:33:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 Processor -- start 2024-09-13T17:33:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 -- start start 2024-09-13T17:33:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bd730 0x5591529cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591529bcb90 con 0x5591529c7fd0 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591529bdca0 con 0x5591529bce10 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.977+0000 7fe187c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591529ba510 con 0x5591529bd730 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bd730 0x5591529cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529bff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38102/0 (socket says 172.21.15.44:38102) 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17dd9b640 1 -- 172.21.15.44:0/3999789524 learned_addr learned my addr 172.21.15.44:0/3999789524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 -- 172.21.15.44:0/3999789524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 msgr2=0x5591529ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 -- 172.21.15.44:0/3999789524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 msgr2=0x5591529bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 -- 172.21.15.44:0/3999789524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591528c1790 con 0x5591529bd730 2024-09-13T17:33:19.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:19.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe17e59c640 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bd730 0x5591529cf2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5591529c6200 tx=0x7fe178005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65756de66b306872 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe1757fa640 1 -- 172.21.15.44:0/3999789524 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178016040 con 0x5591529bd730 2024-09-13T17:33:19.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe1757fa640 1 -- 172.21.15.44:0/3999789524 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe178012e10 con 0x5591529bd730 2024-09-13T17:33:19.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.978+0000 7fe1757fa640 1 -- 172.21.15.44:0/3999789524 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178010430 con 0x5591529bd730 2024-09-13T17:33:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 -- 172.21.15.44:0/3999789524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bd730 msgr2=0x5591529cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bd730 0x5591529cf2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5591529c6200 tx=0x7fe178005e30 comp rx=0 tx=0).stop 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 -- 172.21.15.44:0/3999789524 shutdown_connections 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529bd730 0x5591529cf2a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529c7fd0 0x5591529bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 --2- 172.21.15.44:0/3999789524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 -- 172.21.15.44:0/3999789524 >> 172.21.15.44:0/3999789524 conn(0x5591529b9380 msgr2=0x5591529b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 -- 172.21.15.44:0/3999789524 shutdown_connections 2024-09-13T17:33:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 -- 172.21.15.44:0/3999789524 wait complete. 2024-09-13T17:33:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 -- start start 2024-09-13T17:33:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.979+0000 7fe187c10740 1 Processor -- start 2024-09-13T17:33:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 0x5591529cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x559152a25f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559152a26620 con 0x5591529bd730 2024-09-13T17:33:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559152a26790 con 0x5591529bce10 2024-09-13T17:33:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559152a26950 con 0x5591529c7fd0 2024-09-13T17:33:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 10 client.0 _ll_register_callbacks cb 0x7fe168004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 0x5591529cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 0x5591529cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38104/0 (socket says 172.21.15.44:38104) 2024-09-13T17:33:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 -- 172.21.15.44:0/3565378526 learned_addr learned my addr 172.21.15.44:0/3565378526 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x559152a25f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 -- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 msgr2=0x559152a25f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x559152a25f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 -- 172.21.15.44:0/3565378526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 msgr2=0x5591529cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:19.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 -- 172.21.15.44:0/3565378526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55915291f620 con 0x5591529bd730 2024-09-13T17:33:19.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:19.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe17e59c640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 0x5591529cba60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90366f998b02b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:19.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:19.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe165ffb640 1 -- 172.21.15.44:0/3565378526 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780184b0 con 0x5591529bd730 2024-09-13T17:33:19.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe165ffb640 1 -- 172.21.15.44:0/3565378526 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe178012980 con 0x5591529bd730 2024-09-13T17:33:19.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe168031df0 con 0x5591529bd730 2024-09-13T17:33:19.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe165ffb640 1 -- 172.21.15.44:0/3565378526 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe178012b40 con 0x5591529bd730 2024-09-13T17:33:19.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 10 client.17727 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:19.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.980+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe17800d810 con 0x5591529bd730 2024-09-13T17:33:19.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe165ffb640 1 -- 172.21.15.44:0/3565378526 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe168031df0 con 0x5591529bd730 2024-09-13T17:33:19.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe165ffb640 1 -- 172.21.15.44:0/3565378526 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe178021420 con 0x5591529bd730 2024-09-13T17:33:19.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe187c10740 10 client.17727 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:19.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe187c10740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:19.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe187c10740 20 client.17727 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:19.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe187c10740 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x559152a31180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:19.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe187c10740 4 client.17727 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:33:19.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559152678e30 con 0x5591529d7ad0 2024-09-13T17:33:19.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x559152a31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:19.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe165ffb640 10 client.17727 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe17dd9b640 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x559152a31180 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a93327c016268c1 server_cookie=c82b7d1f9d9ca76b in_seq=0 out_seq=0 2024-09-13T17:33:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.981+0000 7fe165ffb640 1 -- 172.21.15.44:0/3565378526 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fe16804d110 con 0x5591529d7ad0 2024-09-13T17:33:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe165ffb640 10 client.17727 handle_command_reply: tid=0 2024-09-13T17:33:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 client.17727 shutdown 2024-09-13T17:33:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 msgr2=0x5591529cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:19.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 0x5591529cba60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178005ea0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 shutdown_connections 2024-09-13T17:33:19.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5591529d7ad0 0x559152a31180 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591529c7fd0 0x559152a25f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591529bd730 0x5591529cba60 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 --2- 172.21.15.44:0/3565378526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591529bce10 0x5591529cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:19.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe165ffb640 0 client.17727 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:19.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 >> 172.21.15.44:0/3565378526 conn(0x5591529b9380 msgr2=0x5591529bff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:19.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 shutdown_connections 2024-09-13T17:33:19.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 1 -- 172.21.15.44:0/3565378526 wait complete. 2024-09-13T17:33:19.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe187c10740 20 client.17727 trim_cache size 0 max 16384 2024-09-13T17:33:19.998 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:33:19.998 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:33:19.998 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:33:19.998 INFO:teuthology.orchestra.run.smithi044.stdout: "0f04d115-e0f3-4a11-bb07-2031af733979": { 2024-09-13T17:33:19.998 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:33:19.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0f04d115-e0f3-4a11-bb07-2031af733979", 2024-09-13T17:33:19.999 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:33:19.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:33:19.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:33:19.999 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:33:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe16c0a1280 msgr2=0x7fe16c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe16c0a1280 0x7fe16c0a3740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1700964c0 tx=0x7fe170009270 comp rx=0 tx=0).stop 2024-09-13T17:33:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 msgr2=0x7fe18019afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 0x7fe18019afc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe17000eaa0 tx=0x7fe17000ef60 comp rx=0 tx=0).stop 2024-09-13T17:33:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 shutdown_connections 2024-09-13T17:33:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe16c0a1280 0x7fe16c0a3740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1801050e0 0x7fe180196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe18019ab10 0x7fe18019afc0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 --2- 172.21.15.44:0/2438086114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1800fff90 0x7fe1801961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 >> 172.21.15.44:0/2438086114 conn(0x7fe1800fba20 msgr2=0x7fe1800fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 shutdown_connections 2024-09-13T17:33:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:19.982+0000 7fe184ee9640 1 -- 172.21.15.44:0/2438086114 wait complete. 2024-09-13T17:33:20.004 INFO:tasks.cephfs.filesystem:scrub status for tag:0f04d115-e0f3-4a11-bb07-2031af733979 - {'path': '/', 'tag': '0f04d115-e0f3-4a11-bb07-2031af733979', 'options': 'recursive,force'} 2024-09-13T17:33:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:20 smithi100 ceph-mon[35055]: pgmap v203: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:33:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:20 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:33:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:20 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:20 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:33:20.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:20 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:20 smithi044 ceph-mon[31175]: pgmap v203: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:33:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:20 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:33:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:20 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:20 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:33:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:20 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:20 smithi070 ceph-mon[34502]: pgmap v203: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:33:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:20 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:33:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:20 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:20 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:33:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:20 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:22.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:22 smithi100 ceph-mon[35055]: pgmap v204: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:33:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:22 smithi044 ceph-mon[31175]: pgmap v204: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:33:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:22 smithi070 ceph-mon[34502]: pgmap v204: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:33:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:24 smithi100 ceph-mon[35055]: pgmap v205: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-13T17:33:24.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:24 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:33:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:24 smithi044 ceph-mon[31175]: pgmap v205: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-13T17:33:24.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:24 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:33:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:24 smithi070 ceph-mon[34502]: pgmap v205: 129 pgs: 129 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-13T17:33:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:24 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:33:26.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:26 smithi100 ceph-mon[35055]: pgmap v206: 129 pgs: 129 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-13T17:33:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:26 smithi044 ceph-mon[31175]: pgmap v206: 129 pgs: 129 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-13T17:33:26.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:26 smithi070 ceph-mon[34502]: pgmap v206: 129 pgs: 129 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 -- 172.21.15.44:0/2064354946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2040fee20 msgr2=0x7ff2040ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 --2- 172.21.15.44:0/2064354946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2040fee20 0x7ff2040ff200 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009930 tx=0x7ff1f402a420 comp rx=0 tx=0).stop 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 -- 172.21.15.44:0/2064354946 shutdown_connections 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 --2- 172.21.15.44:0/2064354946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2040ff7d0 0x7ff204103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 --2- 172.21.15.44:0/2064354946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff204103f70 0x7ff20410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 --2- 172.21.15.44:0/2064354946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2040fee20 0x7ff2040ff200 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 -- 172.21.15.44:0/2064354946 >> 172.21.15.44:0/2064354946 conn(0x7ff2040fa8f0 msgr2=0x7ff2040fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 -- 172.21.15.44:0/2064354946 shutdown_connections 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.090+0000 7ff20a853640 1 -- 172.21.15.44:0/2064354946 wait complete. 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 Processor -- start 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 -- start start 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2040fee20 0x7ff204196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 0x7ff2041966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20419aaa0 0x7ff20419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20419b490 con 0x7ff204103f70 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20419b600 con 0x7ff2040fee20 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff20a853640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20419b770 con 0x7ff20419aaa0 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff1fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 0x7ff2041966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff1fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 0x7ff2041966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38116/0 (socket says 172.21.15.44:38116) 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff1fbfff640 1 -- 172.21.15.44:0/1210340021 learned_addr learned my addr 172.21.15.44:0/1210340021 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.091+0000 7ff203fff640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2040fee20 0x7ff204196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff208df0640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20419aaa0 0x7ff20419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff1fbfff640 1 -- 172.21.15.44:0/1210340021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20419aaa0 msgr2=0x7ff20419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20419aaa0 0x7ff20419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff1fbfff640 1 -- 172.21.15.44:0/1210340021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2040fee20 msgr2=0x7ff204196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2040fee20 0x7ff204196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff1fbfff640 1 -- 172.21.15.44:0/1210340021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f4009560 con 0x7ff204103f70 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 0x7ff2041966b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00d770 tx=0x7ff1ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff201ffb640 1 -- 172.21.15.44:0/1210340021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec009c30 con 0x7ff204103f70 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2041a2d50 con 0x7ff204103f70 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff201ffb640 1 -- 172.21.15.44:0/1210340021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1ec00de80 con 0x7ff204103f70 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff201ffb640 1 -- 172.21.15.44:0/1210340021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00f620 con 0x7ff204103f70 2024-09-13T17:33:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.092+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2041a31b0 con 0x7ff204103f70 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.095+0000 7ff201ffb640 1 -- 172.21.15.44:0/1210340021 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff1ec010460 con 0x7ff204103f70 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 Processor -- start 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 -- start start 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b86403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b8642cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b859cc30 con 0x5629b8631210 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b859a3c0 con 0x5629b8631b30 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff20d653740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b8595cb0 con 0x5629b8640900 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86315f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38132/0 (socket says 172.21.15.44:38132) 2024-09-13T17:33:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 -- 172.21.15.44:0/2407051370 learned_addr learned my addr 172.21.15.44:0/2407051370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff203fff640 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b86403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b8642cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff201ffb640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1f00a1200 0x7ff1f00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 -- 172.21.15.44:0/2407051370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 msgr2=0x5629b8642cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b8642cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 -- 172.21.15.44:0/2407051370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 msgr2=0x5629b86403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b86403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.096+0000 7ff208df0640 1 -- 172.21.15.44:0/2407051370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b853c910 con 0x5629b8631210 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.097+0000 7ff201ffb640 1 -- 172.21.15.44:0/1210340021 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff1ec0cce90 con 0x7ff204103f70 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.097+0000 7ff208df0640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1f00a1200 0x7ff1f00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.097+0000 7ff208df0640 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86315f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e60 tx=0x7ff1fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cd5d2e300cc571 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.097+0000 7ff1fa7fc640 1 -- 172.21.15.44:0/2407051370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc012a50 con 0x5629b8631210 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.097+0000 7ff1fa7fc640 1 -- 172.21.15.44:0/2407051370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1fc0113e0 con 0x5629b8631210 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.097+0000 7ff1fa7fc640 1 -- 172.21.15.44:0/2407051370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc01a360 con 0x5629b8631210 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 -- 172.21.15.44:0/2407051370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 msgr2=0x5629b86315f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff208df0640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1f00a1200 0x7ff1f00a36c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc011010 tx=0x7ff1fc021000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86315f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e60 tx=0x7ff1fc010740 comp rx=0 tx=0).stop 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 -- 172.21.15.44:0/2407051370 shutdown_connections 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b8642cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b86403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 --2- 172.21.15.44:0/2407051370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86315f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 -- 172.21.15.44:0/2407051370 >> 172.21.15.44:0/2407051370 conn(0x5629b862d8c0 msgr2=0x5629b862dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 -- 172.21.15.44:0/2407051370 shutdown_connections 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 -- 172.21.15.44:0/2407051370 wait complete. 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.098+0000 7ff20d653740 1 -- start start 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 Processor -- start 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b863d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b863db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b8553e80 con 0x5629b8631210 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b8553b20 con 0x5629b8631b30 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b8519260 con 0x5629b8640900 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff20d653740 10 client.0 _ll_register_callbacks cb 0x5629b854edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38138/0 (socket says 172.21.15.44:38138) 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 -- 172.21.15.44:0/3606932616 learned_addr learned my addr 172.21.15.44:0/3606932616 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b863d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff208df0640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b863db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 -- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 msgr2=0x5629b863db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b863db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 -- 172.21.15.44:0/3606932616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 msgr2=0x5629b863d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b863d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff203fff640 1 -- 172.21.15.44:0/3606932616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc009ac0 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.099+0000 7ff208df0640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b863db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff203fff640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86402f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4006010 tx=0x7ff1f40318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8ee62a3f19bd97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff1e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff1e37fe640 1 -- 172.21.15.44:0/3606932616 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4031d70 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff1e37fe640 1 -- 172.21.15.44:0/3606932616 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff1f40044b0 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629b85be680 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff1e37fe640 1 -- 172.21.15.44:0/3606932616 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f40035f0 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 10 client.17739 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5629b8594330 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff1e37fe640 1 -- 172.21.15.44:0/3606932616 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5629b85be680 con 0x5629b8631210 2024-09-13T17:33:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff1e37fe640 1 -- 172.21.15.44:0/3606932616 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1f4046770 con 0x5629b8631210 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 10 client.17739 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 10 client.17739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 20 client.17739 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86503a0 0x5629b8652790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 4 client.17739 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.100+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629b85895d0 con 0x5629b86503a0 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.101+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86503a0 0x5629b8652790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.101+0000 7ff1e37fe640 10 client.17739 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.101+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86503a0 0x5629b8652790 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f4b89e67383c9b server_cookie=2148f446f4175551 in_seq=0 out_seq=0 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff1e37fe640 1 -- 172.21.15.44:0/3606932616 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5629b85895d0 con 0x5629b86503a0 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff1e37fe640 10 client.17739 handle_command_reply: tid=0 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 client.17739 shutdown 2024-09-13T17:33:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 msgr2=0x5629b86402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86402f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4006010 tx=0x7ff1f40318a0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 shutdown_connections 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86503a0 0x5629b8652790 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8640900 0x5629b863db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b863d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff1e37fe640 0 client.17739 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 --2- 172.21.15.44:0/3606932616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8631210 0x5629b86402f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 >> 172.21.15.44:0/3606932616 conn(0x5629b862d8c0 msgr2=0x5629b8633600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 shutdown_connections 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.102+0000 7ff20d653740 1 -- 172.21.15.44:0/3606932616 wait complete. 2024-09-13T17:33:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.103+0000 7ff20d653740 20 client.17739 trim_cache size 0 max 16384 2024-09-13T17:33:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.115+0000 7ff20d653740 1 Processor -- start 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.115+0000 7ff20d653740 1 -- start start 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.115+0000 7ff20d653740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b863fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.115+0000 7ff20d653740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8647680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff20d653740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b8647bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff20d653740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b86328a0 con 0x5629b8640900 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff20d653740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b8638360 con 0x5629b8631b30 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff20d653740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b863e680 con 0x5629b8631210 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b8647bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b8647bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38142/0 (socket says 172.21.15.44:38142) 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 -- 172.21.15.44:0/65490154 learned_addr learned my addr 172.21.15.44:0/65490154 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff203fff640 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b863fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8647680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 -- 172.21.15.44:0/65490154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 msgr2=0x5629b863fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b863fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 -- 172.21.15.44:0/65490154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 msgr2=0x5629b8647680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8647680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 -- 172.21.15.44:0/65490154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b8536300 con 0x5629b8640900 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8647680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.116+0000 7ff208df0640 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b8647bc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc010c40 tx=0x7ff1fc027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680072ed4af2326e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff1faffd640 1 -- 172.21.15.44:0/65490154 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc02c070 con 0x5629b8640900 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff1faffd640 1 -- 172.21.15.44:0/65490154 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1fc012890 con 0x5629b8640900 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff1faffd640 1 -- 172.21.15.44:0/65490154 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0117b0 con 0x5629b8640900 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 -- 172.21.15.44:0/65490154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 msgr2=0x5629b8647bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b8647bc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc010c40 tx=0x7ff1fc027000 comp rx=0 tx=0).stop 2024-09-13T17:33:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 -- 172.21.15.44:0/65490154 shutdown_connections 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b8647bc0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8647680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 --2- 172.21.15.44:0/65490154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b863fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 -- 172.21.15.44:0/65490154 >> 172.21.15.44:0/65490154 conn(0x5629b862d8c0 msgr2=0x5629b863be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 -- 172.21.15.44:0/65490154 shutdown_connections 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 -- 172.21.15.44:0/65490154 wait complete. 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.117+0000 7ff20d653740 1 -- start start 2024-09-13T17:33:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 Processor -- start 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b8643c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8644150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b869e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b869ed80 con 0x5629b8640900 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b869eef0 con 0x5629b8631b30 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b869f0b0 con 0x5629b8631210 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff20d653740 10 client.0 _ll_register_callbacks cb 0x7ff1fc009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b869e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b869e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38156/0 (socket says 172.21.15.44:38156) 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 -- 172.21.15.44:0/2073473203 learned_addr learned my addr 172.21.15.44:0/2073473203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff203fff640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b8643c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8644150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 -- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 msgr2=0x5629b8643c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b8643c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 -- 172.21.15.44:0/2073473203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 msgr2=0x5629b8644150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8644150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.118+0000 7ff208df0640 1 -- 172.21.15.44:0/2073473203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629b853c910 con 0x5629b8640900 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1fbfff640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8644150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff203fff640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b8643c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff208df0640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b869e7c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc033040 tx=0x7ff1fc01bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80545b4243177641 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1e37fe640 1 -- 172.21.15.44:0/2073473203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc02c040 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1e37fe640 1 -- 172.21.15.44:0/2073473203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff1fc004df0 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1fc0113e0 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1e37fe640 1 -- 172.21.15.44:0/2073473203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1fc01be70 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 10 client.17745 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5629b855b8e0 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1e37fe640 1 -- 172.21.15.44:0/2073473203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1fc0113e0 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff1e37fe640 1 -- 172.21.15.44:0/2073473203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1fc041070 con 0x5629b8640900 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 10 client.17745 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 10 client.17745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 20 client.17745 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.119+0000 7ff20d653740 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86a76e0 0x5629b86a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.120+0000 7ff20d653740 4 client.17745 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:33:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.120+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629b8289300 con 0x5629b86a76e0 2024-09-13T17:33:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.120+0000 7ff203fff640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86a76e0 0x5629b86a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.120+0000 7ff1e37fe640 10 client.17745 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.120+0000 7ff203fff640 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86a76e0 0x5629b86a9b00 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c356b807dc01c server_cookie=433a150ea791cd3f in_seq=0 out_seq=0 2024-09-13T17:33:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff1e37fe640 1 -- 172.21.15.44:0/2073473203 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629b8289300 con 0x5629b86a76e0 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff1e37fe640 10 client.17745 handle_command_reply: tid=0 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 client.17745 shutdown 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 msgr2=0x5629b869e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b869e7c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc033040 tx=0x7ff1fc01bc80 comp rx=0 tx=0).stop 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 shutdown_connections 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5629b86a76e0 0x5629b86a9b00 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629b8640900 0x5629b869e7c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629b8631b30 0x5629b8644150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 --2- 172.21.15.44:0/2073473203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629b8631210 0x5629b8643c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff1e37fe640 0 client.17745 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 >> 172.21.15.44:0/2073473203 conn(0x5629b862d8c0 msgr2=0x5629b863bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 shutdown_connections 2024-09-13T17:33:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 1 -- 172.21.15.44:0/2073473203 wait complete. 2024-09-13T17:33:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.135+0000 7ff20d653740 20 client.17745 trim_cache size 0 max 16384 2024-09-13T17:33:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:33:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:33:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:33:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1f00a1200 msgr2=0x7ff1f00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1f00a1200 0x7ff1f00a36c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc011010 tx=0x7ff1fc021000 comp rx=0 tx=0).stop 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 msgr2=0x7ff2041966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 0x7ff2041966b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00d770 tx=0x7ff1ec00dc30 comp rx=0 tx=0).stop 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 shutdown_connections 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff1f00a1200 0x7ff1f00a36c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20419aaa0 0x7ff20419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff204103f70 0x7ff2041966b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 --2- 172.21.15.44:0/1210340021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2040fee20 0x7ff204196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 >> 172.21.15.44:0/1210340021 conn(0x7ff2040fa8f0 msgr2=0x7ff204068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 shutdown_connections 2024-09-13T17:33:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:27.136+0000 7ff20a853640 1 -- 172.21.15.44:0/1210340021 wait complete. 2024-09-13T17:33:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:28 smithi044 ceph-mon[31175]: pgmap v207: 129 pgs: 129 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:28 smithi070 ceph-mon[34502]: pgmap v207: 129 pgs: 129 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:28 smithi100 ceph-mon[35055]: pgmap v207: 129 pgs: 129 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:30.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:30 smithi044 ceph-mon[31175]: pgmap v208: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-09-13T17:33:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:30 smithi070 ceph-mon[34502]: pgmap v208: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-09-13T17:33:30.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:30 smithi100 ceph-mon[35055]: pgmap v208: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-09-13T17:33:32.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:32 smithi044 ceph-mon[31175]: pgmap v209: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:32.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:32 smithi070 ceph-mon[34502]: pgmap v209: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:32.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:32 smithi100 ceph-mon[35055]: pgmap v209: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:34.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:34 smithi044 ceph-mon[31175]: pgmap v210: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:34 smithi070 ceph-mon[34502]: pgmap v210: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:34.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:34 smithi100 ceph-mon[35055]: pgmap v210: 129 pgs: 129 active+clean; 3.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:36.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:36 smithi044 ceph-mon[31175]: pgmap v211: 129 pgs: 129 active+clean; 3.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:33:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:36 smithi070 ceph-mon[34502]: pgmap v211: 129 pgs: 129 active+clean; 3.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:33:36.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:36 smithi100 ceph-mon[35055]: pgmap v211: 129 pgs: 129 active+clean; 3.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:33:38.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:38 smithi044 ceph-mon[31175]: pgmap v212: 129 pgs: 129 active+clean; 3.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:33:38.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:38 smithi070 ceph-mon[34502]: pgmap v212: 129 pgs: 129 active+clean; 3.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:33:38.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:38 smithi100 ceph-mon[35055]: pgmap v212: 129 pgs: 129 active+clean; 3.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.234+0000 7ff02246b640 1 -- 172.21.15.44:0/85268969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0ff7d0 msgr2=0x7ff01c103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.234+0000 7ff02246b640 1 --2- 172.21.15.44:0/85268969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0ff7d0 0x7ff01c103810 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff004009930 tx=0x7ff00402a420 comp rx=0 tx=0).stop 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.234+0000 7ff02246b640 1 -- 172.21.15.44:0/85268969 shutdown_connections 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.234+0000 7ff02246b640 1 --2- 172.21.15.44:0/85268969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c103f70 0x7ff01c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.234+0000 7ff02246b640 1 --2- 172.21.15.44:0/85268969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0ff7d0 0x7ff01c103810 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.234+0000 7ff02246b640 1 --2- 172.21.15.44:0/85268969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff01c0fee20 0x7ff01c0ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- 172.21.15.44:0/85268969 >> 172.21.15.44:0/85268969 conn(0x7ff01c0fa8f0 msgr2=0x7ff01c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- 172.21.15.44:0/85268969 shutdown_connections 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- 172.21.15.44:0/85268969 wait complete. 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 Processor -- start 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- start start 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 0x7ff01c196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff01c103f70 0x7ff01c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c19aa90 0x7ff01c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c19b480 con 0x7ff01c0fee20 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c19b5f0 con 0x7ff01c19aa90 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.235+0000 7ff02246b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c19b760 con 0x7ff01c103f70 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 0x7ff01c196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 0x7ff01c196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52756/0 (socket says 172.21.15.44:52756) 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 -- 172.21.15.44:0/2230973479 learned_addr learned my addr 172.21.15.44:0/2230973479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff013fff640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff01c103f70 0x7ff01c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff020a08640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c19aa90 0x7ff01c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 -- 172.21.15.44:0/2230973479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff01c103f70 msgr2=0x7ff01c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff01c103f70 0x7ff01c1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 -- 172.21.15.44:0/2230973479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c19aa90 msgr2=0x7ff01c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c19aa90 0x7ff01c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 -- 172.21.15.44:0/2230973479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004009560 con 0x7ff01c0fee20 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff020a08640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c19aa90 0x7ff01c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff01bfff640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 0x7ff01c196160 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00c920 tx=0x7ff00c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff019ffb640 1 -- 172.21.15.44:0/2230973479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c007c70 con 0x7ff01c0fee20 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff019ffb640 1 -- 172.21.15.44:0/2230973479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff00c007dd0 con 0x7ff01c0fee20 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff019ffb640 1 -- 172.21.15.44:0/2230973479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c018730 con 0x7ff01c0fee20 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01c1a2d40 con 0x7ff01c0fee20 2024-09-13T17:33:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.236+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c1a31c0 con 0x7ff01c0fee20 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.239+0000 7ff019ffb640 1 -- 172.21.15.44:0/2230973479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff00c00f450 con 0x7ff01c0fee20 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.239+0000 7ff02526d740 1 Processor -- start 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.239+0000 7ff02526d740 1 -- start start 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff02526d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x565209219710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff02526d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219c50 0x565209223fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff02526d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff02526d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x565209184c30 con 0x565209219c50 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff02526d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652091823c0 con 0x565209219330 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff02526d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520917dcb0 con 0x5652092244f0 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff019ffb640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0080a1280 0x7ff0080a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219c50 0x565209223fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092268e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:44134/0 (socket says 172.21.15.44:44134) 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 -- 172.21.15.44:0/299442125 learned_addr learned my addr 172.21.15.44:0/299442125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff020a08640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0080a1280 0x7ff0080a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff020a08640 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x565209219710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff019ffb640 1 -- 172.21.15.44:0/2230973479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff00c0cd160 con 0x7ff01c0fee20 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 -- 172.21.15.44:0/299442125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 msgr2=0x565209219710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x565209219710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 -- 172.21.15.44:0/299442125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219c50 msgr2=0x565209223fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219c50 0x565209223fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 -- 172.21.15.44:0/299442125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565209124910 con 0x5652092244f0 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.240+0000 7ff013fff640 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092268e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff004009900 tx=0x7ff004031670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340105048ddd7971 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff020a08640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0080a1280 0x7ff0080a3740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff014005c40 tx=0x7ff0140094a0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff0127fc640 1 -- 172.21.15.44:0/299442125 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004004300 con 0x5652092244f0 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff0127fc640 1 -- 172.21.15.44:0/299442125 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff004031df0 con 0x5652092244f0 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff0127fc640 1 -- 172.21.15.44:0/299442125 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004003c90 con 0x5652092244f0 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff02526d740 1 -- 172.21.15.44:0/299442125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 msgr2=0x5652092268e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff02526d740 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092268e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff004009900 tx=0x7ff004031670 comp rx=0 tx=0).stop 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff02526d740 1 -- 172.21.15.44:0/299442125 shutdown_connections 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff02526d740 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092268e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff02526d740 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219c50 0x565209223fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.241+0000 7ff02526d740 1 --2- 172.21.15.44:0/299442125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x565209219710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- 172.21.15.44:0/299442125 >> 172.21.15.44:0/299442125 conn(0x5652092158c0 msgr2=0x565209215ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- 172.21.15.44:0/299442125 shutdown_connections 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- 172.21.15.44:0/299442125 wait complete. 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- start start 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 Processor -- start 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 0x565209223ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209221150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520913be80 con 0x565209219330 2024-09-13T17:33:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520913bb20 con 0x565209219c50 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.242+0000 7ff02526d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565209101260 con 0x5652092244f0 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff02526d740 10 client.0 _ll_register_callbacks cb 0x565209136df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 0x565209223ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 0x565209223ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52780/0 (socket says 172.21.15.44:52780) 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 -- 172.21.15.44:0/1722347526 learned_addr learned my addr 172.21.15.44:0/1722347526 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff020a08640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff013fff640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209221150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 -- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 msgr2=0x5652092217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 -- 172.21.15.44:0/1722347526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 msgr2=0x565209221150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209221150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 -- 172.21.15.44:0/1722347526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c10c300 con 0x565209219330 2024-09-13T17:33:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff01bfff640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 0x565209223ed0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff00c099c00 tx=0x7ff00c069e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cf6da5bf14bbae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7feffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7feffb7fe640 1 -- 172.21.15.44:0/1722347526 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0658a0 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7feffb7fe640 1 -- 172.21.15.44:0/1722347526 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff00c09a600 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.243+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652091a6680 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7feffb7fe640 1 -- 172.21.15.44:0/1722347526 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00c099f00 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 10 client.17757 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56520917c330 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7feffb7fe640 1 -- 172.21.15.44:0/1722347526 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5652091a6680 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7feffb7fe640 1 -- 172.21.15.44:0/1722347526 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff00c016020 con 0x565209219330 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 10 client.17757 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 20 client.17757 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209233e10 0x565209236200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 4 client.17757 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652091715d0 con 0x565209233e10 2024-09-13T17:33:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.244+0000 7ff013fff640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209233e10 0x565209236200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.245+0000 7feffb7fe640 10 client.17757 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.245+0000 7ff013fff640 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209233e10 0x565209236200 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1276d71c266dd6af server_cookie=b3fa7895b21e632a in_seq=0 out_seq=0 2024-09-13T17:33:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.245+0000 7feffb7fe640 1 -- 172.21.15.44:0/1722347526 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5652091715d0 con 0x565209233e10 2024-09-13T17:33:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.245+0000 7feffb7fe640 10 client.17757 handle_command_reply: tid=0 2024-09-13T17:33:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.245+0000 7ff02526d740 1 client.17757 shutdown 2024-09-13T17:33:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 msgr2=0x565209223ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 0x565209223ed0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff00c099c00 tx=0x7ff00c069e80 comp rx=0 tx=0).stop 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 shutdown_connections 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209233e10 0x565209236200 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652092244f0 0x5652092217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209221150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7feffb7fe640 0 client.17757 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 --2- 172.21.15.44:0/1722347526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565209219330 0x565209223ed0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 >> 172.21.15.44:0/1722347526 conn(0x5652092158c0 msgr2=0x565209214b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 shutdown_connections 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 1 -- 172.21.15.44:0/1722347526 wait complete. 2024-09-13T17:33:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.246+0000 7ff02526d740 20 client.17757 trim_cache size 0 max 16384 2024-09-13T17:33:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.258+0000 7ff02526d740 1 Processor -- start 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.258+0000 7ff02526d740 1 -- start start 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff02526d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x56520922b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff02526d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219c50 0x565209223a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff02526d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x56520921fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff02526d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x565209216570 con 0x5652092244f0 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff02526d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520921f680 con 0x565209219330 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff02526d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565209221ae0 con 0x565209219c50 2024-09-13T17:33:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x56520921fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x56520921fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52788/0 (socket says 172.21.15.44:52788) 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 -- 172.21.15.44:0/103578309 learned_addr learned my addr 172.21.15.44:0/103578309 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff01bfff640 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x56520922b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff013fff640 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219c50 0x565209223a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 -- 172.21.15.44:0/103578309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219c50 msgr2=0x565209223a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219c50 0x565209223a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 -- 172.21.15.44:0/103578309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 msgr2=0x56520922b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x56520922b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 -- 172.21.15.44:0/103578309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56520911e300 con 0x5652092244f0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.259+0000 7ff020a08640 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x56520921fa00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff014009fc0 tx=0x7ff014009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0179dce3cb195a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff012ffd640 1 -- 172.21.15.44:0/103578309 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014026070 con 0x5652092244f0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff012ffd640 1 -- 172.21.15.44:0/103578309 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff014012dc0 con 0x5652092244f0 2024-09-13T17:33:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff012ffd640 1 -- 172.21.15.44:0/103578309 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01402b470 con 0x5652092244f0 2024-09-13T17:33:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 -- 172.21.15.44:0/103578309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 msgr2=0x56520921fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x56520921fa00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff014009fc0 tx=0x7ff014009280 comp rx=0 tx=0).stop 2024-09-13T17:33:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 -- 172.21.15.44:0/103578309 shutdown_connections 2024-09-13T17:33:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219c50 0x565209223a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x56520921fa00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 --2- 172.21.15.44:0/103578309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219330 0x56520922b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 -- 172.21.15.44:0/103578309 >> 172.21.15.44:0/103578309 conn(0x5652092158c0 msgr2=0x56520921c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 -- 172.21.15.44:0/103578309 shutdown_connections 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.260+0000 7ff02526d740 1 -- 172.21.15.44:0/103578309 wait complete. 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 -- start start 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 Processor -- start 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219330 0x565209287630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209287b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x565209282c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x565209283230 con 0x5652092244f0 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652092833a0 con 0x565209219c50 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565209283560 con 0x565209219330 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff02526d740 10 client.0 _ll_register_callbacks cb 0x7ff004031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff020a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x565209282c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff020a08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x565209282c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52796/0 (socket says 172.21.15.44:52796) 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff020a08640 1 -- 172.21.15.44:0/451576905 learned_addr learned my addr 172.21.15.44:0/451576905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff013fff640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209287b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.261+0000 7ff01bfff640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219330 0x565209287630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff020a08640 1 -- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219330 msgr2=0x565209287630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff020a08640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219330 0x565209287630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff020a08640 1 -- 172.21.15.44:0/451576905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 msgr2=0x565209287b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff020a08640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209287b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff020a08640 1 -- 172.21.15.44:0/451576905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01401a040 con 0x5652092244f0 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff01bfff640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219330 0x565209287630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff013fff640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209287b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff020a08640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x565209282c70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff014021ee0 tx=0x7ff014004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c389f3839d800793 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7feffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7feffb7fe640 1 -- 172.21.15.44:0/451576905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014026040 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7feffb7fe640 1 -- 172.21.15.44:0/451576905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff014021a80 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff004031df0 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7feffb7fe640 1 -- 172.21.15.44:0/451576905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff014009910 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff02526d740 10 client.17763 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652091438e0 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7feffb7fe640 1 -- 172.21.15.44:0/451576905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff004031df0 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7feffb7fe640 1 -- 172.21.15.44:0/451576905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff014032030 con 0x5652092244f0 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff02526d740 10 client.17763 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.262+0000 7ff02526d740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7ff02526d740 20 client.17763 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7ff02526d740 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209290530 0x565209292920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7ff02526d740 4 client.17763 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565208e71300 con 0x565209290530 2024-09-13T17:33:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7ff01bfff640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209290530 0x565209292920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7feffb7fe640 10 client.17763 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.263+0000 7ff01bfff640 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209290530 0x565209292920 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04fba93e388bee3 server_cookie=ec44c2f37ba6362 in_seq=0 out_seq=0 2024-09-13T17:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.277+0000 7feffb7fe640 1 -- 172.21.15.44:0/451576905 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565208e71300 con 0x565209290530 2024-09-13T17:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.277+0000 7feffb7fe640 10 client.17763 handle_command_reply: tid=0 2024-09-13T17:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.277+0000 7ff02526d740 1 client.17763 shutdown 2024-09-13T17:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 msgr2=0x565209282c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x565209282c70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff014021ee0 tx=0x7ff014004cf0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 shutdown_connections 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x565209290530 0x565209292920 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652092244f0 0x565209282c70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565209219c50 0x565209287b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7feffb7fe640 0 client.17763 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 --2- 172.21.15.44:0/451576905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565209219330 0x565209287630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 >> 172.21.15.44:0/451576905 conn(0x5652092158c0 msgr2=0x56520921f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 shutdown_connections 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 1 -- 172.21.15.44:0/451576905 wait complete. 2024-09-13T17:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.278+0000 7ff02526d740 20 client.17763 trim_cache size 0 max 16384 2024-09-13T17:33:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:33:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:33:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:33:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0080a1280 msgr2=0x7ff0080a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0080a1280 0x7ff0080a3740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff014005c40 tx=0x7ff0140094a0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 msgr2=0x7ff01c196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 0x7ff01c196160 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00c920 tx=0x7ff00c00cde0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 shutdown_connections 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0080a1280 0x7ff0080a3740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff01c103f70 0x7ff01c1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff01c19aa90 0x7ff01c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 --2- 172.21.15.44:0/2230973479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff01c0fee20 0x7ff01c196160 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 >> 172.21.15.44:0/2230973479 conn(0x7ff01c0fa8f0 msgr2=0x7ff01c068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 shutdown_connections 2024-09-13T17:33:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:40.279+0000 7ff02246b640 1 -- 172.21.15.44:0/2230973479 wait complete. 2024-09-13T17:33:40.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:40 smithi044 ceph-mon[31175]: pgmap v213: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:40.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:40 smithi070 ceph-mon[34502]: pgmap v213: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:40.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:40 smithi100 ceph-mon[35055]: pgmap v213: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:42.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:42 smithi044 ceph-mon[31175]: pgmap v214: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:42.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:42 smithi070 ceph-mon[34502]: pgmap v214: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:42.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:42 smithi100 ceph-mon[35055]: pgmap v214: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:44.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:44 smithi044 ceph-mon[31175]: pgmap v215: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:44.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:44 smithi070 ceph-mon[34502]: pgmap v215: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:44.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:44 smithi100 ceph-mon[35055]: pgmap v215: 129 pgs: 129 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:33:46.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:46 smithi044 ceph-mon[31175]: pgmap v216: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:46.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:46 smithi070 ceph-mon[34502]: pgmap v216: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:46.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:46 smithi100 ceph-mon[35055]: pgmap v216: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:47.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:47 smithi044 ceph-mon[31175]: pgmap v217: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:33:47.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:47 smithi070 ceph-mon[34502]: pgmap v217: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:33:47.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:47 smithi100 ceph-mon[35055]: pgmap v217: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:33:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:49 smithi044 ceph-mon[31175]: pgmap v218: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:49 smithi070 ceph-mon[34502]: pgmap v218: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:50.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:33:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:49 smithi100 ceph-mon[35055]: pgmap v218: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 -- 172.21.15.44:0/4161951993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 msgr2=0x7f26f810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 --2- 172.21.15.44:0/4161951993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f810bb00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009cd0 tx=0x7f26f002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 -- 172.21.15.44:0/4161951993 shutdown_connections 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 --2- 172.21.15.44:0/4161951993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f80ff810 0x7f26f8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 --2- 172.21.15.44:0/4161951993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f810bb00 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 --2- 172.21.15.44:0/4161951993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f80fee60 0x7f26f80ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 -- 172.21.15.44:0/4161951993 >> 172.21.15.44:0/4161951993 conn(0x7f26f8075c60 msgr2=0x7f26f80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 -- 172.21.15.44:0/4161951993 shutdown_connections 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.121+0000 7f26ff55e640 1 -- 172.21.15.44:0/4161951993 wait complete. 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 Processor -- start 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 -- start start 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f80fee60 0x7f26f8196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f819ab30 0x7f26f819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819b520 con 0x7f26f8103fb0 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819b690 con 0x7f26f819ab30 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26ff55e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819b800 con 0x7f26f80fee60 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f8196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48304/0 (socket says 172.21.15.44:48304) 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 -- 172.21.15.44:0/1306967972 learned_addr learned my addr 172.21.15.44:0/1306967972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f80fee60 0x7f26f8196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fdafb640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f819ab30 0x7f26f819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 -- 172.21.15.44:0/1306967972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f80fee60 msgr2=0x7f26f8196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f80fee60 0x7f26f8196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 -- 172.21.15.44:0/1306967972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f819ab30 msgr2=0x7f26f819afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f819ab30 0x7f26f819afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.122+0000 7f26fcaf9640 1 -- 172.21.15.44:0/1306967972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009900 con 0x7f26f8103fb0 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.123+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f8196740 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c970 tx=0x7f26e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.123+0000 7f26ee7fc640 1 -- 172.21.15.44:0/1306967972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8007bb0 con 0x7f26f8103fb0 2024-09-13T17:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.123+0000 7f26ee7fc640 1 -- 172.21.15.44:0/1306967972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26e8007d10 con 0x7f26f8103fb0 2024-09-13T17:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.123+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f81a2de0 con 0x7f26f8103fb0 2024-09-13T17:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.123+0000 7f26ee7fc640 1 -- 172.21.15.44:0/1306967972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8018660 con 0x7f26f8103fb0 2024-09-13T17:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.123+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f81a3230 con 0x7f26f8103fb0 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.125+0000 7f26ee7fc640 1 -- 172.21.15.44:0/1306967972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f26e800f450 con 0x7f26f8103fb0 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.125+0000 7f2702280740 1 Processor -- start 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 -- start start 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b60532f20 con 0x560b605cc220 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b605306b0 con 0x560b605d73e0 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f2702280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6052bfa0 con 0x560b605ccb40 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48308/0 (socket says 172.21.15.44:48308) 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 -- 172.21.15.44:0/63452465 learned_addr learned my addr 172.21.15.44:0/63452465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26ee7fc640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26e40a1280 0x7f26e40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 -- 172.21.15.44:0/63452465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 msgr2=0x560b605d6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605d6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 -- 172.21.15.44:0/63452465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 msgr2=0x560b605d97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 -- 172.21.15.44:0/63452465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b604e6840 con 0x560b605cc220 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605d6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26ee7fc640 1 -- 172.21.15.44:0/1306967972 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f26e80ccf40 con 0x7f26f8103fb0 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.126+0000 7f26fdafb640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26e40a1280 0x7f26e40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.127+0000 7f26fdafb640 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605cc600 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f26f002ae00 tx=0x7f26f00314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad534195b4a790c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.127+0000 7f26e37fe640 1 -- 172.21.15.44:0/63452465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f004d030 con 0x560b605cc220 2024-09-13T17:33:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.127+0000 7f26e37fe640 1 -- 172.21.15.44:0/63452465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26f0030700 con 0x560b605cc220 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.127+0000 7f26fdafb640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26e40a1280 0x7f26e40a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26f004d000 tx=0x7f26f004e000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.127+0000 7f26e37fe640 1 -- 172.21.15.44:0/63452465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00048a0 con 0x560b605cc220 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 -- 172.21.15.44:0/63452465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 msgr2=0x560b605cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605cc600 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f26f002ae00 tx=0x7f26f00314b0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 -- 172.21.15.44:0/63452465 shutdown_connections 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605d6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 --2- 172.21.15.44:0/63452465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605cc600 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 -- 172.21.15.44:0/63452465 >> 172.21.15.44:0/63452465 conn(0x560b605c8790 msgr2=0x560b605c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 -- 172.21.15.44:0/63452465 shutdown_connections 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 -- 172.21.15.44:0/63452465 wait complete. 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 -- start start 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.128+0000 7f2702280740 1 Processor -- start 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605cc220 0x560b605d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 0x560b605d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b604cd2a0 con 0x560b605ccb40 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b60204430 con 0x560b605d73e0 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b604af680 con 0x560b605cc220 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 10 client.0 _ll_register_callbacks cb 0x560b60525610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 0x560b605d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 0x560b605d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48322/0 (socket says 172.21.15.44:48322) 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 -- 172.21.15.44:0/987726887 learned_addr learned my addr 172.21.15.44:0/987726887 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fdafb640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605cc220 0x560b605d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 -- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605cc220 msgr2=0x560b605d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605cc220 0x560b605d6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 -- 172.21.15.44:0/987726887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 msgr2=0x560b605d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 -- 172.21.15.44:0/987726887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f810c340 con 0x560b605ccb40 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605cc220 0x560b605d6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 0x560b605d4090 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c940 tx=0x7f26e8007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130cfe3d27db09d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26e0ff9640 1 -- 172.21.15.44:0/987726887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e80bb040 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f26e0ff9640 1 -- 172.21.15.44:0/987726887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f26e809a3e0 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.129+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b602307e0 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f26e0ff9640 1 -- 172.21.15.44:0/987726887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e8099640 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 10 client.17775 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b6052a620 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f26e0ff9640 1 -- 172.21.15.44:0/987726887 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560b602307e0 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f26e0ff9640 1 -- 172.21.15.44:0/987726887 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26e8016020 con 0x560b605ccb40 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 10 client.17775 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:50.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 20 client.17775 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b605e7110 0x560b605e9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 4 client.17775 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b60283e30 con 0x560b605e7110 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.130+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b605e7110 0x560b605e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b605e7110 0x560b605e9500 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fc56fc4376d465 server_cookie=38b167fb2b7e7ea7 in_seq=0 out_seq=0 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f26e0ff9640 10 client.17775 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f26e0ff9640 1 -- 172.21.15.44:0/987726887 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560b60283e30 con 0x560b605e7110 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f26e0ff9640 10 client.17775 handle_command_reply: tid=0 2024-09-13T17:33:50.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f2702280740 1 client.17775 shutdown 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 msgr2=0x560b605d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.131+0000 7f2702280740 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 0x560b605d4090 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c940 tx=0x7f26e8007e80 comp rx=0 tx=0).stop 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 shutdown_connections 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b605e7110 0x560b605e9500 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605d73e0 0x560b605d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f26e0ff9640 0 client.17775 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605ccb40 0x560b605d4090 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 --2- 172.21.15.44:0/987726887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605cc220 0x560b605d6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 >> 172.21.15.44:0/987726887 conn(0x560b605c8790 msgr2=0x560b605c3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 shutdown_connections 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 1 -- 172.21.15.44:0/987726887 wait complete. 2024-09-13T17:33:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.132+0000 7f2702280740 20 client.17775 trim_cache size 0 max 16384 2024-09-13T17:33:50.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 Processor -- start 2024-09-13T17:33:50.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 -- start start 2024-09-13T17:33:50.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605cc220 0x560b605c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 0x560b605cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b602307e0 con 0x560b605d73e0 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b605c2b40 con 0x560b605cc220 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f2702280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b605e3e80 con 0x560b605ccb40 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f26fdafb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 0x560b605cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f26fdafb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 0x560b605cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48336/0 (socket says 172.21.15.44:48336) 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.146+0000 7f26fdafb640 1 -- 172.21.15.44:0/1832228634 learned_addr learned my addr 172.21.15.44:0/1832228634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605cc220 0x560b605c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fdafb640 1 -- 172.21.15.44:0/1832228634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 msgr2=0x560b605cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fdafb640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fdafb640 1 -- 172.21.15.44:0/1832228634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605cc220 msgr2=0x560b605c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fdafb640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605cc220 0x560b605c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fdafb640 1 -- 172.21.15.44:0/1832228634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b604cc790 con 0x560b605d73e0 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605cc220 0x560b605c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26fdafb640 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 0x560b605cf840 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f26f00095f0 tx=0x7f26f00316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2006423a75de089 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26e3fff640 1 -- 172.21.15.44:0/1832228634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0058070 con 0x560b605d73e0 2024-09-13T17:33:50.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26e3fff640 1 -- 172.21.15.44:0/1832228634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26f0004660 con 0x560b605d73e0 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.147+0000 7f26e3fff640 1 -- 172.21.15.44:0/1832228634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0048530 con 0x560b605d73e0 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 -- 172.21.15.44:0/1832228634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 msgr2=0x560b605cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 0x560b605cf840 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f26f00095f0 tx=0x7f26f00316a0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 -- 172.21.15.44:0/1832228634 shutdown_connections 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605ccb40 0x560b605cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605d73e0 0x560b605cf840 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 --2- 172.21.15.44:0/1832228634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605cc220 0x560b605c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 -- 172.21.15.44:0/1832228634 >> 172.21.15.44:0/1832228634 conn(0x560b605c8790 msgr2=0x560b605c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 -- 172.21.15.44:0/1832228634 shutdown_connections 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 -- 172.21.15.44:0/1832228634 wait complete. 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 -- start start 2024-09-13T17:33:50.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.148+0000 7f2702280740 1 Processor -- start 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605ccb40 0x560b605de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605d73e0 0x560b605da510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b605dabe0 con 0x560b605cc220 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b605dad50 con 0x560b605ccb40 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b605daf10 con 0x560b605d73e0 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 10 client.0 _ll_register_callbacks cb 0x7f26e80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605dde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48340/0 (socket says 172.21.15.44:48340) 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 -- 172.21.15.44:0/2280025590 learned_addr learned my addr 172.21.15.44:0/2280025590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fdafb640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605d73e0 0x560b605da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605ccb40 0x560b605de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 -- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605d73e0 msgr2=0x560b605da510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605d73e0 0x560b605da510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 -- 172.21.15.44:0/2280025590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605ccb40 msgr2=0x560b605de340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605ccb40 0x560b605de340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 -- 172.21.15.44:0/2280025590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b604e6840 con 0x560b605cc220 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26fd2fa640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605dde00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x560b605e9de0 tx=0x7f26f4013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd30728fbac69df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26e0ff9640 1 -- 172.21.15.44:0/2280025590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4017040 con 0x560b605cc220 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f26e0ff9640 1 -- 172.21.15.44:0/2280025590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f26f401c5a0 con 0x560b605cc220 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.149+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b6021f4e0 con 0x560b605cc220 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f26e0ff9640 1 -- 172.21.15.44:0/2280025590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f401c760 con 0x560b605cc220 2024-09-13T17:33:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 10 client.17781 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b604f1ea0 con 0x560b605cc220 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f26e0ff9640 1 -- 172.21.15.44:0/2280025590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560b6021f4e0 con 0x560b605cc220 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f26e0ff9640 1 -- 172.21.15.44:0/2280025590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26f400f420 con 0x560b605cc220 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 10 client.17781 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 20 client.17781 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b6063e750 0x560b60640b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 4 client.17781 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b60641380 con 0x560b6063e750 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.150+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b6063e750 0x560b60640b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f26e0ff9640 10 client.17781 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f26fcaf9640 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b6063e750 0x560b60640b70 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d30691b75c83683 server_cookie=7094b33422b124c1 in_seq=0 out_seq=0 2024-09-13T17:33:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f26e0ff9640 1 -- 172.21.15.44:0/2280025590 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560b60641380 con 0x560b6063e750 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f26e0ff9640 10 client.17781 handle_command_reply: tid=0 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 client.17781 shutdown 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 msgr2=0x560b605dde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605dde00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x560b605e9de0 tx=0x7f26f4013830 comp rx=0 tx=0).stop 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 shutdown_connections 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b6063e750 0x560b60640b70 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b605d73e0 0x560b605da510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f26e0ff9640 0 client.17781 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b605ccb40 0x560b605de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 --2- 172.21.15.44:0/2280025590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b605cc220 0x560b605dde00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 >> 172.21.15.44:0/2280025590 conn(0x560b605c8790 msgr2=0x560b605c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 shutdown_connections 2024-09-13T17:33:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.151+0000 7f2702280740 1 -- 172.21.15.44:0/2280025590 wait complete. 2024-09-13T17:33:50.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f2702280740 20 client.17781 trim_cache size 0 max 16384 2024-09-13T17:33:50.162 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:33:50.162 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:33:50.162 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:33:50.162 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26e40a1280 msgr2=0x7f26e40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26e40a1280 0x7f26e40a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26f004d000 tx=0x7f26f004e000 comp rx=0 tx=0).stop 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 msgr2=0x7f26f8196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f8196740 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c970 tx=0x7f26e800ce30 comp rx=0 tx=0).stop 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 shutdown_connections 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f26e40a1280 0x7f26e40a3740 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f819ab30 0x7f26f819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f8103fb0 0x7f26f8196740 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 --2- 172.21.15.44:0/1306967972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f80fee60 0x7f26f8196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 >> 172.21.15.44:0/1306967972 conn(0x7f26f8075c60 msgr2=0x7f26f8067740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 shutdown_connections 2024-09-13T17:33:50.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.152+0000 7f26ff55e640 1 -- 172.21.15.44:0/1306967972 wait complete. 2024-09-13T17:33:50.166 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:33:50.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:33:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 -- 172.21.15.44:0/3724583345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5100fff70 msgr2=0x7fb510100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 --2- 172.21.15.44:0/3724583345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5100fff70 0x7fb510100350 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb500009cd0 tx=0x7fb50002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 -- 172.21.15.44:0/3724583345 shutdown_connections 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 --2- 172.21.15.44:0/3724583345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb510100920 0x7fb510104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 --2- 172.21.15.44:0/3724583345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101050c0 0x7fb51010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 --2- 172.21.15.44:0/3724583345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5100fff70 0x7fb510100350 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 -- 172.21.15.44:0/3724583345 >> 172.21.15.44:0/3724583345 conn(0x7fb5100fba20 msgr2=0x7fb5100fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 -- 172.21.15.44:0/3724583345 shutdown_connections 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.267+0000 7fb515cd0640 1 -- 172.21.15.44:0/3724583345 wait complete. 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 Processor -- start 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 -- start start 2024-09-13T17:33:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5100fff70 0x7fb510198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 0x7fb5101989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb51019cdb0 0x7fb51019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51019d7a0 con 0x7fb5101050c0 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51019d910 con 0x7fb51019cdb0 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb515cd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51019da80 con 0x7fb5100fff70 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb50effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 0x7fb5101989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb50effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 0x7fb5101989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48366/0 (socket says 172.21.15.44:48366) 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb50effd640 1 -- 172.21.15.44:0/2473347852 learned_addr learned my addr 172.21.15.44:0/2473347852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb50ffff640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb51019cdb0 0x7fb51019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.268+0000 7fb50f7fe640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5100fff70 0x7fb510198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50effd640 1 -- 172.21.15.44:0/2473347852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5100fff70 msgr2=0x7fb510198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50effd640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5100fff70 0x7fb510198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50effd640 1 -- 172.21.15.44:0/2473347852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb51019cdb0 msgr2=0x7fb51019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50effd640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb51019cdb0 0x7fb51019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50effd640 1 -- 172.21.15.44:0/2473347852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500009900 con 0x7fb5101050c0 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50f7fe640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5100fff70 0x7fb510198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50effd640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 0x7fb5101989c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800d740 tx=0x7fb4f800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8009c30 con 0x7fb5101050c0 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4f800de50 con 0x7fb5101050c0 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8004d30 con 0x7fb5101050c0 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5101a5060 con 0x7fb5101050c0 2024-09-13T17:33:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.269+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5101a54c0 con 0x7fb5101050c0 2024-09-13T17:33:50.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.270+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb510101440 con 0x7fb5101050c0 2024-09-13T17:33:50.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.271+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4f8004e90 con 0x7fb5101050c0 2024-09-13T17:33:50.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.272+0000 7fb50cff9640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4fc0a1280 0x7fb4fc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.272+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb4f80cc9b0 con 0x7fb5101050c0 2024-09-13T17:33:50.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.272+0000 7fb50f7fe640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4fc0a1280 0x7fb4fc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.272+0000 7fb50f7fe640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4fc0a1280 0x7fb4fc0a3740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb500006010 tx=0x7fb5000093f0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.276+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb4f8095a90 con 0x7fb5101050c0 2024-09-13T17:33:50.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.485+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb510100350 con 0x7fb5101050c0 2024-09-13T17:33:50.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.486+0000 7fb50cff9640 1 -- 172.21.15.44:0/2473347852 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fb4f80990b0 con 0x7fb5101050c0 2024-09-13T17:33:50.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:33:50.488 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:33:50.488 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4fc0a1280 msgr2=0x7fb4fc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4fc0a1280 0x7fb4fc0a3740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb500006010 tx=0x7fb5000093f0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 msgr2=0x7fb5101989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 0x7fb5101989c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800d740 tx=0x7fb4f800dc00 comp rx=0 tx=0).stop 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 shutdown_connections 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4fc0a1280 0x7fb4fc0a3740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb51019cdb0 0x7fb51019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5101050c0 0x7fb5101989c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 --2- 172.21.15.44:0/2473347852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5100fff70 0x7fb510198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 >> 172.21.15.44:0/2473347852 conn(0x7fb5100fba20 msgr2=0x7fb5100fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 shutdown_connections 2024-09-13T17:33:50.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.488+0000 7fb515cd0640 1 -- 172.21.15.44:0/2473347852 wait complete. 2024-09-13T17:33:50.501 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:33:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 -- 172.21.15.44:0/1260557799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00fff90 msgr2=0x7f16a0100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 --2- 172.21.15.44:0/1260557799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00fff90 0x7f16a0100370 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1690009930 tx=0x7f169002a420 comp rx=0 tx=0).stop 2024-09-13T17:33:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 -- 172.21.15.44:0/1260557799 shutdown_connections 2024-09-13T17:33:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 --2- 172.21.15.44:0/1260557799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a01050e0 0x7f16a010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 --2- 172.21.15.44:0/1260557799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16a0100940 0x7f16a0104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 --2- 172.21.15.44:0/1260557799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00fff90 0x7f16a0100370 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 -- 172.21.15.44:0/1260557799 >> 172.21.15.44:0/1260557799 conn(0x7f16a00fba20 msgr2=0x7f16a00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 -- 172.21.15.44:0/1260557799 shutdown_connections 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 -- 172.21.15.44:0/1260557799 wait complete. 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 Processor -- start 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.604+0000 7f16a66e5640 1 -- start start 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a66e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16a00fff90 0x7f16a00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a66e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 0x7f16a0072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a66e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0073230 0x7f16a01a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a66e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01a4980 con 0x7f16a01050e0 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a66e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01a4af0 con 0x7f16a00fff90 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a66e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01a4cb0 con 0x7f16a0073230 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 0x7f16a0072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 0x7f16a0072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48382/0 (socket says 172.21.15.44:48382) 2024-09-13T17:33:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 -- 172.21.15.44:0/3418881957 learned_addr learned my addr 172.21.15.44:0/3418881957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f16a4c82640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0073230 0x7f16a01a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169ffff640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16a00fff90 0x7f16a00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 -- 172.21.15.44:0/3418881957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0073230 msgr2=0x7f16a01a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0073230 0x7f16a01a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 -- 172.21.15.44:0/3418881957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16a00fff90 msgr2=0x7f16a00727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16a00fff90 0x7f16a00727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 -- 172.21.15.44:0/3418881957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009560 con 0x7f16a01050e0 2024-09-13T17:33:50.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.605+0000 7f169f7fe640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 0x7f16a0072cf0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1694005fd0 tx=0x7f169400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.606+0000 7f169d7fa640 1 -- 172.21.15.44:0/3418881957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694009c30 con 0x7f16a01050e0 2024-09-13T17:33:50.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.606+0000 7f169d7fa640 1 -- 172.21.15.44:0/3418881957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16940044b0 con 0x7f16a01050e0 2024-09-13T17:33:50.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.606+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a01a4f90 con 0x7f16a01050e0 2024-09-13T17:33:50.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.606+0000 7f169d7fa640 1 -- 172.21.15.44:0/3418881957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694003da0 con 0x7f16a01050e0 2024-09-13T17:33:50.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.606+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a01a5410 con 0x7f16a01050e0 2024-09-13T17:33:50.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f169d7fa640 1 -- 172.21.15.44:0/3418881957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1694003f00 con 0x7f16a01050e0 2024-09-13T17:33:50.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 Processor -- start 2024-09-13T17:33:50.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 -- start start 2024-09-13T17:33:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 0x55869187eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x55869188bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586917e8cc0 con 0x55869187e790 2024-09-13T17:33:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586917e6450 con 0x55869187f0b0 2024-09-13T17:33:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a9403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586917e1d40 con 0x558691889950 2024-09-13T17:33:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a4c82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 0x55869187eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a4c82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 0x55869187eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48392/0 (socket says 172.21.15.44:48392) 2024-09-13T17:33:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.608+0000 7f16a4c82640 1 -- 172.21.15.44:0/1976506485 learned_addr learned my addr 172.21.15.44:0/1976506485 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f169d7fa640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f16880a1200 0x7f16880a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f169ffff640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f169f7fe640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x55869188bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 -- 172.21.15.44:0/1976506485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 msgr2=0x55869188bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x55869188bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 -- 172.21.15.44:0/1976506485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 msgr2=0x558691889410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 -- 172.21.15.44:0/1976506485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586917a9620 con 0x55869187e790 2024-09-13T17:33:50.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f169f7fe640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x55869188bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:50.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f169ffff640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f169d7fa640 1 -- 172.21.15.44:0/3418881957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f16940ccc00 con 0x7f16a01050e0 2024-09-13T17:33:50.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f16880a1200 0x7f16880a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16a4c82640 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 0x55869187eb70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1698010440 tx=0x7f1698010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f39a3b085711ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16867fc640 1 -- 172.21.15.44:0/1976506485 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698022070 con 0x55869187e790 2024-09-13T17:33:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16867fc640 1 -- 172.21.15.44:0/1976506485 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169801d890 con 0x55869187e790 2024-09-13T17:33:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.609+0000 7f16867fc640 1 -- 172.21.15.44:0/1976506485 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169801c610 con 0x55869187e790 2024-09-13T17:33:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 -- 172.21.15.44:0/1976506485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 msgr2=0x55869187eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 0x55869187eb70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1698010440 tx=0x7f1698010900 comp rx=0 tx=0).stop 2024-09-13T17:33:50.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a4c82640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f16880a1200 0x7f16880a36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1698021000 tx=0x7f169801e000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 -- 172.21.15.44:0/1976506485 shutdown_connections 2024-09-13T17:33:50.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x55869188bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 --2- 172.21.15.44:0/1976506485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187e790 0x55869187eb70 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 -- 172.21.15.44:0/1976506485 >> 172.21.15.44:0/1976506485 conn(0x55869187ad00 msgr2=0x55869187b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 -- 172.21.15.44:0/1976506485 shutdown_connections 2024-09-13T17:33:50.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 -- 172.21.15.44:0/1976506485 wait complete. 2024-09-13T17:33:50.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.610+0000 7f16a9403740 1 -- start start 2024-09-13T17:33:50.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 Processor -- start 2024-09-13T17:33:50.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x558691888e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x5586918893a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918863d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586917a0180 con 0x55869187f0b0 2024-09-13T17:33:50.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586914ba430 con 0x55869187e790 2024-09-13T17:33:50.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586917655e0 con 0x558691889950 2024-09-13T17:33:50.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a9403740 10 client.0 _ll_register_callbacks cb 0x5586917d1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x5586918893a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x5586918893a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48400/0 (socket says 172.21.15.44:48400) 2024-09-13T17:33:50.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 -- 172.21.15.44:0/1901308316 learned_addr learned my addr 172.21.15.44:0/1901308316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169ffff640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x558691888e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f16a4c82640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918863d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 -- 172.21.15.44:0/1901308316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 msgr2=0x5586918863d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918863d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 -- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 msgr2=0x558691888e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x558691888e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 -- 172.21.15.44:0/1901308316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1698009c20 con 0x55869187f0b0 2024-09-13T17:33:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f169f7fe640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x5586918893a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f16940ca5e0 tx=0x7f1694068c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb61bd43ce39118 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.611+0000 7f167f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f167f7fe640 1 -- 172.21.15.44:0/1901308316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169409d7d0 con 0x55869187f0b0 2024-09-13T17:33:50.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f167f7fe640 1 -- 172.21.15.44:0/1901308316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f16940a1e20 con 0x55869187f0b0 2024-09-13T17:33:50.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55869180bb80 con 0x55869187f0b0 2024-09-13T17:33:50.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f167f7fe640 1 -- 172.21.15.44:0/1901308316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16940041b0 con 0x55869187f0b0 2024-09-13T17:33:50.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 10 client.17799 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5586917e03c0 con 0x55869187f0b0 2024-09-13T17:33:50.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f167f7fe640 1 -- 172.21.15.44:0/1901308316 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55869180bb80 con 0x55869187f0b0 2024-09-13T17:33:50.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f167f7fe640 1 -- 172.21.15.44:0/1901308316 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1694022020 con 0x55869187f0b0 2024-09-13T17:33:50.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 10 client.17799 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 10 client.17799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:33:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 20 client.17799 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x558691899050 0x55869189b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 4 client.17799 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586917d41f0 con 0x558691899050 2024-09-13T17:33:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.612+0000 7f169ffff640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x558691899050 0x55869189b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.613+0000 7f169ffff640 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x558691899050 0x55869189b440 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd0de216c0b7eec3 server_cookie=a2c4b5c735e414ac in_seq=0 out_seq=0 2024-09-13T17:33:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.613+0000 7f167f7fe640 10 client.17799 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.613+0000 7f167f7fe640 1 -- 172.21.15.44:0/1901308316 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5586917d41f0 con 0x558691899050 2024-09-13T17:33:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.613+0000 7f167f7fe640 10 client.17799 handle_command_reply: tid=0 2024-09-13T17:33:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.613+0000 7f16a9403740 1 client.17799 shutdown 2024-09-13T17:33:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 msgr2=0x5586918893a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x5586918893a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f16940ca5e0 tx=0x7f1694068c60 comp rx=0 tx=0).stop 2024-09-13T17:33:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 shutdown_connections 2024-09-13T17:33:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x558691899050 0x55869189b440 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918863d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x5586918893a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f167f7fe640 0 client.17799 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 --2- 172.21.15.44:0/1901308316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x558691888e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 >> 172.21.15.44:0/1901308316 conn(0x55869187ad00 msgr2=0x558691880a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 shutdown_connections 2024-09-13T17:33:50.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 1 -- 172.21.15.44:0/1901308316 wait complete. 2024-09-13T17:33:50.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.614+0000 7f16a9403740 20 client.17799 trim_cache size 0 max 16384 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 Processor -- start 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 -- start start 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55869187e790 0x5586918950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 0x558691890620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586917d41f0 con 0x558691889950 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869180bb80 con 0x55869187f0b0 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a9403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558691887610 con 0x55869187e790 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a4c82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 0x558691890620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a4c82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 0x558691890620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48406/0 (socket says 172.21.15.44:48406) 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f16a4c82640 1 -- 172.21.15.44:0/889141540 learned_addr learned my addr 172.21.15.44:0/889141540 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f169ffff640 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55869187e790 0x5586918950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.625+0000 7f169f7fe640 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a4c82640 1 -- 172.21.15.44:0/889141540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55869187e790 msgr2=0x5586918950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a4c82640 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55869187e790 0x5586918950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a4c82640 1 -- 172.21.15.44:0/889141540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 msgr2=0x558691889160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a4c82640 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a4c82640 1 -- 172.21.15.44:0/889141540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558691782770 con 0x558691889950 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f169f7fe640 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a4c82640 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 0x558691890620 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1698010e00 tx=0x7f1698028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a4ea191b772f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f1686ffd640 1 -- 172.21.15.44:0/889141540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698022030 con 0x558691889950 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f1686ffd640 1 -- 172.21.15.44:0/889141540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1698023040 con 0x558691889950 2024-09-13T17:33:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f1686ffd640 1 -- 172.21.15.44:0/889141540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698003bb0 con 0x558691889950 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a9403740 1 -- 172.21.15.44:0/889141540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 msgr2=0x558691890620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a9403740 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 0x558691890620 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1698010e00 tx=0x7f1698028000 comp rx=0 tx=0).stop 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a9403740 1 -- 172.21.15.44:0/889141540 shutdown_connections 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a9403740 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558691889950 0x558691890620 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a9403740 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187f0b0 0x558691889160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.626+0000 7f16a9403740 1 --2- 172.21.15.44:0/889141540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55869187e790 0x5586918950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- 172.21.15.44:0/889141540 >> 172.21.15.44:0/889141540 conn(0x55869187ad00 msgr2=0x558691888a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- 172.21.15.44:0/889141540 shutdown_connections 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- 172.21.15.44:0/889141540 wait complete. 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- start start 2024-09-13T17:33:50.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 Processor -- start 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x55869188f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x55869188fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558691890330 con 0x55869187f0b0 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586918efab0 con 0x55869187e790 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586918efc20 con 0x558691889950 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f16a9403740 10 client.0 _ll_register_callbacks cb 0x7f1698009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x55869188fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x55869188fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48422/0 (socket says 172.21.15.44:48422) 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.627+0000 7f169f7fe640 1 -- 172.21.15.44:0/2786311462 learned_addr learned my addr 172.21.15.44:0/2786311462 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a4c82640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169f7fe640 1 -- 172.21.15.44:0/2786311462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 msgr2=0x5586918ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169f7fe640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169f7fe640 1 -- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 msgr2=0x55869188f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169ffff640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x55869188f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169f7fe640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x55869188f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169f7fe640 1 -- 172.21.15.44:0/2786311462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586917a9620 con 0x55869187f0b0 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f169f7fe640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x55869188fc10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f16940a18f0 tx=0x7f1694005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a6b08e36895740 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f167f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f167f7fe640 1 -- 172.21.15.44:0/2786311462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169409a330 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f167f7fe640 1 -- 172.21.15.44:0/2786311462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f16940a1990 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f169801d890 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f167f7fe640 1 -- 172.21.15.44:0/2786311462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1694099770 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5586917a7e80 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f167f7fe640 1 -- 172.21.15.44:0/2786311462 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169801d890 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f167f7fe640 1 -- 172.21.15.44:0/2786311462 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1694022040 con 0x55869187f0b0 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 20 client.17805 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5586918f0eb0 0x5586918f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 4 client.17805 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:33:50.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.628+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5586914d54e0 con 0x5586918f0eb0 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.629+0000 7f169ffff640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5586918f0eb0 0x5586918f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.629+0000 7f169ffff640 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5586918f0eb0 0x5586918f32a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f6acc565b700871 server_cookie=31fc967334010cd0 in_seq=0 out_seq=0 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.629+0000 7f167f7fe640 10 client.17805 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.629+0000 7f167f7fe640 1 -- 172.21.15.44:0/2786311462 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5586914d54e0 con 0x5586918f0eb0 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.629+0000 7f167f7fe640 10 client.17805 handle_command_reply: tid=0 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.629+0000 7f16a9403740 1 client.17805 shutdown 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 msgr2=0x55869188fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x55869188fc10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f16940a18f0 tx=0x7f1694005ad0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 shutdown_connections 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5586918f0eb0 0x5586918f32a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558691889950 0x5586918ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55869187f0b0 0x55869188fc10 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f167f7fe640 0 client.17805 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:33:50.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 --2- 172.21.15.44:0/2786311462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55869187e790 0x55869188f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 >> 172.21.15.44:0/2786311462 conn(0x55869187ad00 msgr2=0x5586918891f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 shutdown_connections 2024-09-13T17:33:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 1 -- 172.21.15.44:0/2786311462 wait complete. 2024-09-13T17:33:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a9403740 20 client.17805 trim_cache size 0 max 16384 2024-09-13T17:33:50.662 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:33:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f16880a1200 msgr2=0x7f16880a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f16880a1200 0x7f16880a36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1698021000 tx=0x7f169801e000 comp rx=0 tx=0).stop 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 msgr2=0x7f16a0072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 0x7f16a0072cf0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1694005fd0 tx=0x7f169400dde0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 shutdown_connections 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f16880a1200 0x7f16880a36c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0073230 0x7f16a01a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a01050e0 0x7f16a0072cf0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 --2- 172.21.15.44:0/3418881957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16a00fff90 0x7f16a00727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 >> 172.21.15.44:0/3418881957 conn(0x7f16a00fba20 msgr2=0x7f16a00fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.630+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 shutdown_connections 2024-09-13T17:33:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.631+0000 7f16a66e5640 1 -- 172.21.15.44:0/3418881957 wait complete. 2024-09-13T17:33:50.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:33:50.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 -- 172.21.15.44:0/1319473655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36080ff830 msgr2=0x7f3608103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 --2- 172.21.15.44:0/1319473655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36080ff830 0x7f3608103870 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009930 tx=0x7f35fc02a420 comp rx=0 tx=0).stop 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 -- 172.21.15.44:0/1319473655 shutdown_connections 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 --2- 172.21.15.44:0/1319473655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608103fd0 0x7f360810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 --2- 172.21.15.44:0/1319473655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36080ff830 0x7f3608103870 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 --2- 172.21.15.44:0/1319473655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36080fee80 0x7f36080ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 -- 172.21.15.44:0/1319473655 >> 172.21.15.44:0/1319473655 conn(0x7f36080fa930 msgr2=0x7f36080fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 -- 172.21.15.44:0/1319473655 shutdown_connections 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 -- 172.21.15.44:0/1319473655 wait complete. 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.767+0000 7f360d4c9640 1 Processor -- start 2024-09-13T17:33:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 -- start start 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36080fee80 0x7f36081983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 0x7f36081988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360819cce0 0x7f360819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360819d6d0 con 0x7f3608103fd0 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360819d840 con 0x7f360819cce0 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f360d4c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360819d9b0 con 0x7f36080fee80 2024-09-13T17:33:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 0x7f36081988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 0x7f36081988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48442/0 (socket says 172.21.15.44:48442) 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 -- 172.21.15.44:0/3084015236 learned_addr learned my addr 172.21.15.44:0/3084015236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f3606ffd640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36080fee80 0x7f36081983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36077fe640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360819cce0 0x7f360819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 -- 172.21.15.44:0/3084015236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36080fee80 msgr2=0x7f36081983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36080fee80 0x7f36081983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 -- 172.21.15.44:0/3084015236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360819cce0 msgr2=0x7f360819d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360819cce0 0x7f360819d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.768+0000 7f36067fc640 1 -- 172.21.15.44:0/3084015236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc009560 con 0x7f3608103fd0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.769+0000 7f36067fc640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 0x7f36081988f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f35fc005ab0 tx=0x7f35fc0329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.769+0000 7f35effff640 1 -- 172.21.15.44:0/3084015236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc032df0 con 0x7f3608103fd0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.769+0000 7f35effff640 1 -- 172.21.15.44:0/3084015236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35fc005490 con 0x7f3608103fd0 2024-09-13T17:33:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.769+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36081a4f90 con 0x7f3608103fd0 2024-09-13T17:33:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.769+0000 7f35effff640 1 -- 172.21.15.44:0/3084015236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0035f0 con 0x7f3608103fd0 2024-09-13T17:33:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.769+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36081a5300 con 0x7f3608103fd0 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.771+0000 7f35effff640 1 -- 172.21.15.44:0/3084015236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f35fc0047c0 con 0x7f3608103fd0 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.771+0000 7f36101e6740 1 Processor -- start 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.771+0000 7f36101e6740 1 -- start start 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36101e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5306b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36101e6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36101e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f5313d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36101e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5270cc0 con 0x55e4f5311950 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36101e6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f526e450 con 0x55e4f53070b0 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36101e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5269d40 con 0x55e4f5306790 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f5313d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5306b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5306b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:49476/0 (socket says 172.21.15.44:49476) 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 -- 172.21.15.44:0/150083071 learned_addr learned my addr 172.21.15.44:0/150083071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f3606ffd640 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f35effff640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 -- 172.21.15.44:0/150083071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 msgr2=0x55e4f5311410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 -- 172.21.15.44:0/150083071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 msgr2=0x55e4f5313d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f5313d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 -- 172.21.15.44:0/150083071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f5231620 con 0x55e4f5306790 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36067fc640 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f5313d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f3606ffd640 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f35effff640 1 -- 172.21.15.44:0/3084015236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f35fc0edb40 con 0x7f3608103fd0 2024-09-13T17:33:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.772+0000 7f36077fe640 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5306b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3600009f50 tx=0x7f36000118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abaab7fb7b819e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f35ed7fa640 1 -- 172.21.15.44:0/150083071 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600020030 con 0x55e4f5306790 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f35ed7fa640 1 -- 172.21.15.44:0/150083071 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f360000fe30 con 0x55e4f5306790 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f35ed7fa640 1 -- 172.21.15.44:0/150083071 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360001b850 con 0x55e4f5306790 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 -- 172.21.15.44:0/150083071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 msgr2=0x55e4f5306b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36077fe640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3600012fd0 tx=0x7f3600021000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5306b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3600009f50 tx=0x7f36000118d0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 -- 172.21.15.44:0/150083071 shutdown_connections 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f5313d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 --2- 172.21.15.44:0/150083071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5306b70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 -- 172.21.15.44:0/150083071 >> 172.21.15.44:0/150083071 conn(0x55e4f5302d00 msgr2=0x55e4f53030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.773+0000 7f36101e6740 1 -- 172.21.15.44:0/150083071 shutdown_connections 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 -- 172.21.15.44:0/150083071 wait complete. 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 -- start start 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 Processor -- start 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5310ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f530e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5228180 con 0x55e4f5311950 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f4f42430 con 0x55e4f53070b0 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f51ed5e0 con 0x55e4f5306790 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36101e6740 10 client.0 _ll_register_callbacks cb 0x55e4f5259aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f530e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f530e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48460/0 (socket says 172.21.15.44:48460) 2024-09-13T17:33:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36077fe640 1 -- 172.21.15.44:0/3142562558 learned_addr learned my addr 172.21.15.44:0/3142562558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f3606ffd640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5310ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.774+0000 7f36067fc640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36077fe640 1 -- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 msgr2=0x55e4f5310ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36077fe640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5310ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36077fe640 1 -- 172.21.15.44:0/3142562558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 msgr2=0x55e4f5311410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36077fe640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36077fe640 1 -- 172.21.15.44:0/3142562558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3600009c00 con 0x55e4f5311950 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36077fe640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f530e510 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3600004af0 tx=0x7f3600004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a2dc40b7cc69d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f35eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f35eaff5640 1 -- 172.21.15.44:0/3142562558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600004110 con 0x55e4f5311950 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f35eaff5640 1 -- 172.21.15.44:0/3142562558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f360002a070 con 0x55e4f5311950 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4f5293b80 con 0x55e4f5311950 2024-09-13T17:33:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f35eaff5640 1 -- 172.21.15.44:0/3142562558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f360000f920 con 0x55e4f5311950 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36101e6740 10 client.17817 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4f52683c0 con 0x55e4f5311950 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.775+0000 7f35eaff5640 1 -- 172.21.15.44:0/3142562558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e4f5293b80 con 0x55e4f5311950 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f35eaff5640 1 -- 172.21.15.44:0/3142562558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f360002fbb0 con 0x55e4f5311950 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f36101e6740 10 client.17817 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f36101e6740 10 client.17817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f36101e6740 20 client.17817 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f36101e6740 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5321240 0x55e4f5323630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f36101e6740 4 client.17817 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4f525c1f0 con 0x55e4f5321240 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f3606ffd640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5321240 0x55e4f5323630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f35eaff5640 10 client.17817 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.776+0000 7f3606ffd640 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5321240 0x55e4f5323630 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2708557d0c285878 server_cookie=431f33335e393130 in_seq=0 out_seq=0 2024-09-13T17:33:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f35eaff5640 1 -- 172.21.15.44:0/3142562558 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e4f525c1f0 con 0x55e4f5321240 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f35eaff5640 10 client.17817 handle_command_reply: tid=0 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 client.17817 shutdown 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 msgr2=0x55e4f530e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f530e510 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3600004af0 tx=0x7f3600004760 comp rx=0 tx=0).stop 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 shutdown_connections 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5321240 0x55e4f5323630 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5311950 0x55e4f530e510 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5311410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f35eaff5640 0 client.17817 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 --2- 172.21.15.44:0/3142562558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5306790 0x55e4f5310ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 >> 172.21.15.44:0/3142562558 conn(0x55e4f5302d00 msgr2=0x55e4f5308a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 shutdown_connections 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 1 -- 172.21.15.44:0/3142562558 wait complete. 2024-09-13T17:33:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.777+0000 7f36101e6740 20 client.17817 trim_cache size 0 max 16384 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 Processor -- start 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 -- start start 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f53030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5303620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5311950 0x55e4f5310cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5321070 con 0x55e4f5306790 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5303e90 con 0x55e4f53070b0 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f36101e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f530f750 con 0x55e4f5311950 2024-09-13T17:33:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f3606ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f53030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f3606ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f53030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48476/0 (socket says 172.21.15.44:48476) 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.788+0000 7f3606ffd640 1 -- 172.21.15.44:0/1908730343 learned_addr learned my addr 172.21.15.44:0/1908730343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f36077fe640 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5311950 0x55e4f5310cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f3606ffd640 1 -- 172.21.15.44:0/1908730343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5311950 msgr2=0x55e4f5310cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f3606ffd640 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5311950 0x55e4f5310cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f3606ffd640 1 -- 172.21.15.44:0/1908730343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 msgr2=0x55e4f5303620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f3606ffd640 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5303620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f3606ffd640 1 -- 172.21.15.44:0/1908730343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f520a770 con 0x55e4f5306790 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f3606ffd640 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f53030e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x55e4f5323ee0 tx=0x7f35f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7bb262720dbdb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f35edffb640 1 -- 172.21.15.44:0/1908730343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8017030 con 0x55e4f5306790 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f35edffb640 1 -- 172.21.15.44:0/1908730343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35f8018040 con 0x55e4f5306790 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.789+0000 7f35edffb640 1 -- 172.21.15.44:0/1908730343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f801f2d0 con 0x55e4f5306790 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- 172.21.15.44:0/1908730343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 msgr2=0x55e4f53030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f53030e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x55e4f5323ee0 tx=0x7f35f8005f90 comp rx=0 tx=0).stop 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- 172.21.15.44:0/1908730343 shutdown_connections 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f5311950 0x55e4f5310cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f53070b0 0x55e4f5303620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- 172.21.15.44:0/1908730343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f53030e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- 172.21.15.44:0/1908730343 >> 172.21.15.44:0/1908730343 conn(0x55e4f5302d00 msgr2=0x55e4f5327f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- 172.21.15.44:0/1908730343 shutdown_connections 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- 172.21.15.44:0/1908730343 wait complete. 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- start start 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 Processor -- start 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f5315390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f53070b0 0x55e4f53158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f5311950 0x55e4f5315e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5377850 con 0x55e4f5306790 2024-09-13T17:33:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f53779c0 con 0x55e4f5311950 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f5377b80 con 0x55e4f53070b0 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36101e6740 10 client.0 _ll_register_callbacks cb 0x7f3600009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f3606ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f5315390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f3606ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f5315390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48492/0 (socket says 172.21.15.44:48492) 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f3606ffd640 1 -- 172.21.15.44:0/2207451438 learned_addr learned my addr 172.21.15.44:0/2207451438 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.790+0000 7f36067fc640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f53070b0 0x55e4f53158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f36077fe640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f5311950 0x55e4f5315e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f3606ffd640 1 -- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f53070b0 msgr2=0x55e4f53158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f3606ffd640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f53070b0 0x55e4f53158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f3606ffd640 1 -- 172.21.15.44:0/2207451438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f5311950 msgr2=0x55e4f5315e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f3606ffd640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f5311950 0x55e4f5315e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f3606ffd640 1 -- 172.21.15.44:0/2207451438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f8009db0 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f36077fe640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f5311950 0x55e4f5315e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f3606ffd640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f5315390 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005780 tx=0x7f35f80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ffb7fc6511c7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f35eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f35eaff5640 1 -- 172.21.15.44:0/2207451438 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8017030 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f35eaff5640 1 -- 172.21.15.44:0/2207451438 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f35f80037f0 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f360000fe30 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f35eaff5640 1 -- 172.21.15.44:0/2207451438 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f80044e0 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f36101e6740 10 client.17823 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4f522fe80 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.791+0000 7f35eaff5640 1 -- 172.21.15.44:0/2207451438 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f360000fe30 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f35eaff5640 1 -- 172.21.15.44:0/2207451438 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35f8026070 con 0x55e4f5306790 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 10 client.17823 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 10 client.17823 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:33:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 20 client.17823 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5379140 0x55e4f537b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 4 client.17823 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:33:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4f4f5d4e0 con 0x55e4f5379140 2024-09-13T17:33:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36067fc640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5379140 0x55e4f537b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f35eaff5640 10 client.17823 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36067fc640 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5379140 0x55e4f537b530 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4286c055af60a40d server_cookie=c974bb8d77bafe89 in_seq=0 out_seq=0 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f35eaff5640 1 -- 172.21.15.44:0/2207451438 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4f4f5d4e0 con 0x55e4f5379140 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f35eaff5640 10 client.17823 handle_command_reply: tid=0 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 1 client.17823 shutdown 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 msgr2=0x55e4f5315390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.792+0000 7f36101e6740 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f5315390 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005780 tx=0x7f35f80040e0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 shutdown_connections 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e4f5379140 0x55e4f537b530 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4f53070b0 0x55e4f53158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f35eaff5640 0 client.17823 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e4f5311950 0x55e4f5315e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 --2- 172.21.15.44:0/2207451438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4f5306790 0x55e4f5315390 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 >> 172.21.15.44:0/2207451438 conn(0x55e4f5302d00 msgr2=0x55e4f53030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 shutdown_connections 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 1 -- 172.21.15.44:0/2207451438 wait complete. 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f36101e6740 20 client.17823 trim_cache size 0 max 16384 2024-09-13T17:33:50.803 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:33:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 msgr2=0x7f35f00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3600012fd0 tx=0x7f3600021000 comp rx=0 tx=0).stop 2024-09-13T17:33:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 msgr2=0x7f36081988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 0x7f36081988f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f35fc005ab0 tx=0x7f35fc0329f0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 shutdown_connections 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360819cce0 0x7f360819d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3608103fd0 0x7f36081988f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 --2- 172.21.15.44:0/3084015236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36080fee80 0x7f36081983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 >> 172.21.15.44:0/3084015236 conn(0x7f36080fa930 msgr2=0x7f3608068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 shutdown_connections 2024-09-13T17:33:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.793+0000 7f360d4c9640 1 -- 172.21.15.44:0/3084015236 wait complete. 2024-09-13T17:33:50.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2473347852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:33:50.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:33:50.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 -- 172.21.15.44:0/796837121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 msgr2=0x7f6b800ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 --2- 172.21.15.44:0/796837121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b800ff260 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009930 tx=0x7f6b7002a420 comp rx=0 tx=0).stop 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 -- 172.21.15.44:0/796837121 shutdown_connections 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 --2- 172.21.15.44:0/796837121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b800ff830 0x7f6b80103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 --2- 172.21.15.44:0/796837121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80103fd0 0x7f6b8010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 --2- 172.21.15.44:0/796837121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b800ff260 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 -- 172.21.15.44:0/796837121 >> 172.21.15.44:0/796837121 conn(0x7f6b800fa930 msgr2=0x7f6b800fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 -- 172.21.15.44:0/796837121 shutdown_connections 2024-09-13T17:33:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.909+0000 7f6b848b8640 1 -- 172.21.15.44:0/796837121 wait complete. 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 Processor -- start 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 -- start start 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b80196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80103fd0 0x7f6b80197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b80197a30 0x7f6b801a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b801a2700 con 0x7f6b800fee80 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b801a2870 con 0x7f6b80103fd0 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b848b8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b801a2a30 con 0x7f6b80197a30 2024-09-13T17:33:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b80196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b80196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48516/0 (socket says 172.21.15.44:48516) 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 -- 172.21.15.44:0/560343824 learned_addr learned my addr 172.21.15.44:0/560343824 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80103fd0 0x7f6b80197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b80197a30 0x7f6b801a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 -- 172.21.15.44:0/560343824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b80197a30 msgr2=0x7f6b801a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b80197a30 0x7f6b801a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 -- 172.21.15.44:0/560343824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80103fd0 msgr2=0x7f6b80197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80103fd0 0x7f6b80197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.910+0000 7f6b7e59c640 1 -- 172.21.15.44:0/560343824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b70009560 con 0x7f6b800fee80 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.911+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b80196f10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005ab0 tx=0x7f6b70032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.911+0000 7f6b6f7fe640 1 -- 172.21.15.44:0/560343824 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70032e10 con 0x7f6b800fee80 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.911+0000 7f6b6f7fe640 1 -- 172.21.15.44:0/560343824 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b700046c0 con 0x7f6b800fee80 2024-09-13T17:33:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.911+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b801a2cb0 con 0x7f6b800fee80 2024-09-13T17:33:50.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.911+0000 7f6b6f7fe640 1 -- 172.21.15.44:0/560343824 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003850 con 0x7f6b800fee80 2024-09-13T17:33:50.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.911+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b801a3160 con 0x7f6b800fee80 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.913+0000 7f6b6f7fe640 1 -- 172.21.15.44:0/560343824 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6b70004830 con 0x7f6b800fee80 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.913+0000 7f6b875da740 1 Processor -- start 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 -- start start 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f1bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f26410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848e85cc0 con 0x55c848f1c0b0 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848e83450 con 0x55c848f26950 2024-09-13T17:33:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b875da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848e7ed40 con 0x55c848f1b790 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f26410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f26410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48522/0 (socket says 172.21.15.44:48522) 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b7e59c640 1 -- 172.21.15.44:0/1309115161 learned_addr learned my addr 172.21.15.44:0/1309115161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f1bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b6f7fe640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6b680a1200 0x7f6b680a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.914+0000 7f6b7e59c640 1 -- 172.21.15.44:0/1309115161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 msgr2=0x55c848f1bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f1bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7e59c640 1 -- 172.21.15.44:0/1309115161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 msgr2=0x55c848f28d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f28d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7e59c640 1 -- 172.21.15.44:0/1309115161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c848e46620 con 0x55c848f1c0b0 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6b680a1200 0x7f6b680a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b6f7fe640 1 -- 172.21.15.44:0/560343824 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6b700edd10 con 0x7f6b800fee80 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f26410 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6b700ba750 tx=0x7f6b70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f6209b77d5b80c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b6cff9640 1 -- 172.21.15.44:0/1309115161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700ba590 con 0x55c848f1c0b0 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6b680a1200 0x7f6b680a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6b780114a0 tx=0x7f6b78004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b6cff9640 1 -- 172.21.15.44:0/1309115161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b700ba3b0 con 0x55c848f1c0b0 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.915+0000 7f6b6cff9640 1 -- 172.21.15.44:0/1309115161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700be880 con 0x55c848f1c0b0 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- 172.21.15.44:0/1309115161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 msgr2=0x55c848f26410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f26410 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6b700ba750 tx=0x7f6b70009260 comp rx=0 tx=0).stop 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- 172.21.15.44:0/1309115161 shutdown_connections 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f28d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f26410 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- 172.21.15.44:0/1309115161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- 172.21.15.44:0/1309115161 >> 172.21.15.44:0/1309115161 conn(0x55c848f17d00 msgr2=0x55c848f180e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- 172.21.15.44:0/1309115161 shutdown_connections 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- 172.21.15.44:0/1309115161 wait complete. 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- start start 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 Processor -- start 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f25f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f23290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848e3d180 con 0x55c848f1c0b0 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848b57430 con 0x55c848f26950 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.916+0000 7f6b875da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848e025e0 con 0x55c848f1b790 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b875da740 10 client.0 _ll_register_callbacks cb 0x55c848e6eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f23290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f23290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48524/0 (socket says 172.21.15.44:48524) 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 -- 172.21.15.44:0/1535069983 learned_addr learned my addr 172.21.15.44:0/1535069983 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f25f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 -- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 msgr2=0x55c848f25f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f25f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 -- 172.21.15.44:0/1535069983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 msgr2=0x55c848f237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 -- 172.21.15.44:0/1535069983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b801a3160 con 0x55c848f1c0b0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f23290 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005fd0 tx=0x7f6b74011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4f338e0c47c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b65ffb640 1 -- 172.21.15.44:0/1535069983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74016070 con 0x55c848f1c0b0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b65ffb640 1 -- 172.21.15.44:0/1535069983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6b74004020 con 0x55c848f1c0b0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c848ea8b80 con 0x55c848f1c0b0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.917+0000 7f6b65ffb640 1 -- 172.21.15.44:0/1535069983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b740041e0 con 0x55c848f1c0b0 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 10 client.17835 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c848e7d3c0 con 0x55c848f1c0b0 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b65ffb640 1 -- 172.21.15.44:0/1535069983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c848ea8b80 con 0x55c848f1c0b0 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b65ffb640 1 -- 172.21.15.44:0/1535069983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b74003680 con 0x55c848f1c0b0 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 10 client.17835 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 20 client.17835 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f36060 0x55c848f38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 4 client.17835 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c848e711f0 con 0x55c848f36060 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.918+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f36060 0x55c848f38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.919+0000 7f6b65ffb640 10 client.17835 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.919+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f36060 0x55c848f38480 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffefdefbba754182 server_cookie=f77b1a338801bd60 in_seq=0 out_seq=0 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.919+0000 7f6b65ffb640 1 -- 172.21.15.44:0/1535069983 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c848e711f0 con 0x55c848f36060 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.919+0000 7f6b65ffb640 10 client.17835 handle_command_reply: tid=0 2024-09-13T17:33:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.919+0000 7f6b875da740 1 client.17835 shutdown 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 msgr2=0x55c848f23290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f23290 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005fd0 tx=0x7f6b74011810 comp rx=0 tx=0).stop 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 shutdown_connections 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f36060 0x55c848f38480 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f26950 0x55c848f237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f1c0b0 0x55c848f23290 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b65ffb640 0 client.17835 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 --2- 172.21.15.44:0/1535069983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 >> 172.21.15.44:0/1535069983 conn(0x55c848f17d00 msgr2=0x55c848f1da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 shutdown_connections 2024-09-13T17:33:50.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 1 -- 172.21.15.44:0/1535069983 wait complete. 2024-09-13T17:33:50.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.920+0000 7f6b875da740 20 client.17835 trim_cache size 0 max 16384 2024-09-13T17:33:50.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 Processor -- start 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 -- start start 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1c0b0 0x55c848f2d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848e711f0 con 0x55c848f26950 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848ea8b80 con 0x55c848f1c0b0 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b875da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848f1c930 con 0x55c848f1b790 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48540/0 (socket says 172.21.15.44:48540) 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3785067380 learned_addr learned my addr 172.21.15.44:0/3785067380 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1c0b0 0x55c848f2d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3785067380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 msgr2=0x55c848f176a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f176a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3785067380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1c0b0 msgr2=0x55c848f2d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1c0b0 0x55c848f2d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3785067380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c848e1f770 con 0x55c848f26950 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1c0b0 0x55c848f2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.931+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2d940 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6b78012890 tx=0x7f6b78014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153a0325526170c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b6d7fa640 1 -- 172.21.15.44:0/3785067380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7801fdf0 con 0x55c848f26950 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b6d7fa640 1 -- 172.21.15.44:0/3785067380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b78012e10 con 0x55c848f26950 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b6d7fa640 1 -- 172.21.15.44:0/3785067380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780286a0 con 0x55c848f26950 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 -- 172.21.15.44:0/3785067380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 msgr2=0x55c848f2d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2d940 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6b78012890 tx=0x7f6b78014000 comp rx=0 tx=0).stop 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 -- 172.21.15.44:0/3785067380 shutdown_connections 2024-09-13T17:33:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2d940 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1c0b0 0x55c848f2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 --2- 172.21.15.44:0/3785067380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1b790 0x55c848f176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 -- 172.21.15.44:0/3785067380 >> 172.21.15.44:0/3785067380 conn(0x55c848f17d00 msgr2=0x55c848f3ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 -- 172.21.15.44:0/3785067380 shutdown_connections 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 -- 172.21.15.44:0/3785067380 wait complete. 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.932+0000 7f6b875da740 1 -- start start 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 Processor -- start 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1b790 0x55c848f2d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1c0b0 0x55c848f29b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848f2a760 con 0x55c848f26950 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848f2a8d0 con 0x55c848f1b790 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c848f8c5a0 con 0x55c848f1c0b0 2024-09-13T17:33:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b875da740 10 client.0 _ll_register_callbacks cb 0x7f6b700f1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48546/0 (socket says 172.21.15.44:48546) 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3997999490 learned_addr learned my addr 172.21.15.44:0/3997999490 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1b790 0x55c848f2d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7dd9b640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1c0b0 0x55c848f29b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1c0b0 msgr2=0x55c848f29b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1c0b0 0x55c848f29b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3997999490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1b790 msgr2=0x55c848f2d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1b790 0x55c848f2d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 -- 172.21.15.44:0/3997999490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78012570 con 0x55c848f26950 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b7ed9d640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2a060 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6b7801bbf0 tx=0x7f6b78004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d126b4194368906 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.933+0000 7f6b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b65ffb640 1 -- 172.21.15.44:0/3997999490 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7802f070 con 0x55c848f26950 2024-09-13T17:33:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b700ba3b0 con 0x55c848f26950 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b65ffb640 1 -- 172.21.15.44:0/3997999490 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6b7801b660 con 0x55c848f26950 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b65ffb640 1 -- 172.21.15.44:0/3997999490 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b7801be90 con 0x55c848f26950 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 10 client.17841 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c848e44e80 con 0x55c848f26950 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b65ffb640 1 -- 172.21.15.44:0/3997999490 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b700ba3b0 con 0x55c848f26950 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b65ffb640 1 -- 172.21.15.44:0/3997999490 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b78005830 con 0x55c848f26950 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 10 client.17841 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 20 client.17841 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f8d9b0 0x55c848f8fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 4 client.17841 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c848b724e0 con 0x55c848f8d9b0 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.934+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f8d9b0 0x55c848f8fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b65ffb640 10 client.17841 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b7e59c640 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f8d9b0 0x55c848f8fdd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5360ada512cefcd1 server_cookie=190495ce13fc12dc in_seq=0 out_seq=0 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b65ffb640 1 -- 172.21.15.44:0/3997999490 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c848b724e0 con 0x55c848f8d9b0 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b65ffb640 10 client.17841 handle_command_reply: tid=0 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 client.17841 shutdown 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 msgr2=0x55c848f2a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2a060 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6b7801bbf0 tx=0x7f6b78004ce0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 shutdown_connections 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c848f8d9b0 0x55c848f8fdd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c848f1c0b0 0x55c848f29b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c848f26950 0x55c848f2a060 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b65ffb640 0 client.17841 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 --2- 172.21.15.44:0/3997999490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c848f1b790 0x55c848f2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 >> 172.21.15.44:0/3997999490 conn(0x55c848f17d00 msgr2=0x55c848f21b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 shutdown_connections 2024-09-13T17:33:50.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 1 -- 172.21.15.44:0/3997999490 wait complete. 2024-09-13T17:33:50.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.935+0000 7f6b875da740 20 client.17841 trim_cache size 0 max 16384 2024-09-13T17:33:50.944 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6b680a1200 msgr2=0x7f6b680a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6b680a1200 0x7f6b680a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6b780114a0 tx=0x7f6b78004a40 comp rx=0 tx=0).stop 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 msgr2=0x7f6b80196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b80196f10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005ab0 tx=0x7f6b70032c20 comp rx=0 tx=0).stop 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 shutdown_connections 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6b680a1200 0x7f6b680a36c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b80197a30 0x7f6b801a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80103fd0 0x7f6b80197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 --2- 172.21.15.44:0/560343824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b800fee80 0x7f6b80196f10 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 >> 172.21.15.44:0/560343824 conn(0x7f6b800fa930 msgr2=0x7f6b800fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:50.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 shutdown_connections 2024-09-13T17:33:50.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:50.936+0000 7f6b848b8640 1 -- 172.21.15.44:0/560343824 wait complete. 2024-09-13T17:33:50.946 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2473347852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:33:51.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2473347852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:33:51.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:51 smithi044 ceph-mon[31175]: pgmap v219: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:33:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:51 smithi070 ceph-mon[34502]: pgmap v219: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:33:51.949 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:33:51.949 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:33:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:51 smithi100 ceph-mon[35055]: pgmap v219: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/3074355616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc100f40 msgr2=0x7fbdbc101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/3074355616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc100f40 0x7fbdbc101320 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009900 tx=0x7fbdb002a410 comp rx=0 tx=0).stop 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/3074355616 shutdown_connections 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/3074355616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdbc10a310 0x7fbdbc10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/3074355616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdbc101860 0x7fbdbc109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/3074355616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc100f40 0x7fbdbc101320 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/3074355616 >> 172.21.15.44:0/3074355616 conn(0x7fbdbc0fa930 msgr2=0x7fbdbc0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/3074355616 shutdown_connections 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.067+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/3074355616 wait complete. 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 Processor -- start 2024-09-13T17:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 -- start start 2024-09-13T17:33:52.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdbc100f40 0x7fbdbc196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdbc101860 0x7fbdbc196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 0x7fbdbc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc19b030 con 0x7fbdbc10a310 2024-09-13T17:33:52.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc19b1a0 con 0x7fbdbc101860 2024-09-13T17:33:52.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc2b5e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc19b360 con 0x7fbdbc100f40 2024-09-13T17:33:52.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc10fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 0x7fbdbc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc10fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 0x7fbdbc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48562/0 (socket says 172.21.15.44:48562) 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc10fb640 1 -- 172.21.15.44:0/1933092695 learned_addr learned my addr 172.21.15.44:0/1933092695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdbc100f40 0x7fbdbc196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.068+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdbc101860 0x7fbdbc196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc10fb640 1 -- 172.21.15.44:0/1933092695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdbc100f40 msgr2=0x7fbdbc196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc10fb640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdbc100f40 0x7fbdbc196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc10fb640 1 -- 172.21.15.44:0/1933092695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdbc101860 msgr2=0x7fbdbc196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc10fb640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdbc101860 0x7fbdbc196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc10fb640 1 -- 172.21.15.44:0/1933092695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0009560 con 0x7fbdbc10a310 2024-09-13T17:33:52.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc10fb640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 0x7fbdbc19aa30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400cce0 tx=0x7fbdb400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdb9ffb640 1 -- 172.21.15.44:0/1933092695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb400f450 con 0x7fbdbc10a310 2024-09-13T17:33:52.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc19b640 con 0x7fbdbc10a310 2024-09-13T17:33:52.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdb9ffb640 1 -- 172.21.15.44:0/1933092695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbdb400fa40 con 0x7fbdbc10a310 2024-09-13T17:33:52.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdb9ffb640 1 -- 172.21.15.44:0/1933092695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40189a0 con 0x7fbdbc10a310 2024-09-13T17:33:52.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.069+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc1a2ec0 con 0x7fbdbc10a310 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.071+0000 7fbdb9ffb640 1 -- 172.21.15.44:0/1933092695 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbdb400fbb0 con 0x7fbdbc10a310 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 Processor -- start 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 -- start start 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdb9ffb640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbdac0a1200 0x7fbdac0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f2326a0 0x55bb2f232a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 0x55bb2f241850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f241d90 0x55bb2f244180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f19dc60 con 0x55bb2f232fc0 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f19b3f0 con 0x55bb2f241d90 2024-09-13T17:33:52.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdc5881740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f196ce0 con 0x55bb2f2326a0 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbdac0a1200 0x7fbdac0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdb9ffb640 1 -- 172.21.15.44:0/1933092695 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fbdb40cd4e0 con 0x7fbdbc10a310 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.072+0000 7fbdbbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 0x55bb2f241850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 0x55bb2f241850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48572/0 (socket says 172.21.15.44:48572) 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 -- 172.21.15.44:0/3357480946 learned_addr learned my addr 172.21.15.44:0/3357480946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f2326a0 0x55bb2f232a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdc10fb640 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f241d90 0x55bb2f244180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 -- 172.21.15.44:0/3357480946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f2326a0 msgr2=0x55bb2f232a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f2326a0 0x55bb2f232a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 -- 172.21.15.44:0/3357480946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f241d90 msgr2=0x55bb2f244180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f241d90 0x55bb2f244180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 -- 172.21.15.44:0/3357480946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2f152970 con 0x55bb2f232fc0 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbdac0a1200 0x7fbdac0a36c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbda80138f0 tx=0x7fbda800bad0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 0x55bb2f241850 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fbda80107a0 tx=0x7fbda801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721c44ab738ee50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbda6ffd640 1 -- 172.21.15.44:0/3357480946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8010ee0 con 0x55bb2f232fc0 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbda6ffd640 1 -- 172.21.15.44:0/3357480946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbda8019040 con 0x55bb2f232fc0 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.073+0000 7fbda6ffd640 1 -- 172.21.15.44:0/3357480946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8005690 con 0x55bb2f232fc0 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 -- 172.21.15.44:0/3357480946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 msgr2=0x55bb2f241850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 0x55bb2f241850 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fbda80107a0 tx=0x7fbda801a000 comp rx=0 tx=0).stop 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 -- 172.21.15.44:0/3357480946 shutdown_connections 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f241d90 0x55bb2f244180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f232fc0 0x55bb2f241850 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3357480946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f2326a0 0x55bb2f232a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 -- 172.21.15.44:0/3357480946 >> 172.21.15.44:0/3357480946 conn(0x55bb2f22ed50 msgr2=0x55bb2f22f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 -- 172.21.15.44:0/3357480946 shutdown_connections 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 -- 172.21.15.44:0/3357480946 wait complete. 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 -- start start 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.074+0000 7fbdc5881740 1 Processor -- start 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f2417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f23ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f23f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f154f80 con 0x55bb2f2326a0 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f154c20 con 0x55bb2f232fc0 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f121ab0 con 0x55bb2f241d90 2024-09-13T17:33:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc5881740 10 client.0 _ll_register_callbacks cb 0x55bb2f190350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f2417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f2417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48578/0 (socket says 172.21.15.44:48578) 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 -- 172.21.15.44:0/1978202634 learned_addr learned my addr 172.21.15.44:0/1978202634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f23ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc10fb640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f23f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 -- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 msgr2=0x55bb2f23f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f23f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 -- 172.21.15.44:0/1978202634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 msgr2=0x55bb2f23ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f23ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.075+0000 7fbdc08fa640 1 -- 172.21.15.44:0/1978202634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80109d0 con 0x55bb2f2326a0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f2417b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009900 tx=0x7fbdb00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee8784a64c24cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbd9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbd9bfff640 1 -- 172.21.15.44:0/1978202634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0031d30 con 0x55bb2f2326a0 2024-09-13T17:33:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbd9bfff640 1 -- 172.21.15.44:0/1978202634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fbdb00035f0 con 0x55bb2f2326a0 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb2ee8a590 con 0x55bb2f2326a0 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbd9bfff640 1 -- 172.21.15.44:0/1978202634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdb0031e90 con 0x55bb2f2326a0 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 10 client.17853 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb2f195360 con 0x55bb2f2326a0 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbd9bfff640 1 -- 172.21.15.44:0/1978202634 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bb2ee8a590 con 0x55bb2f2326a0 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbd9bfff640 1 -- 172.21.15.44:0/1978202634 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdb00468a0 con 0x55bb2f2326a0 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 10 client.17853 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 20 client.17853 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f251bd0 0x55bb2f253fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 4 client.17853 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:52.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.076+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb2ee9b860 con 0x55bb2f251bd0 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.077+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f251bd0 0x55bb2f253fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.077+0000 7fbd9bfff640 10 client.17853 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.077+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f251bd0 0x55bb2f253fc0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18d8acd40f40b38 server_cookie=4f45578f65db15e4 in_seq=0 out_seq=0 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbd9bfff640 1 -- 172.21.15.44:0/1978202634 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bb2ee9b860 con 0x55bb2f251bd0 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbd9bfff640 10 client.17853 handle_command_reply: tid=0 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 client.17853 shutdown 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 msgr2=0x55bb2f2417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f2417b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009900 tx=0x7fbdb00043b0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 shutdown_connections 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f251bd0 0x55bb2f253fc0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f23f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f23ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbd9bfff640 0 client.17853 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 --2- 172.21.15.44:0/1978202634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f2417b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 >> 172.21.15.44:0/1978202634 conn(0x55bb2f22ed50 msgr2=0x55bb2f22e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 shutdown_connections 2024-09-13T17:33:52.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 1 -- 172.21.15.44:0/1978202634 wait complete. 2024-09-13T17:33:52.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.078+0000 7fbdc5881740 20 client.17853 trim_cache size 0 max 16384 2024-09-13T17:33:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 Processor -- start 2024-09-13T17:33:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 -- start start 2024-09-13T17:33:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f23e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f241310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f243180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f22a940 con 0x55bb2f2326a0 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f251a20 con 0x55bb2f232fc0 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc5881740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f220ef0 con 0x55bb2f241d90 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f23e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f23e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48594/0 (socket says 172.21.15.44:48594) 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 -- 172.21.15.44:0/2784327577 learned_addr learned my addr 172.21.15.44:0/2784327577 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f241310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 -- 172.21.15.44:0/2784327577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 msgr2=0x55bb2f243180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f243180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 -- 172.21.15.44:0/2784327577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 msgr2=0x55bb2f241310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f241310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 -- 172.21.15.44:0/2784327577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2f1371c0 con 0x55bb2f2326a0 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f241310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.093+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f23e680 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0005ab0 tx=0x7fbdb002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a2102bdd22a7a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbda77fe640 1 -- 172.21.15.44:0/2784327577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0031d30 con 0x55bb2f2326a0 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbda77fe640 1 -- 172.21.15.44:0/2784327577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbdb0031e90 con 0x55bb2f2326a0 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbda77fe640 1 -- 172.21.15.44:0/2784327577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb003d840 con 0x55bb2f2326a0 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 -- 172.21.15.44:0/2784327577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 msgr2=0x55bb2f23e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f23e680 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0005ab0 tx=0x7fbdb002a410 comp rx=0 tx=0).stop 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 -- 172.21.15.44:0/2784327577 shutdown_connections 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f243180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f241310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 --2- 172.21.15.44:0/2784327577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f23e680 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 -- 172.21.15.44:0/2784327577 >> 172.21.15.44:0/2784327577 conn(0x55bb2f22ed50 msgr2=0x55bb2f23df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 -- 172.21.15.44:0/2784327577 shutdown_connections 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.094+0000 7fbdc5881740 1 -- 172.21.15.44:0/2784327577 wait complete. 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 -- start start 2024-09-13T17:33:52.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 Processor -- start 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f248db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f246af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f247030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f247640 con 0x55bb2f2326a0 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f2477b0 con 0x55bb2f232fc0 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f247970 con 0x55bb2f241d90 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc5881740 10 client.0 _ll_register_callbacks cb 0x7fbdb0031610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:52.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc08fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f248db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc08fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f248db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48606/0 (socket says 172.21.15.44:48606) 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc08fa640 1 -- 172.21.15.44:0/3969274178 learned_addr learned my addr 172.21.15.44:0/3969274178 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f246af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.095+0000 7fbdc10fb640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f247030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc08fa640 1 -- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 msgr2=0x55bb2f247030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f247030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc08fa640 1 -- 172.21.15.44:0/3969274178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 msgr2=0x55bb2f246af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f246af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc08fa640 1 -- 172.21.15.44:0/3969274178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2f195360 con 0x55bb2f2326a0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc08fa640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f248db0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0046d50 tx=0x7fbdb0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebbdb5da38ef4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbd9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbd9bfff640 1 -- 172.21.15.44:0/3969274178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0046890 con 0x55bb2f2326a0 2024-09-13T17:33:52.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbd9bfff640 1 -- 172.21.15.44:0/3969274178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fbdb0000e40 con 0x55bb2f2326a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbda8019040 con 0x55bb2f2326a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbd9bfff640 1 -- 172.21.15.44:0/3969274178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdb003d5e0 con 0x55bb2f2326a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc5881740 10 client.17859 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb2f15c990 con 0x55bb2f2326a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbd9bfff640 1 -- 172.21.15.44:0/3969274178 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbda8019040 con 0x55bb2f2326a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbd9bfff640 1 -- 172.21.15.44:0/3969274178 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdb0003850 con 0x55bb2f2326a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.096+0000 7fbdc5881740 10 client.17859 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdc5881740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdc5881740 20 client.17859 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f2aa1a0 0x55bb2f2ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdc5881740 4 client.17859 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bb2eeeeeb0 con 0x55bb2f2aa1a0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f2aa1a0 0x55bb2f2ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbdbbfff640 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f2aa1a0 0x55bb2f2ac590 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0945320b16d0f55 server_cookie=ed853e2a57577d8f in_seq=0 out_seq=0 2024-09-13T17:33:52.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.097+0000 7fbd9bfff640 10 client.17859 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.098+0000 7fbd9bfff640 1 -- 172.21.15.44:0/3969274178 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bb2eeeeeb0 con 0x55bb2f2aa1a0 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.098+0000 7fbd9bfff640 10 client.17859 handle_command_reply: tid=0 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.098+0000 7fbdc5881740 1 client.17859 shutdown 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.098+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 msgr2=0x55bb2f248db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.098+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f248db0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0046d50 tx=0x7fbdb0005a20 comp rx=0 tx=0).stop 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 shutdown_connections 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bb2f2aa1a0 0x55bb2f2ac590 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb2f241d90 0x55bb2f247030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb2f232fc0 0x55bb2f246af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 --2- 172.21.15.44:0/3969274178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb2f2326a0 0x55bb2f248db0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbd9bfff640 0 client.17859 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 >> 172.21.15.44:0/3969274178 conn(0x55bb2f22ed50 msgr2=0x55bb2f2349d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 shutdown_connections 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 1 -- 172.21.15.44:0/3969274178 wait complete. 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc5881740 20 client.17859 trim_cache size 0 max 16384 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "3b24448c-617c-4202-bf1d-68087165276c", 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:33:52.108 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbdac0a1200 msgr2=0x7fbdac0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbdac0a1200 0x7fbdac0a36c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbda80138f0 tx=0x7fbda800bad0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 msgr2=0x7fbdbc19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 0x7fbdbc19aa30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400cce0 tx=0x7fbdb400a620 comp rx=0 tx=0).stop 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 shutdown_connections 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.099+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbdac0a1200 0x7fbdac0a36c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.100+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdbc10a310 0x7fbdbc19aa30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.100+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdbc101860 0x7fbdbc196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.100+0000 7fbdc2b5e640 1 --2- 172.21.15.44:0/1933092695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdbc100f40 0x7fbdbc196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.100+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 >> 172.21.15.44:0/1933092695 conn(0x7fbdbc0fa930 msgr2=0x7fbdbc0fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.100+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 shutdown_connections 2024-09-13T17:33:52.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.100+0000 7fbdc2b5e640 1 -- 172.21.15.44:0/1933092695 wait complete. 2024-09-13T17:33:52.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:33:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 -- 172.21.15.44:0/874576442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f80ffef0 msgr2=0x7f39f81002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 --2- 172.21.15.44:0/874576442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f80ffef0 0x7f39f81002d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009860 tx=0x7f39f402a340 comp rx=0 tx=0).stop 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 -- 172.21.15.44:0/874576442 shutdown_connections 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 --2- 172.21.15.44:0/874576442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39f81008a0 0x7f39f81048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 --2- 172.21.15.44:0/874576442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f8105040 0x7f39f810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 --2- 172.21.15.44:0/874576442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f80ffef0 0x7f39f81002d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.215+0000 7f3a006b7640 1 -- 172.21.15.44:0/874576442 >> 172.21.15.44:0/874576442 conn(0x7f39f80fb980 msgr2=0x7f39f80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 -- 172.21.15.44:0/874576442 shutdown_connections 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 -- 172.21.15.44:0/874576442 wait complete. 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 Processor -- start 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 -- start start 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 0x7f39f8196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39f8105040 0x7f39f81967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f819ab60 0x7f39f819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819b550 con 0x7f39f80ffef0 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819b6c0 con 0x7f39f819ab60 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f3a006b7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819b830 con 0x7f39f8105040 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f39fe453640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 0x7f39f8196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f39fe453640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 0x7f39f8196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48620/0 (socket says 172.21.15.44:48620) 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.216+0000 7f39fe453640 1 -- 172.21.15.44:0/2279898550 learned_addr learned my addr 172.21.15.44:0/2279898550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fec54640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f819ab60 0x7f39f819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fdc52640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39f8105040 0x7f39f81967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fe453640 1 -- 172.21.15.44:0/2279898550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39f8105040 msgr2=0x7f39f81967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fe453640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39f8105040 0x7f39f81967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fe453640 1 -- 172.21.15.44:0/2279898550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f819ab60 msgr2=0x7f39f819b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fe453640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f819ab60 0x7f39f819b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fe453640 1 -- 172.21.15.44:0/2279898550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f0009900 con 0x7f39f80ffef0 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39fe453640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 0x7f39f8196280 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009980 tx=0x7f39f4033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39ef7fe640 1 -- 172.21.15.44:0/2279898550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4030ed0 con 0x7f39f80ffef0 2024-09-13T17:33:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f4009490 con 0x7f39f80ffef0 2024-09-13T17:33:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39ef7fe640 1 -- 172.21.15.44:0/2279898550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39f40044d0 con 0x7f39f80ffef0 2024-09-13T17:33:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f39ef7fe640 1 -- 172.21.15.44:0/2279898550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4003e80 con 0x7f39f80ffef0 2024-09-13T17:33:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.217+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81a3100 con 0x7f39f80ffef0 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.219+0000 7f39ef7fe640 1 -- 172.21.15.44:0/2279898550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f39f4039040 con 0x7f39f80ffef0 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.219+0000 7f3a033dc740 1 Processor -- start 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 -- start start 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 0x562b4cd5aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd5d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4ccbaf20 con 0x562b4cd50730 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4ccb86b0 con 0x562b4cd4fe10 2024-09-13T17:33:52.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f3a033dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4ccb3fa0 con 0x562b4cd5afd0 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f39fe453640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 0x562b4cd5aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f39fe453640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 0x562b4cd5aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48628/0 (socket says 172.21.15.44:48628) 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f39fe453640 1 -- 172.21.15.44:0/256122220 learned_addr learned my addr 172.21.15.44:0/256122220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f39fec54640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f39ef7fe640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f39e40a1280 0x7f39e40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.220+0000 7f39fdc52640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd5d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fe453640 1 -- 172.21.15.44:0/256122220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 msgr2=0x562b4cd5d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fe453640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd5d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fe453640 1 -- 172.21.15.44:0/256122220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 msgr2=0x562b4cd501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fe453640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fe453640 1 -- 172.21.15.44:0/256122220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b4cc6e840 con 0x562b4cd50730 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fdc52640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd5d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fec54640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fec54640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f39e40a1280 0x7f39e40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39ef7fe640 1 -- 172.21.15.44:0/2279898550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f39f40ed460 con 0x7f39f80ffef0 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fe453640 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 0x562b4cd5aa90 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f39f40ea830 tx=0x7f39f40857f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc40de939cb94aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39ecff9640 1 -- 172.21.15.44:0/256122220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40b9df0 con 0x562b4cd50730 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39fec54640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f39e40a1280 0x7f39e40a3740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39f00049d0 tx=0x7f39f0004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39ecff9640 1 -- 172.21.15.44:0/256122220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39f40b9c10 con 0x562b4cd50730 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.221+0000 7f39ecff9640 1 -- 172.21.15.44:0/256122220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40ba9b0 con 0x562b4cd50730 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 -- 172.21.15.44:0/256122220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 msgr2=0x562b4cd5aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 0x562b4cd5aa90 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f39f40ea830 tx=0x7f39f40857f0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 -- 172.21.15.44:0/256122220 shutdown_connections 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd5d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd50730 0x562b4cd5aa90 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 --2- 172.21.15.44:0/256122220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 -- 172.21.15.44:0/256122220 >> 172.21.15.44:0/256122220 conn(0x562b4cd4c380 msgr2=0x562b4cd4c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 -- 172.21.15.44:0/256122220 shutdown_connections 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 -- 172.21.15.44:0/256122220 wait complete. 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 -- start start 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.222+0000 7f3a033dc740 1 Processor -- start 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 0x562b4cd5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cc552a0 con 0x562b4cd4fe10 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4c98c430 con 0x562b4cd50730 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cc37680 con 0x562b4cd5afd0 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 10 client.0 _ll_register_callbacks cb 0x562b4ccad610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 0x562b4cd5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 0x562b4cd5a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48636/0 (socket says 172.21.15.44:48636) 2024-09-13T17:33:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 -- 172.21.15.44:0/1592440727 learned_addr learned my addr 172.21.15.44:0/1592440727 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fdc52640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fec54640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 -- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 msgr2=0x562b4cd580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 -- 172.21.15.44:0/1592440727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 msgr2=0x562b4cd57b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd57b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 -- 172.21.15.44:0/1592440727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81a3100 con 0x562b4cd4fe10 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fec54640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fdc52640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39fe453640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 0x562b4cd5a8e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f39f40e7be0 tx=0x7f39f40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1def4efc1f6903 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39e1ffb640 1 -- 172.21.15.44:0/1592440727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4033e70 con 0x562b4cd4fe10 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39e1ffb640 1 -- 172.21.15.44:0/1592440727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f39f40ba9b0 con 0x562b4cd4fe10 2024-09-13T17:33:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b4c9a74e0 con 0x562b4cd4fe10 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f39e1ffb640 1 -- 172.21.15.44:0/1592440727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f408d940 con 0x562b4cd4fe10 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.223+0000 7f3a033dc740 10 client.17871 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b4ccb2620 con 0x562b4cd4fe10 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f39e1ffb640 1 -- 172.21.15.44:0/1592440727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562b4c9a74e0 con 0x562b4cd4fe10 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f39e1ffb640 1 -- 172.21.15.44:0/1592440727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39f40ed0a0 con 0x562b4cd4fe10 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 10 client.17871 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 20 client.17871 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cd6a560 0x562b4cd6c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 4 client.17871 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b4c9b87e0 con 0x562b4cd6a560 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.224+0000 7f39fdc52640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cd6a560 0x562b4cd6c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.225+0000 7f39e1ffb640 10 client.17871 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.225+0000 7f39fdc52640 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cd6a560 0x562b4cd6c980 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf650e648996a9b6 server_cookie=8382ba8de942a7d7 in_seq=0 out_seq=0 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.225+0000 7f39e1ffb640 1 -- 172.21.15.44:0/1592440727 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562b4c9b87e0 con 0x562b4cd6a560 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.225+0000 7f39e1ffb640 10 client.17871 handle_command_reply: tid=0 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.225+0000 7f3a033dc740 1 client.17871 shutdown 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 msgr2=0x562b4cd5a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 0x562b4cd5a8e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f39f40e7be0 tx=0x7f39f40035f0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 shutdown_connections 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cd6a560 0x562b4cd6c980 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd5afd0 0x562b4cd580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f39e1ffb640 0 client.17871 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 --2- 172.21.15.44:0/1592440727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd4fe10 0x562b4cd5a8e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 >> 172.21.15.44:0/1592440727 conn(0x562b4cd4c380 msgr2=0x562b4cd520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 shutdown_connections 2024-09-13T17:33:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 1 -- 172.21.15.44:0/1592440727 wait complete. 2024-09-13T17:33:52.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.226+0000 7f3a033dc740 20 client.17871 trim_cache size 0 max 16384 2024-09-13T17:33:52.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 Processor -- start 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 -- start start 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd4fe10 0x562b4cd52e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd53390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd56410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4c9a74e0 con 0x562b4cd5afd0 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cd529e0 con 0x562b4cd50730 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f3a033dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cd50fb0 con 0x562b4cd4fe10 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd56410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd56410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48652/0 (socket says 172.21.15.44:48652) 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 -- 172.21.15.44:0/2244710392 learned_addr learned my addr 172.21.15.44:0/2244710392 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fdc52640 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd53390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fe453640 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd4fe10 0x562b4cd52e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 -- 172.21.15.44:0/2244710392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd4fe10 msgr2=0x562b4cd52e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd4fe10 0x562b4cd52e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 -- 172.21.15.44:0/2244710392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 msgr2=0x562b4cd53390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd53390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.240+0000 7f39fec54640 1 -- 172.21.15.44:0/2244710392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b4cc54790 con 0x562b4cd5afd0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f39fec54640 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd56410 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f39f000da60 tx=0x7f39f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8a5af8f68d6f95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f39ed7fa640 1 -- 172.21.15.44:0/2244710392 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0019d00 con 0x562b4cd5afd0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f39ed7fa640 1 -- 172.21.15.44:0/2244710392 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39f0019e60 con 0x562b4cd5afd0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f39ed7fa640 1 -- 172.21.15.44:0/2244710392 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00185b0 con 0x562b4cd5afd0 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 -- 172.21.15.44:0/2244710392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 msgr2=0x562b4cd56410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd56410 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f39f000da60 tx=0x7f39f0017000 comp rx=0 tx=0).stop 2024-09-13T17:33:52.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 -- 172.21.15.44:0/2244710392 shutdown_connections 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd56410 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd50730 0x562b4cd53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 --2- 172.21.15.44:0/2244710392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd4fe10 0x562b4cd52e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 -- 172.21.15.44:0/2244710392 >> 172.21.15.44:0/2244710392 conn(0x562b4cd4c380 msgr2=0x562b4cd4c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 -- 172.21.15.44:0/2244710392 shutdown_connections 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.241+0000 7f3a033dc740 1 -- 172.21.15.44:0/2244710392 wait complete. 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 -- start start 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 Processor -- start 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd50730 0x562b4cd60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cd60fb0 con 0x562b4cd5afd0 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cd61120 con 0x562b4cd4fe10 2024-09-13T17:33:52.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4cdc0a30 con 0x562b4cd50730 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f3a033dc740 10 client.0 _ll_register_callbacks cb 0x7f39f40b6680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48668/0 (socket says 172.21.15.44:48668) 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 -- 172.21.15.44:0/3586385293 learned_addr learned my addr 172.21.15.44:0/3586385293 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fe453640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fdc52640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd50730 0x562b4cd60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 -- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd50730 msgr2=0x562b4cd60310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd50730 0x562b4cd60310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 -- 172.21.15.44:0/3586385293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 msgr2=0x562b4cd61e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd61e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.242+0000 7f39fec54640 1 -- 172.21.15.44:0/3586385293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f000d660 con 0x562b4cd5afd0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39fec54640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd608b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f39f000d980 tx=0x7f39f0026660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cf943910dccc2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39e1ffb640 1 -- 172.21.15.44:0/3586385293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0004cb0 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39e1ffb640 1 -- 172.21.15.44:0/3586385293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f39f0021040 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39f40b9c10 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39e1ffb640 1 -- 172.21.15.44:0/3586385293 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f0026c20 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 10 client.17877 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b4cc79ea0 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39e1ffb640 1 -- 172.21.15.44:0/3586385293 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39f40b9c10 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f39e1ffb640 1 -- 172.21.15.44:0/3586385293 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39f0005810 con 0x562b4cd5afd0 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 10 client.17877 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 20 client.17877 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cdc1fd0 0x562b4cdc43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 4 client.17877 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:33:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.243+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b4ca0be30 con 0x562b4cdc1fd0 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f39fe453640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cdc1fd0 0x562b4cdc43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f39fe453640 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cdc1fd0 0x562b4cdc43f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc064927b143bdaf server_cookie=302f5352a5520212 in_seq=0 out_seq=0 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f39e1ffb640 10 client.17877 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f39e1ffb640 1 -- 172.21.15.44:0/3586385293 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562b4ca0be30 con 0x562b4cdc1fd0 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f39e1ffb640 10 client.17877 handle_command_reply: tid=0 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f3a033dc740 1 client.17877 shutdown 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 msgr2=0x562b4cd608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f3a033dc740 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd608b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f39f000d980 tx=0x7f39f0026660 comp rx=0 tx=0).stop 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 shutdown_connections 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f3a033dc740 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562b4cdc1fd0 0x562b4cdc43f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.244+0000 7f3a033dc740 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b4cd50730 0x562b4cd60310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f39e1ffb640 0 client.17877 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a033dc740 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4cd5afd0 0x562b4cd608b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a033dc740 1 --2- 172.21.15.44:0/3586385293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562b4cd4fe10 0x562b4cd61e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 >> 172.21.15.44:0/3586385293 conn(0x562b4cd4c380 msgr2=0x562b4cd4c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 shutdown_connections 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a033dc740 1 -- 172.21.15.44:0/3586385293 wait complete. 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a033dc740 20 client.17877 trim_cache size 0 max 16384 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout: "3b24448c-617c-4202-bf1d-68087165276c": { 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "3b24448c-617c-4202-bf1d-68087165276c", 2024-09-13T17:33:52.254 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:33:52.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:33:52.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:33:52.255 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:33:52.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f39e40a1280 msgr2=0x7f39e40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f39e40a1280 0x7f39e40a3740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39f00049d0 tx=0x7f39f0004a40 comp rx=0 tx=0).stop 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 msgr2=0x7f39f8196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 0x7f39f8196280 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009980 tx=0x7f39f4033d00 comp rx=0 tx=0).stop 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 shutdown_connections 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f39e40a1280 0x7f39e40a3740 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39f8105040 0x7f39f81967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39f819ab60 0x7f39f819b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 --2- 172.21.15.44:0/2279898550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39f80ffef0 0x7f39f8196280 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 >> 172.21.15.44:0/2279898550 conn(0x7f39f80fb980 msgr2=0x7f39f80fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 shutdown_connections 2024-09-13T17:33:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:33:52.245+0000 7f3a006b7640 1 -- 172.21.15.44:0/2279898550 wait complete. 2024-09-13T17:33:52.258 INFO:tasks.cephfs.filesystem:scrub status for tag:3b24448c-617c-4202-bf1d-68087165276c - {'path': '/', 'tag': '3b24448c-617c-4202-bf1d-68087165276c', 'options': 'recursive,force'} 2024-09-13T17:33:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 -- 172.21.15.44:0/1719476234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 msgr2=0x7f452010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 --2- 172.21.15.44:0/1719476234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f452010ba40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4518009930 tx=0x7f451802a420 comp rx=0 tx=0).stop 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 -- 172.21.15.44:0/1719476234 shutdown_connections 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 --2- 172.21.15.44:0/1719476234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f452010ba40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 --2- 172.21.15.44:0/1719476234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45200feff0 0x7f45200ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 --2- 172.21.15.44:0/1719476234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4520104c30 0x7f4520105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 -- 172.21.15.44:0/1719476234 >> 172.21.15.44:0/1719476234 conn(0x7f4520074940 msgr2=0x7f4520074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 -- 172.21.15.44:0/1719476234 shutdown_connections 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.378+0000 7f4527ca3640 1 -- 172.21.15.44:0/1719476234 wait complete. 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 Processor -- start 2024-09-13T17:33:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 -- start start 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45200feff0 0x7f45201961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f45201966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4520104c30 0x7f452019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019b080 con 0x7f45200ff970 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019b1f0 con 0x7f4520104c30 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4527ca3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019b3b0 con 0x7f45200feff0 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f45201966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f45201966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48688/0 (socket says 172.21.15.44:48688) 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 -- 172.21.15.44:0/996845843 learned_addr learned my addr 172.21.15.44:0/996845843 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4525a3f640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45200feff0 0x7f45201961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f4526240640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4520104c30 0x7f452019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 -- 172.21.15.44:0/996845843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45200feff0 msgr2=0x7f45201961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45200feff0 0x7f45201961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 -- 172.21.15.44:0/996845843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4520104c30 msgr2=0x7f452019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4520104c30 0x7f452019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.379+0000 7f452523e640 1 -- 172.21.15.44:0/996845843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4518009560 con 0x7f45200ff970 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.380+0000 7f452523e640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f45201966e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f451400c890 tx=0x7f451400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.380+0000 7f451effd640 1 -- 172.21.15.44:0/996845843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514004220 con 0x7f45200ff970 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.380+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452019b690 con 0x7f45200ff970 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.380+0000 7f451effd640 1 -- 172.21.15.44:0/996845843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4514007e40 con 0x7f45200ff970 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.380+0000 7f451effd640 1 -- 172.21.15.44:0/996845843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514004a00 con 0x7f45200ff970 2024-09-13T17:33:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.380+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45201a2f10 con 0x7f45200ff970 2024-09-13T17:33:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.382+0000 7f451effd640 1 -- 172.21.15.44:0/996845843 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f45140079b0 con 0x7f45200ff970 2024-09-13T17:33:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.382+0000 7f452a9c8740 1 Processor -- start 2024-09-13T17:33:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 -- start start 2024-09-13T17:33:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1add710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1aea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1a48c30 con 0x559dc1ae84f0 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1a463c0 con 0x559dc1addc50 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452a9c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1a41cb0 con 0x559dc1add330 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452523e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1aea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f451effd640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45080a1200 0x7f45080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f4525a3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452523e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1aea8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48696/0 (socket says 172.21.15.44:48696) 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f452523e640 1 -- 172.21.15.44:0/1596222389 learned_addr learned my addr 172.21.15.44:0/1596222389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f4526240640 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1add710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.383+0000 7f4526240640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45080a1200 0x7f45080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4525a3f640 1 -- 172.21.15.44:0/1596222389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 msgr2=0x559dc1add710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4525a3f640 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1add710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f451effd640 1 -- 172.21.15.44:0/996845843 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f45140cd020 con 0x7f45200ff970 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4525a3f640 1 -- 172.21.15.44:0/1596222389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 msgr2=0x559dc1aea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4525a3f640 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1aea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4525a3f640 1 -- 172.21.15.44:0/1596222389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dc19e8910 con 0x559dc1addc50 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4526240640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45080a1200 0x7f45080a36c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4518005ab0 tx=0x7f4518005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4525a3f640 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae7fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f30 tx=0x7f4510010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa646a4ea3a9d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4507fff640 1 -- 172.21.15.44:0/1596222389 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510004d20 con 0x559dc1addc50 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4507fff640 1 -- 172.21.15.44:0/1596222389 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4510004e80 con 0x559dc1addc50 2024-09-13T17:33:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.384+0000 7f4507fff640 1 -- 172.21.15.44:0/1596222389 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100053a0 con 0x559dc1addc50 2024-09-13T17:33:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 -- 172.21.15.44:0/1596222389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 msgr2=0x559dc1ae7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae7fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f30 tx=0x7f4510010800 comp rx=0 tx=0).stop 2024-09-13T17:33:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 -- 172.21.15.44:0/1596222389 shutdown_connections 2024-09-13T17:33:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1aea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae7fb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 --2- 172.21.15.44:0/1596222389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1add710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 -- 172.21.15.44:0/1596222389 >> 172.21.15.44:0/1596222389 conn(0x559dc1ad98c0 msgr2=0x559dc1ad9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 -- 172.21.15.44:0/1596222389 shutdown_connections 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 -- 172.21.15.44:0/1596222389 wait complete. 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.385+0000 7f452a9c8740 1 -- start start 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 Processor -- start 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 0x559dc1ae7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1addc50 0x559dc1ae51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1ae84f0 0x559dc1ae5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc19ffe80 con 0x559dc1add330 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc19ffb20 con 0x559dc1ae84f0 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc19c5260 con 0x559dc1addc50 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452a9c8740 10 client.0 _ll_register_callbacks cb 0x559dc19fadf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f4525a3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 0x559dc1ae7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f4525a3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 0x559dc1ae7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48700/0 (socket says 172.21.15.44:48700) 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f4525a3f640 1 -- 172.21.15.44:0/52058456 learned_addr learned my addr 172.21.15.44:0/52058456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f4526240640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1ae84f0 0x559dc1ae5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.386+0000 7f452523e640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1addc50 0x559dc1ae51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f4525a3f640 1 -- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1addc50 msgr2=0x559dc1ae51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f4525a3f640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1addc50 0x559dc1ae51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f4525a3f640 1 -- 172.21.15.44:0/52058456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1ae84f0 msgr2=0x559dc1ae5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f4525a3f640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1ae84f0 0x559dc1ae5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f4525a3f640 1 -- 172.21.15.44:0/52058456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4510009b90 con 0x559dc1add330 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f452523e640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1addc50 0x559dc1ae51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f4525a3f640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 0x559dc1ae7ed0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4510006010 tx=0x7f4510004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b74568397e802fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f45057fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f45057fa640 1 -- 172.21.15.44:0/52058456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510021070 con 0x559dc1add330 2024-09-13T17:33:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f45057fa640 1 -- 172.21.15.44:0/52058456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4510012720 con 0x559dc1add330 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dc1a355d0 con 0x559dc1add330 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f45057fa640 1 -- 172.21.15.44:0/52058456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45100128e0 con 0x559dc1add330 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f452a9c8740 10 client.17889 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559dc1a40330 con 0x559dc1add330 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.387+0000 7f45057fa640 1 -- 172.21.15.44:0/52058456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559dc1a355d0 con 0x559dc1add330 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f45057fa640 1 -- 172.21.15.44:0/52058456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4510025410 con 0x559dc1add330 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452a9c8740 10 client.17889 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452a9c8740 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452a9c8740 20 client.17889 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452a9c8740 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1af7e50 0x559dc1afa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452a9c8740 4 client.17889 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dc1735300 con 0x559dc1af7e50 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452523e640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1af7e50 0x559dc1afa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f452523e640 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1af7e50 0x559dc1afa240 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d4ccac6a42da2a server_cookie=7405c5d87ded49e in_seq=0 out_seq=0 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.388+0000 7f45057fa640 10 client.17889 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f45057fa640 1 -- 172.21.15.44:0/52058456 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559dc1735300 con 0x559dc1af7e50 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f45057fa640 10 client.17889 handle_command_reply: tid=0 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 client.17889 shutdown 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 msgr2=0x559dc1ae7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 0x559dc1ae7ed0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4510006010 tx=0x7f4510004c70 comp rx=0 tx=0).stop 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 shutdown_connections 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1af7e50 0x559dc1afa240 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1addc50 0x559dc1ae51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1ae84f0 0x559dc1ae5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f45057fa640 0 client.17889 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 --2- 172.21.15.44:0/52058456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1add330 0x559dc1ae7ed0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.389+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 >> 172.21.15.44:0/52058456 conn(0x559dc1ad98c0 msgr2=0x559dc1ad8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.390+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 shutdown_connections 2024-09-13T17:33:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.390+0000 7f452a9c8740 1 -- 172.21.15.44:0/52058456 wait complete. 2024-09-13T17:33:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.390+0000 7f452a9c8740 20 client.17889 trim_cache size 0 max 16384 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.402+0000 7f452a9c8740 1 Processor -- start 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.402+0000 7f452a9c8740 1 -- start start 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1ae79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1ae3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452a9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1ae44f0 con 0x559dc1ae84f0 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452a9c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1adda40 con 0x559dc1addc50 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452a9c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1ade250 con 0x559dc1add330 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1ae3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1ae3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48712/0 (socket says 172.21.15.44:48712) 2024-09-13T17:33:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 -- 172.21.15.44:0/546095918 learned_addr learned my addr 172.21.15.44:0/546095918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452523e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4525a3f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1ae79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 -- 172.21.15.44:0/546095918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 msgr2=0x559dc1ae79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1ae79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 -- 172.21.15.44:0/546095918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 msgr2=0x559dc1ae3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 -- 172.21.15.44:0/546095918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dc19e2300 con 0x559dc1ae84f0 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f452523e640 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f4526240640 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1ae3fb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f451802aa10 tx=0x7f4518005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57db7b1b3572092b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.403+0000 7f451cff9640 1 -- 172.21.15.44:0/546095918 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518003640 con 0x559dc1ae84f0 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f451cff9640 1 -- 172.21.15.44:0/546095918 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4518003c30 con 0x559dc1ae84f0 2024-09-13T17:33:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f451cff9640 1 -- 172.21.15.44:0/546095918 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451803db10 con 0x559dc1ae84f0 2024-09-13T17:33:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 -- 172.21.15.44:0/546095918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 msgr2=0x559dc1ae3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1ae3fb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f451802aa10 tx=0x7f4518005890 comp rx=0 tx=0).stop 2024-09-13T17:33:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 -- 172.21.15.44:0/546095918 shutdown_connections 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1ae84f0 0x559dc1ae3fb0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1addc50 0x559dc1ae3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 --2- 172.21.15.44:0/546095918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1add330 0x559dc1ae79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 -- 172.21.15.44:0/546095918 >> 172.21.15.44:0/546095918 conn(0x559dc1ad98c0 msgr2=0x559dc1ad4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 -- 172.21.15.44:0/546095918 shutdown_connections 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.404+0000 7f452a9c8740 1 -- 172.21.15.44:0/546095918 wait complete. 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 -- start start 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 Processor -- start 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1add330 0x559dc1aef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 0x559dc1aeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1ae84f0 0x559dc1aebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1aec470 con 0x559dc1addc50 2024-09-13T17:33:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1aec5e0 con 0x559dc1add330 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc1b4e2f0 con 0x559dc1ae84f0 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452a9c8740 10 client.0 _ll_register_callbacks cb 0x7f4510009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452523e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 0x559dc1aeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452523e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 0x559dc1aeb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48720/0 (socket says 172.21.15.44:48720) 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f452523e640 1 -- 172.21.15.44:0/2822173896 learned_addr learned my addr 172.21.15.44:0/2822173896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f4525a3f640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1add330 0x559dc1aef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.405+0000 7f4526240640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1ae84f0 0x559dc1aebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452523e640 1 -- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1ae84f0 msgr2=0x559dc1aebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452523e640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1ae84f0 0x559dc1aebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452523e640 1 -- 172.21.15.44:0/2822173896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1add330 msgr2=0x559dc1aef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452523e640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1add330 0x559dc1aef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452523e640 1 -- 172.21.15.44:0/2822173896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45201a2f10 con 0x559dc1addc50 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452523e640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 0x559dc1aeb830 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4514005fd0 tx=0x7f451401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a42f60c5b43e1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:33:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f45057fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f45057fa640 1 -- 172.21.15.44:0/2822173896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514099e70 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f45057fa640 1 -- 172.21.15.44:0/2822173896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f451409dbf0 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dc1a6a680 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f45057fa640 1 -- 172.21.15.44:0/2822173896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45140a2240 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452a9c8740 10 client.17895 fetch_fsmap learned FSMap version 25 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f4518045040 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f45057fa640 1 -- 172.21.15.44:0/2822173896 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f451407c020 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.406+0000 7f45057fa640 1 -- 172.21.15.44:0/2822173896 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4514020070 con 0x559dc1addc50 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f452a9c8740 10 client.17895 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f452a9c8740 10 client.17895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f452a9c8740 20 client.17895 populate_metadata read hostname 'smithi044' 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f452a9c8740 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1b4f820 0x559dc1b51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f452a9c8740 4 client.17895 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:33:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559dc1746600 con 0x559dc1b4f820 2024-09-13T17:33:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f4525a3f640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1b4f820 0x559dc1b51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:33:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f4525a3f640 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1b4f820 0x559dc1b51c40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9858fb6d929afec server_cookie=304a20c1e43172e9 in_seq=0 out_seq=0 2024-09-13T17:33:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.407+0000 7f45057fa640 10 client.17895 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f45057fa640 1 -- 172.21.15.44:0/2822173896 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559dc1746600 con 0x559dc1b4f820 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f45057fa640 10 client.17895 handle_command_reply: tid=0 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 client.17895 shutdown 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 msgr2=0x559dc1aeb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 0x559dc1aeb830 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4514005fd0 tx=0x7f451401b000 comp rx=0 tx=0).stop 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 shutdown_connections 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559dc1b4f820 0x559dc1b51c40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559dc1ae84f0 0x559dc1aebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc1addc50 0x559dc1aeb830 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 --2- 172.21.15.44:0/2822173896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559dc1add330 0x559dc1aef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f45057fa640 0 client.17895 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 >> 172.21.15.44:0/2822173896 conn(0x559dc1ad98c0 msgr2=0x559dc1ad4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 shutdown_connections 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 1 -- 172.21.15.44:0/2822173896 wait complete. 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.421+0000 7f452a9c8740 20 client.17895 trim_cache size 0 max 16384 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:33:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45080a1200 msgr2=0x7f45080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45080a1200 0x7f45080a36c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4518005ab0 tx=0x7f4518005a20 comp rx=0 tx=0).stop 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 msgr2=0x7f45201966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f45201966e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f451400c890 tx=0x7f451400cd50 comp rx=0 tx=0).stop 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 shutdown_connections 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45080a1200 0x7f45080a36c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4520104c30 0x7f452019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45200ff970 0x7f45201966e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 --2- 172.21.15.44:0/996845843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45200feff0 0x7f45201961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:33:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 >> 172.21.15.44:0/996845843 conn(0x7f4520074940 msgr2=0x7f45200fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:33:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 shutdown_connections 2024-09-13T17:33:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:33:53.422+0000 7f4527ca3640 1 -- 172.21.15.44:0/996845843 wait complete. 2024-09-13T17:33:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:53 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:33:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:53 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:53 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:33:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:53 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:53.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:53 smithi044 ceph-mon[31175]: pgmap v220: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:33:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:53 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:33:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:53 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:53 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:33:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:53 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:53 smithi070 ceph-mon[34502]: pgmap v220: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:33:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:53 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:33:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:53 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:53 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:33:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:53 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:33:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:53 smithi100 ceph-mon[35055]: pgmap v220: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:33:54.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:54 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:33:54.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:54 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:33:55.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:54 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:33:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:55 smithi044 ceph-mon[31175]: pgmap v221: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 22 op/s 2024-09-13T17:33:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:55 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:33:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:55 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:33:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:55 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:33:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:55 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:33:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:55 smithi070 ceph-mon[34502]: pgmap v221: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 22 op/s 2024-09-13T17:33:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:55 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:33:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:55 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:33:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:55 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:33:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:55 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:33:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:55 smithi100 ceph-mon[35055]: pgmap v221: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 22 op/s 2024-09-13T17:33:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:55 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:33:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:55 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:33:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:55 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:33:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:55 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:33:57.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:57 smithi044 ceph-mon[31175]: pgmap v222: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:57.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:57 smithi070 ceph-mon[34502]: pgmap v222: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:57 smithi100 ceph-mon[35055]: pgmap v222: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:33:59.827 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:33:59 smithi100 ceph-mon[35055]: pgmap v223: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:33:59.867 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:33:59 smithi044 ceph-mon[31175]: pgmap v223: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:33:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:33:59 smithi070 ceph-mon[34502]: pgmap v223: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:34:01.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:01 smithi044 ceph-mon[31175]: pgmap v224: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:01 smithi070 ceph-mon[34502]: pgmap v224: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:02.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:01 smithi100 ceph-mon[35055]: pgmap v224: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:03.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:03 smithi044 ceph-mon[31175]: pgmap v225: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:03 smithi070 ceph-mon[34502]: pgmap v225: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:04.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:03 smithi100 ceph-mon[35055]: pgmap v225: 129 pgs: 129 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:05 smithi044 ceph-mon[31175]: pgmap v226: 129 pgs: 129 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-13T17:34:05.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:05 smithi044 ceph-mon[31175]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T17:34:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:05 smithi070 ceph-mon[34502]: pgmap v226: 129 pgs: 129 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-13T17:34:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:05 smithi070 ceph-mon[34502]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T17:34:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:05 smithi100 ceph-mon[35055]: pgmap v226: 129 pgs: 129 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-13T17:34:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:05 smithi100 ceph-mon[35055]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.522+0000 7f5f12b84640 1 -- 172.21.15.44:0/828577854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c103f50 msgr2=0x7f5f0c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.522+0000 7f5f12b84640 1 --2- 172.21.15.44:0/828577854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c103f50 0x7f5f0c10baa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009cd0 tx=0x7f5f0402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 -- 172.21.15.44:0/828577854 shutdown_connections 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 --2- 172.21.15.44:0/828577854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c103f50 0x7f5f0c10baa0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 --2- 172.21.15.44:0/828577854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0ff7b0 0x7f5f0c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 --2- 172.21.15.44:0/828577854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c0fee00 0x7f5f0c0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 -- 172.21.15.44:0/828577854 >> 172.21.15.44:0/828577854 conn(0x7f5f0c0fa8d0 msgr2=0x7f5f0c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 -- 172.21.15.44:0/828577854 shutdown_connections 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 -- 172.21.15.44:0/828577854 wait complete. 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 Processor -- start 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.523+0000 7f5f12b84640 1 -- start start 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f12b84640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0fee00 0x7f5f0c196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f12b84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c103f50 0x7f5f0c196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f12b84640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 0x7f5f0c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f12b84640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c19b440 con 0x7f5f0c19aa50 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f12b84640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c19b5b0 con 0x7f5f0c0fee00 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f12b84640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c19b720 con 0x7f5f0c103f50 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 0x7f5f0c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 0x7f5f0c19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36588/0 (socket says 172.21.15.44:36588) 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 -- 172.21.15.44:0/2704473 learned_addr learned my addr 172.21.15.44:0/2704473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c103f50 0x7f5f0c196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f10920640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0fee00 0x7f5f0c196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 -- 172.21.15.44:0/2704473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c103f50 msgr2=0x7f5f0c196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c103f50 0x7f5f0c196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 -- 172.21.15.44:0/2704473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0fee00 msgr2=0x7f5f0c196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0fee00 0x7f5f0c196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 -- 172.21.15.44:0/2704473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f04009900 con 0x7f5f0c19aa50 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f10920640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0fee00 0x7f5f0c196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.524+0000 7f5f11121640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 0x7f5f0c19af00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5f04004af0 tx=0x7f5f04030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.525+0000 7f5f09ffb640 1 -- 172.21.15.44:0/2704473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04031b80 con 0x7f5f0c19aa50 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.525+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c1a2d00 con 0x7f5f0c19aa50 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.525+0000 7f5f09ffb640 1 -- 172.21.15.44:0/2704473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f04003e00 con 0x7f5f0c19aa50 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.525+0000 7f5f09ffb640 1 -- 172.21.15.44:0/2704473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04045410 con 0x7f5f0c19aa50 2024-09-13T17:34:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.525+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c1a3180 con 0x7f5f0c19aa50 2024-09-13T17:34:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f09ffb640 1 -- 172.21.15.44:0/2704473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5f0400a040 con 0x7f5f0c19aa50 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 Processor -- start 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 -- start start 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c36710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c40fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542ba1c30 con 0x55e542c36330 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542b9f3c0 con 0x55e542c414f0 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.528+0000 7f5f158a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542b9acb0 con 0x55e542c36c50 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c36710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c36710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36592/0 (socket says 172.21.15.44:36592) 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 -- 172.21.15.44:0/3516434269 learned_addr learned my addr 172.21.15.44:0/3516434269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f10920640 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c40fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 -- 172.21.15.44:0/3516434269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 msgr2=0x55e542c40fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c40fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 -- 172.21.15.44:0/3516434269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 msgr2=0x55e542c438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 -- 172.21.15.44:0/3516434269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e542b41910 con 0x55e542c36330 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f09ffb640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5efc0a1200 0x7f5efc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f10920640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5efc0a1200 0x7f5efc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c36710 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b6c0 tx=0x7f5ef80127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfbe487a6c2bb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5f09ffb640 1 -- 172.21.15.44:0/2704473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f5f040ee520 con 0x7f5f0c19aa50 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5ef6ffd640 1 -- 172.21.15.44:0/3516434269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80120d0 con 0x55e542c36330 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5ef6ffd640 1 -- 172.21.15.44:0/3516434269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ef80044b0 con 0x55e542c36330 2024-09-13T17:34:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.529+0000 7f5ef6ffd640 1 -- 172.21.15.44:0/3516434269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8003e80 con 0x55e542c36330 2024-09-13T17:34:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 -- 172.21.15.44:0/3516434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 msgr2=0x55e542c36710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f10920640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5efc0a1200 0x7f5efc0a36c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55e542c323c0 tx=0x7f5f00015040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c36710 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b6c0 tx=0x7f5ef80127c0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 -- 172.21.15.44:0/3516434269 shutdown_connections 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c40fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3516434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c36710 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 -- 172.21.15.44:0/3516434269 >> 172.21.15.44:0/3516434269 conn(0x55e542c328c0 msgr2=0x55e542c32ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 -- 172.21.15.44:0/3516434269 shutdown_connections 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 -- 172.21.15.44:0/3516434269 wait complete. 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.530+0000 7f5f158a3740 1 -- start start 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 Processor -- start 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c40e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c3e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c3e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542b58e80 con 0x55e542c414f0 2024-09-13T17:34:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542b58b20 con 0x55e542c36c50 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542b1e260 con 0x55e542c36330 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f158a3740 10 client.0 _ll_register_callbacks cb 0x55e542b53df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f11121640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c3e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f11121640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c3e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36594/0 (socket says 172.21.15.44:36594) 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f11121640 1 -- 172.21.15.44:0/3828920088 learned_addr learned my addr 172.21.15.44:0/3828920088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f10920640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c40e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.531+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c3e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f11121640 1 -- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 msgr2=0x55e542c40e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f11121640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c40e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f11121640 1 -- 172.21.15.44:0/3828920088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 msgr2=0x55e542c3e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f11121640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c3e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f11121640 1 -- 172.21.15.44:0/3828920088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef800b960 con 0x55e542c414f0 2024-09-13T17:34:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f11121640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c3e650 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5f040b7700 tx=0x7f5f040bb900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59a154bf7736726 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5eebfff640 1 -- 172.21.15.44:0/3828920088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f040d1070 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5eebfff640 1 -- 172.21.15.44:0/3828920088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f5f040bf030 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e542bc3680 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5eebfff640 1 -- 172.21.15.44:0/3828920088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f040c3680 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f158a3740 10 client.17907 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e542b99330 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.532+0000 7f5eebfff640 1 -- 172.21.15.44:0/3828920088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f040a1080 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5eebfff640 1 -- 172.21.15.44:0/3828920088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f040be030 con 0x55e542c414f0 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f158a3740 10 client.17907 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f158a3740 10 client.17907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f158a3740 20 client.17907 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542c50c30 0x55e542c53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f158a3740 4 client.17907 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e542b8e5d0 con 0x55e542c50c30 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f10920640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542c50c30 0x55e542c53020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5eebfff640 10 client.17907 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.533+0000 7f5f10920640 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542c50c30 0x55e542c53020 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76b3ec826242cea server_cookie=acbaad0874841c53 in_seq=0 out_seq=0 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5eebfff640 1 -- 172.21.15.44:0/3828920088 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e542b8e5d0 con 0x55e542c50c30 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5eebfff640 10 client.17907 handle_command_reply: tid=0 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 client.17907 shutdown 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 msgr2=0x55e542c3e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c3e650 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5f040b7700 tx=0x7f5f040bb900 comp rx=0 tx=0).stop 2024-09-13T17:34:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 shutdown_connections 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542c50c30 0x55e542c53020 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c3e650 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c3e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 --2- 172.21.15.44:0/3828920088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c40e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5eebfff640 0 client.17907 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 >> 172.21.15.44:0/3828920088 conn(0x55e542c328c0 msgr2=0x55e542c31b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 shutdown_connections 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.534+0000 7f5f158a3740 1 -- 172.21.15.44:0/3828920088 wait complete. 2024-09-13T17:34:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.535+0000 7f5f158a3740 20 client.17907 trim_cache size 0 max 16384 2024-09-13T17:34:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.547+0000 7f5f158a3740 1 Processor -- start 2024-09-13T17:34:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.547+0000 7f5f158a3740 1 -- start start 2024-09-13T17:34:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.547+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c3c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f158a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542c43fd0 con 0x55e542c36330 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f158a3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542c3df40 con 0x55e542c414f0 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f158a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542c533a0 con 0x55e542c36c50 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c3e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36610/0 (socket says 172.21.15.44:36610) 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 -- 172.21.15.44:0/946216586 learned_addr learned my addr 172.21.15.44:0/946216586 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f11121640 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c3c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 -- 172.21.15.44:0/946216586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 msgr2=0x55e542c3e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c3e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 -- 172.21.15.44:0/946216586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 msgr2=0x55e542c3c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c3c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 -- 172.21.15.44:0/946216586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e542b3b300 con 0x55e542c36330 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f11121640 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c3c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5f10920640 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c3e0d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55e542c538d0 tx=0x7f5f0001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81400e7f23f17dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5ef77fe640 1 -- 172.21.15.44:0/946216586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00030180 con 0x55e542c36330 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5ef77fe640 1 -- 172.21.15.44:0/946216586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f00019040 con 0x55e542c36330 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.548+0000 7f5ef77fe640 1 -- 172.21.15.44:0/946216586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0002dbd0 con 0x55e542c36330 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 -- 172.21.15.44:0/946216586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 msgr2=0x55e542c3e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c3e0d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55e542c538d0 tx=0x7f5f0001b000 comp rx=0 tx=0).stop 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 -- 172.21.15.44:0/946216586 shutdown_connections 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36c50 0x55e542c3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c414f0 0x55e542c3c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 --2- 172.21.15.44:0/946216586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c36330 0x55e542c3e0d0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 -- 172.21.15.44:0/946216586 >> 172.21.15.44:0/946216586 conn(0x55e542c328c0 msgr2=0x55e542c39470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 -- 172.21.15.44:0/946216586 shutdown_connections 2024-09-13T17:34:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 -- 172.21.15.44:0/946216586 wait complete. 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 -- start start 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.549+0000 7f5f158a3740 1 Processor -- start 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c481f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542c46850 con 0x55e542c414f0 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542c469c0 con 0x55e542c36c50 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e542c46b80 con 0x55e542c36330 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f158a3740 10 client.0 _ll_register_callbacks cb 0x7f5ef800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36614/0 (socket says 172.21.15.44:36614) 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 -- 172.21.15.44:0/1235252022 learned_addr learned my addr 172.21.15.44:0/1235252022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f10920640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 -- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 msgr2=0x55e542c481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 -- 172.21.15.44:0/1235252022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 msgr2=0x55e542c48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 -- 172.21.15.44:0/1235252022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f00020420 con 0x55e542c414f0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f0bfff640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f10920640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5f11121640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c461d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5f04006010 tx=0x7f5f040040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b1ac855ca5ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.550+0000 7f5eebfff640 1 -- 172.21.15.44:0/1235252022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f040d1070 con 0x55e542c414f0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5eebfff640 1 -- 172.21.15.44:0/1235252022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f5f040c3400 con 0x55e542c414f0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ef80044b0 con 0x55e542c414f0 2024-09-13T17:34:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5eebfff640 1 -- 172.21.15.44:0/1235252022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f040ee330 con 0x55e542c414f0 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 10 client.17913 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e542b608e0 con 0x55e542c414f0 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5eebfff640 1 -- 172.21.15.44:0/1235252022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ef80044b0 con 0x55e542c414f0 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5eebfff640 1 -- 172.21.15.44:0/1235252022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f040be070 con 0x55e542c414f0 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 10 client.17913 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 10 client.17913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 20 client.17913 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542ca8d00 0x55e542cab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 4 client.17913 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.551+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e54288e300 con 0x55e542ca8d00 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.552+0000 7f5f10920640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542ca8d00 0x55e542cab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.552+0000 7f5f10920640 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542ca8d00 0x55e542cab0f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fefd9e5f7db739 server_cookie=fcf4737e35c6f3ac in_seq=0 out_seq=0 2024-09-13T17:34:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.552+0000 7f5eebfff640 10 client.17913 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5eebfff640 1 -- 172.21.15.44:0/1235252022 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e54288e300 con 0x55e542ca8d00 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5eebfff640 10 client.17913 handle_command_reply: tid=0 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 client.17913 shutdown 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 msgr2=0x55e542c461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c461d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5f04006010 tx=0x7f5f040040b0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 shutdown_connections 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e542ca8d00 0x55e542cab0f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e542c414f0 0x55e542c461d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e542c36c50 0x55e542c48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 --2- 172.21.15.44:0/1235252022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e542c36330 0x55e542c481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5eebfff640 0 client.17913 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.566+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 >> 172.21.15.44:0/1235252022 conn(0x55e542c328c0 msgr2=0x55e542c3e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 shutdown_connections 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f158a3740 1 -- 172.21.15.44:0/1235252022 wait complete. 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f158a3740 20 client.17913 trim_cache size 0 max 16384 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:34:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:34:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5efc0a1200 msgr2=0x7f5efc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5efc0a1200 0x7f5efc0a36c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55e542c323c0 tx=0x7f5f00015040 comp rx=0 tx=0).stop 2024-09-13T17:34:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 msgr2=0x7f5f0c19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 0x7f5f0c19af00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5f04004af0 tx=0x7f5f04030e50 comp rx=0 tx=0).stop 2024-09-13T17:34:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 shutdown_connections 2024-09-13T17:34:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5efc0a1200 0x7f5efc0a36c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c103f50 0x7f5f0c196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c19aa50 0x7f5f0c19af00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 --2- 172.21.15.44:0/2704473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c0fee00 0x7f5f0c196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 >> 172.21.15.44:0/2704473 conn(0x7f5f0c0fa8d0 msgr2=0x7f5f0c068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 shutdown_connections 2024-09-13T17:34:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:06.567+0000 7f5f12b84640 1 -- 172.21.15.44:0/2704473 wait complete. 2024-09-13T17:34:07.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:07 smithi044 ceph-mon[31175]: pgmap v228: 129 pgs: 129 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 95 KiB/s wr, 32 op/s 2024-09-13T17:34:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:07 smithi070 ceph-mon[34502]: pgmap v228: 129 pgs: 129 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 95 KiB/s wr, 32 op/s 2024-09-13T17:34:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:07 smithi100 ceph-mon[35055]: pgmap v228: 129 pgs: 129 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 95 KiB/s wr, 32 op/s 2024-09-13T17:34:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:09 smithi044 ceph-mon[31175]: pgmap v229: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-09-13T17:34:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:09 smithi070 ceph-mon[34502]: pgmap v229: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-09-13T17:34:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:09 smithi100 ceph-mon[35055]: pgmap v229: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-09-13T17:34:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:12 smithi100 ceph-mon[35055]: pgmap v230: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-09-13T17:34:12.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:12 smithi100 ceph-mon[35055]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T17:34:12.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:12 smithi044 ceph-mon[31175]: pgmap v230: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-09-13T17:34:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:12 smithi044 ceph-mon[31175]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T17:34:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:12 smithi070 ceph-mon[34502]: pgmap v230: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 35 op/s 2024-09-13T17:34:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:12 smithi070 ceph-mon[34502]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:313/600: open fd = 3 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:314/600: open fd = 3 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:315/600: open fd = 3 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:316/600: open fd = 3 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:317/600: open fd = 3 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:318/600: open fd = 3 2024-09-13T17:34:13.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:319/600: open fd = 3 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:320/600: open fd = 3 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:321/600: open fd = 3 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:322/600: open fd = 3 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:323/600: open fd = 3 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:324/600: open fd = 3 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.152 INFO:tasks.workunit.client.0.smithi044.stdout:325/600: open fd = 3 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:326/600: open fd = 3 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:327/600: open fd = 3 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:328/600: open fd = 3 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:329/600: open fd = 3 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:330/600: open fd = 3 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.154 INFO:tasks.workunit.client.0.smithi044.stdout:331/600: open fd = 3 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:332/600: open fd = 3 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:333/600: open fd = 3 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:334/600: open fd = 3 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:335/600: open fd = 3 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:336/600: open fd = 3 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:337/600: open fd = 3 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.156 INFO:tasks.workunit.client.0.smithi044.stdout:338/600: open fd = 3 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:339/600: open fd = 3 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:340/600: open fd = 3 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:341/600: open fd = 3 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.157 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:342/600: open fd = 3 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:343/600: open fd = 3 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:344/600: open fd = 3 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:345/600: open fd = 3 2024-09-13T17:34:13.158 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:346/600: open fd = 3 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:347/600: open fd = 3 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:348/600: open fd = 3 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.159 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:349/600: open fd = 3 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:350/600: open fd = 3 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:351/600: open fd = 3 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:352/600: open fd = 3 2024-09-13T17:34:13.160 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.161 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.161 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.161 INFO:tasks.workunit.client.0.smithi044.stdout:353/600: open fd = 3 2024-09-13T17:34:13.161 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.161 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:354/600: open fd = 3 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:355/600: open fd = 3 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:356/600: open fd = 3 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:357/600: open fd = 3 2024-09-13T17:34:13.162 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:358/600: open fd = 3 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:359/600: open fd = 3 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:360/600: open fd = 3 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.163 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:361/600: open fd = 3 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:362/600: open fd = 3 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:363/600: open fd = 3 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.164 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:364/600: open fd = 3 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:365/600: open fd = 3 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.165 INFO:tasks.workunit.client.0.smithi044.stdout:366/600: open fd = 3 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:367/600: open fd = 3 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:368/600: open fd = 3 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:369/600: open fd = 3 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.166 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:370/600: open fd = 3 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:371/600: open fd = 3 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:372/600: open fd = 3 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.167 INFO:tasks.workunit.client.0.smithi044.stdout:373/600: open fd = 3 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:374/600: open fd = 3 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:34:13.168 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:34:14.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:14 smithi100 ceph-mon[35055]: pgmap v232: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-13T17:34:14.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:14 smithi044 ceph-mon[31175]: pgmap v232: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-13T17:34:14.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:14 smithi070 ceph-mon[34502]: pgmap v232: 129 pgs: 129 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-13T17:34:16.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:16 smithi100 ceph-mon[35055]: pgmap v233: 129 pgs: 129 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:34:16.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:16 smithi044 ceph-mon[31175]: pgmap v233: 129 pgs: 129 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:34:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:16 smithi070 ceph-mon[34502]: pgmap v233: 129 pgs: 129 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:34:18.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:18 smithi100 ceph-mon[35055]: pgmap v234: 129 pgs: 129 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:34:18.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:18 smithi044 ceph-mon[31175]: pgmap v234: 129 pgs: 129 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:34:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:18 smithi070 ceph-mon[34502]: pgmap v234: 129 pgs: 129 active+clean; 5.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 -- 172.21.15.44:0/2477857551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd4100ec0 msgr2=0x7fbfd41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/2477857551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd4100ec0 0x7fbfd41012a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402a480 comp rx=0 tx=0).stop 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 -- 172.21.15.44:0/2477857551 shutdown_connections 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/2477857551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd410a290 0x7fbfd410c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/2477857551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd41017e0 0x7fbfd4109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/2477857551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd4100ec0 0x7fbfd41012a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 -- 172.21.15.44:0/2477857551 >> 172.21.15.44:0/2477857551 conn(0x7fbfd40fa890 msgr2=0x7fbfd40fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 -- 172.21.15.44:0/2477857551 shutdown_connections 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.668+0000 7fbfda9ae640 1 -- 172.21.15.44:0/2477857551 wait complete. 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 Processor -- start 2024-09-13T17:34:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 -- start start 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd4100ec0 0x7fbfd4196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 0x7fbfd4196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd410a290 0x7fbfd419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd419b050 con 0x7fbfd41017e0 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd419b1c0 con 0x7fbfd4100ec0 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfda9ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd419b380 con 0x7fbfd410a290 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.669+0000 7fbfcbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 0x7fbfd4196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 0x7fbfd4196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36096/0 (socket says 172.21.15.44:36096) 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1809622511 learned_addr learned my addr 172.21.15.44:0/1809622511 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd410a290 0x7fbfd419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd4100ec0 0x7fbfd4196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1809622511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd410a290 msgr2=0x7fbfd419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd410a290 0x7fbfd419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1809622511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd4100ec0 msgr2=0x7fbfd4196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd4100ec0 0x7fbfd4196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1809622511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009560 con 0x7fbfd41017e0 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd4100ec0 0x7fbfd4196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 0x7fbfd4196660 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00e9f0 tx=0x7fbfbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfd1ffb640 1 -- 172.21.15.44:0/1809622511 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc009700 con 0x7fbfd41017e0 2024-09-13T17:34:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfd1ffb640 1 -- 172.21.15.44:0/1809622511 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc00cd80 con 0x7fbfd41017e0 2024-09-13T17:34:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd419b660 con 0x7fbfd41017e0 2024-09-13T17:34:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfd1ffb640 1 -- 172.21.15.44:0/1809622511 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0187e0 con 0x7fbfd41017e0 2024-09-13T17:34:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.670+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd41a2ee0 con 0x7fbfd41017e0 2024-09-13T17:34:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfd1ffb640 1 -- 172.21.15.44:0/1809622511 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbfbc010430 con 0x7fbfd41017e0 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 Processor -- start 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 -- start start 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389d95610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389d00c30 con 0x560389da4900 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389cfe3c0 con 0x560389d95b50 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389cf9cb0 con 0x560389d95230 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfcbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfcbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36098/0 (socket says 172.21.15.44:36098) 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfcbfff640 1 -- 172.21.15.44:0/4085555997 learned_addr learned my addr 172.21.15.44:0/4085555997 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.673+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389d95610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfcbfff640 1 -- 172.21.15.44:0/4085555997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 msgr2=0x560389d95610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389d95610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfcbfff640 1 -- 172.21.15.44:0/4085555997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 msgr2=0x560389da43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfcbfff640 1 -- 172.21.15.44:0/4085555997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560389ca0910 con 0x560389da4900 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfd1ffb640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc00a1200 0x7fbfc00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc00a1200 0x7fbfc00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da6cf0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0ca750 tx=0x7fbfbc0d5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b7e9ccf7d0ea05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfd1ffb640 1 -- 172.21.15.44:0/1809622511 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fbfbc014070 con 0x7fbfd41017e0 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfca7fc640 1 -- 172.21.15.44:0/4085555997 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0964e0 con 0x560389da4900 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfca7fc640 1 -- 172.21.15.44:0/4085555997 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc099f70 con 0x560389da4900 2024-09-13T17:34:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.674+0000 7fbfca7fc640 1 -- 172.21.15.44:0/4085555997 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc09a8a0 con 0x560389da4900 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/4085555997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 msgr2=0x560389da6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc00a1200 0x7fbfc00a36c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005fd0 tx=0x7fbfcc0096a0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da6cf0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0ca750 tx=0x7fbfbc0d5780 comp rx=0 tx=0).stop 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/4085555997 shutdown_connections 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da6cf0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/4085555997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389d95610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/4085555997 >> 172.21.15.44:0/4085555997 conn(0x560389d918c0 msgr2=0x560389d91ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/4085555997 shutdown_connections 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/4085555997 wait complete. 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.675+0000 7fbfdd6d1740 1 -- start start 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 Processor -- start 2024-09-13T17:34:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 0x560389da4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389cb7e80 con 0x560389d95230 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389cb7b20 con 0x560389d95b50 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389c7d260 con 0x560389da4900 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfdd6d1740 10 client.0 _ll_register_callbacks cb 0x560389cb2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 0x560389da4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 0x560389da4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36112/0 (socket says 172.21.15.44:36112) 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfd3fff640 1 -- 172.21.15.44:0/182622140 learned_addr learned my addr 172.21.15.44:0/182622140 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.676+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd3fff640 1 -- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 msgr2=0x560389da1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd3fff640 1 -- 172.21.15.44:0/182622140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 msgr2=0x560389da15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd3fff640 1 -- 172.21.15.44:0/182622140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd41a2ee0 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 0x560389da4320 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fbfc402a9a0 tx=0x7fbfc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d62c2ef08624ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfb37fe640 1 -- 172.21.15.44:0/182622140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4031920 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560389d22680 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfb37fe640 1 -- 172.21.15.44:0/182622140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fbfc4031a80 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfb37fe640 1 -- 172.21.15.44:0/182622140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc4031c40 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfdd6d1740 10 client.17925 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560389cf8330 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.677+0000 7fbfb37fe640 1 -- 172.21.15.44:0/182622140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560389d22680 con 0x560389d95230 2024-09-13T17:34:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfb37fe640 1 -- 172.21.15.44:0/182622140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbfc4046470 con 0x560389d95230 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfdd6d1740 10 client.17925 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfdd6d1740 10 client.17925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfdd6d1740 20 client.17925 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389db4360 0x560389db6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfdd6d1740 4 client.17925 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560389ced5d0 con 0x560389db4360 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389db4360 0x560389db6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389db4360 0x560389db6750 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fdf152f7e68a69 server_cookie=abcf209ba791bcdf in_seq=0 out_seq=0 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.678+0000 7fbfb37fe640 10 client.17925 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfb37fe640 1 -- 172.21.15.44:0/182622140 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560389ced5d0 con 0x560389db4360 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfb37fe640 10 client.17925 handle_command_reply: tid=0 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 client.17925 shutdown 2024-09-13T17:34:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 msgr2=0x560389da4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 0x560389da4320 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fbfc402a9a0 tx=0x7fbfc4004270 comp rx=0 tx=0).stop 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 shutdown_connections 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389db4360 0x560389db6750 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389da15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/182622140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95230 0x560389da4320 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfb37fe640 0 client.17925 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 >> 172.21.15.44:0/182622140 conn(0x560389d918c0 msgr2=0x560389d975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 shutdown_connections 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/182622140 wait complete. 2024-09-13T17:34:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.679+0000 7fbfdd6d1740 20 client.17925 trim_cache size 0 max 16384 2024-09-13T17:34:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 Processor -- start 2024-09-13T17:34:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 -- start start 2024-09-13T17:34:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389da3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389d9fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389d226b0 con 0x560389da4900 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389db2c60 con 0x560389d95b50 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389da73b0 con 0x560389d95230 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfd8f4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfd8f4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36118/0 (socket says 172.21.15.44:36118) 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.692+0000 7fbfd8f4b640 1 -- 172.21.15.44:0/3008655813 learned_addr learned my addr 172.21.15.44:0/3008655813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389da3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389d9fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd8f4b640 1 -- 172.21.15.44:0/3008655813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 msgr2=0x560389da3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389da3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd8f4b640 1 -- 172.21.15.44:0/3008655813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 msgr2=0x560389d9fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389d9fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd8f4b640 1 -- 172.21.15.44:0/3008655813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560389c9a300 con 0x560389da4900 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389d9fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da03b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc010700 tx=0x7fbfcc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ba9a4bf0e94a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfcaffd640 1 -- 172.21.15.44:0/3008655813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc004b20 con 0x560389da4900 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfcaffd640 1 -- 172.21.15.44:0/3008655813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfcc004c80 con 0x560389da4900 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.693+0000 7fbfcaffd640 1 -- 172.21.15.44:0/3008655813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc01e630 con 0x560389da4900 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/3008655813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 msgr2=0x560389da03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da03b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc010700 tx=0x7fbfcc015000 comp rx=0 tx=0).stop 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/3008655813 shutdown_connections 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389da4900 0x560389da03b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95b50 0x560389d9fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/3008655813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389d95230 0x560389da3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/3008655813 >> 172.21.15.44:0/3008655813 conn(0x560389d918c0 msgr2=0x560389d8ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/3008655813 shutdown_connections 2024-09-13T17:34:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/3008655813 wait complete. 2024-09-13T17:34:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 -- start start 2024-09-13T17:34:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.694+0000 7fbfdd6d1740 1 Processor -- start 2024-09-13T17:34:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95230 0x560389dabad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 0x560389da7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389da8b10 con 0x560389d95b50 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389da8c80 con 0x560389d95230 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560389e0a860 con 0x560389da4900 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 10 client.0 _ll_register_callbacks cb 0x7fbfbc0d0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 0x560389da7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 0x560389da7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36130/0 (socket says 172.21.15.44:36130) 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1860020383 learned_addr learned my addr 172.21.15.44:0/1860020383 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95230 0x560389dabad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 msgr2=0x560389da8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1860020383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95230 msgr2=0x560389dabad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95230 0x560389dabad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 -- 172.21.15.44:0/1860020383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc018070 con 0x560389d95b50 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfd8f4b640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95230 0x560389dabad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfcbfff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 0x560389da7e00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc099ea0 tx=0x7fbfbc00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f5f8accf5b1f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfb37fe640 1 -- 172.21.15.44:0/1860020383 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0d0860 con 0x560389d95b50 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfb37fe640 1 -- 172.21.15.44:0/1860020383 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fbfbc09a8a0 con 0x560389d95b50 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.695+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfbc099f70 con 0x560389d95b50 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfb37fe640 1 -- 172.21.15.44:0/1860020383 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfbc099ce0 con 0x560389d95b50 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 10 client.17931 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560389cbf8e0 con 0x560389d95b50 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfb37fe640 1 -- 172.21.15.44:0/1860020383 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc099f70 con 0x560389d95b50 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfb37fe640 1 -- 172.21.15.44:0/1860020383 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbfbc0caa60 con 0x560389d95b50 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 10 client.17931 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 20 client.17931 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389e0bbf0 0x560389e0e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 4 client.17931 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603899ed300 con 0x560389e0bbf0 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.696+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389e0bbf0 0x560389e0e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.697+0000 7fbfb37fe640 10 client.17931 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.697+0000 7fbfd3fff640 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389e0bbf0 0x560389e0e010 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0df6931738358c server_cookie=c790da694fa63a8b in_seq=0 out_seq=0 2024-09-13T17:34:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfb37fe640 1 -- 172.21.15.44:0/1860020383 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603899ed300 con 0x560389e0bbf0 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfb37fe640 10 client.17931 handle_command_reply: tid=0 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 client.17931 shutdown 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 msgr2=0x560389da7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 0x560389da7e00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc099ea0 tx=0x7fbfbc00cef0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 shutdown_connections 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560389e0bbf0 0x560389e0e010 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560389da4900 0x560389da8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560389d95b50 0x560389da7e00 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfb37fe640 0 client.17931 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 --2- 172.21.15.44:0/1860020383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560389d95230 0x560389dabad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 >> 172.21.15.44:0/1860020383 conn(0x560389d918c0 msgr2=0x560389d8ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 shutdown_connections 2024-09-13T17:34:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 1 -- 172.21.15.44:0/1860020383 wait complete. 2024-09-13T17:34:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.709+0000 7fbfdd6d1740 20 client.17931 trim_cache size 0 max 16384 2024-09-13T17:34:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:34:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:34:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:34:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc00a1200 msgr2=0x7fbfc00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc00a1200 0x7fbfc00a36c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005fd0 tx=0x7fbfcc0096a0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 msgr2=0x7fbfd4196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 0x7fbfd4196660 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00e9f0 tx=0x7fbfbc00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 shutdown_connections 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fbfc00a1200 0x7fbfc00a36c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfd410a290 0x7fbfd419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd41017e0 0x7fbfd4196660 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 --2- 172.21.15.44:0/1809622511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd4100ec0 0x7fbfd4196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 >> 172.21.15.44:0/1809622511 conn(0x7fbfd40fa890 msgr2=0x7fbfd40fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 shutdown_connections 2024-09-13T17:34:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:19.710+0000 7fbfda9ae640 1 -- 172.21.15.44:0/1809622511 wait complete. 2024-09-13T17:34:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:20 smithi044 ceph-mon[31175]: pgmap v235: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-13T17:34:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:20 smithi070 ceph-mon[34502]: pgmap v235: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-13T17:34:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:20 smithi100 ceph-mon[35055]: pgmap v235: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-13T17:34:22.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:34:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 -- 172.21.15.44:0/3694083167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974100e50 msgr2=0x7fc974101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 --2- 172.21.15.44:0/3694083167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974100e50 0x7fc974101230 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc968009900 tx=0x7fc96802a410 comp rx=0 tx=0).stop 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 -- 172.21.15.44:0/3694083167 shutdown_connections 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 --2- 172.21.15.44:0/3694083167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc97410a350 0x7fc97410c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 --2- 172.21.15.44:0/3694083167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974101800 0x7fc974109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 --2- 172.21.15.44:0/3694083167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974100e50 0x7fc974101230 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 -- 172.21.15.44:0/3694083167 >> 172.21.15.44:0/3694083167 conn(0x7fc9740fa930 msgr2=0x7fc9740fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 -- 172.21.15.44:0/3694083167 shutdown_connections 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.375+0000 7fc97af25640 1 -- 172.21.15.44:0/3694083167 wait complete. 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 Processor -- start 2024-09-13T17:34:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 -- start start 2024-09-13T17:34:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974100e50 0x7fc974196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 0x7fc974196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97410a350 0x7fc97419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419b080 con 0x7fc974101800 2024-09-13T17:34:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419b1f0 con 0x7fc97410a350 2024-09-13T17:34:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc97af25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419b3b0 con 0x7fc974100e50 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 0x7fc974196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc973fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 0x7fc974196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36154/0 (socket says 172.21.15.44:36154) 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc973fff640 1 -- 172.21.15.44:0/840200963 learned_addr learned my addr 172.21.15.44:0/840200963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc978cc1640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974100e50 0x7fc974196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.376+0000 7fc9794c2640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97410a350 0x7fc97419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc973fff640 1 -- 172.21.15.44:0/840200963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974100e50 msgr2=0x7fc974196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc973fff640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974100e50 0x7fc974196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc973fff640 1 -- 172.21.15.44:0/840200963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97410a350 msgr2=0x7fc97419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc973fff640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97410a350 0x7fc97419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc973fff640 1 -- 172.21.15.44:0/840200963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968009560 con 0x7fc974101800 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc973fff640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 0x7fc974196690 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc96000d950 tx=0x7fc96000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc971ffb640 1 -- 172.21.15.44:0/840200963 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960009c30 con 0x7fc974101800 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97419b690 con 0x7fc974101800 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc971ffb640 1 -- 172.21.15.44:0/840200963 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc960009d90 con 0x7fc974101800 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc971ffb640 1 -- 172.21.15.44:0/840200963 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000f620 con 0x7fc974101800 2024-09-13T17:34:22.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.377+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9741a2f10 con 0x7fc974101800 2024-09-13T17:34:22.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.379+0000 7fc971ffb640 1 -- 172.21.15.44:0/840200963 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc9600109f0 con 0x7fc974101800 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 Processor -- start 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 -- start start 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927ce10 0x55aac927d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9287a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc971ffb640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9640a1200 0x7fc9640a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac91e7f20 con 0x55aac927d730 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac91e56b0 con 0x55aac927ce10 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac91e0fa0 con 0x55aac9287fd0 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc978cc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9287a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc978cc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9287a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36170/0 (socket says 172.21.15.44:36170) 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc978cc1640 1 -- 172.21.15.44:0/3035034037 learned_addr learned my addr 172.21.15.44:0/3035034037 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.380+0000 7fc971ffb640 1 -- 172.21.15.44:0/840200963 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc9600ccc00 con 0x7fc974101800 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc973fff640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9640a1200 0x7fc9640a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc9794c2640 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927ce10 0x55aac927d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc9794c2640 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc978cc1640 1 -- 172.21.15.44:0/3035034037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 msgr2=0x55aac928a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc978cc1640 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc978cc1640 1 -- 172.21.15.44:0/3035034037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927ce10 msgr2=0x55aac927d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc978cc1640 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927ce10 0x55aac927d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc978cc1640 1 -- 172.21.15.44:0/3035034037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac919b840 con 0x55aac927d730 2024-09-13T17:34:22.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc973fff640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9640a1200 0x7fc9640a36c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc960098950 tx=0x7fc96006bad0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc978cc1640 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9287a90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc96802a930 tx=0x7fc968031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce52556dc5b1f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc95effd640 1 -- 172.21.15.44:0/3035034037 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968004300 con 0x55aac927d730 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc95effd640 1 -- 172.21.15.44:0/3035034037 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc968031df0 con 0x55aac927d730 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.381+0000 7fc95effd640 1 -- 172.21.15.44:0/3035034037 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968003cb0 con 0x55aac927d730 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 -- 172.21.15.44:0/3035034037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 msgr2=0x55aac9287a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9287a90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc96802a930 tx=0x7fc968031670 comp rx=0 tx=0).stop 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 -- 172.21.15.44:0/3035034037 shutdown_connections 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9287a90 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/3035034037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927ce10 0x55aac927d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 -- 172.21.15.44:0/3035034037 >> 172.21.15.44:0/3035034037 conn(0x55aac9279380 msgr2=0x55aac9279760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 -- 172.21.15.44:0/3035034037 shutdown_connections 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 -- 172.21.15.44:0/3035034037 wait complete. 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 -- start start 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.382+0000 7fc97dc4a740 1 Processor -- start 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac927ce10 0x55aac92879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9284ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac9287fd0 0x55aac92852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac91822a0 con 0x55aac927d730 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac8eb9430 con 0x55aac9287fd0 2024-09-13T17:34:22.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac9164680 con 0x55aac927ce10 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc97dc4a740 10 client.0 _ll_register_callbacks cb 0x55aac91da610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9284ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9284ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36176/0 (socket says 172.21.15.44:36176) 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 -- 172.21.15.44:0/1761189102 learned_addr learned my addr 172.21.15.44:0/1761189102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc9794c2640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac9287fd0 0x55aac92852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc9794c2640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac927ce10 0x55aac92879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 -- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac927ce10 msgr2=0x55aac92879f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac927ce10 0x55aac92879f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 -- 172.21.15.44:0/1761189102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac9287fd0 msgr2=0x55aac92852f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac9287fd0 0x55aac92852f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc978cc1640 1 -- 172.21.15.44:0/1761189102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97410d420 con 0x55aac927d730 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.383+0000 7fc9794c2640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac927ce10 0x55aac92879f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc978cc1640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9284ce0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc968000c00 tx=0x7fc968003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f252ead7fa283523 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc953fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc953fff640 1 -- 172.21.15.44:0/1761189102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96804b070 con 0x55aac927d730 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc953fff640 1 -- 172.21.15.44:0/1761189102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc968004100 con 0x55aac927d730 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aac8ed44e0 con 0x55aac927d730 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc953fff640 1 -- 172.21.15.44:0/1761189102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9680042e0 con 0x55aac927d730 2024-09-13T17:34:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 10 client.17943 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aac91df620 con 0x55aac927d730 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc953fff640 1 -- 172.21.15.44:0/1761189102 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aac8ed44e0 con 0x55aac927d730 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc953fff640 1 -- 172.21.15.44:0/1761189102 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9680468f0 con 0x55aac927d730 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 10 client.17943 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 20 client.17943 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:22.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac9297c20 0x55aac929a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 4 client.17943 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.384+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aac8ee57e0 con 0x55aac9297c20 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.385+0000 7fc9794c2640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac9297c20 0x55aac929a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.385+0000 7fc9794c2640 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac9297c20 0x55aac929a010 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a36bd62b6f5c29 server_cookie=37f811a59ebd870c in_seq=0 out_seq=0 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.385+0000 7fc953fff640 10 client.17943 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc953fff640 1 -- 172.21.15.44:0/1761189102 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55aac8ee57e0 con 0x55aac9297c20 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc953fff640 10 client.17943 handle_command_reply: tid=0 2024-09-13T17:34:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 client.17943 shutdown 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 msgr2=0x55aac9284ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9284ce0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc968000c00 tx=0x7fc968003850 comp rx=0 tx=0).stop 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 shutdown_connections 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac9297c20 0x55aac929a010 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac9287fd0 0x55aac92852f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927d730 0x55aac9284ce0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc953fff640 0 client.17943 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/1761189102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac927ce10 0x55aac92879f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 >> 172.21.15.44:0/1761189102 conn(0x55aac9279380 msgr2=0x55aac927f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 shutdown_connections 2024-09-13T17:34:22.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 1 -- 172.21.15.44:0/1761189102 wait complete. 2024-09-13T17:34:22.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.386+0000 7fc97dc4a740 20 client.17943 trim_cache size 0 max 16384 2024-09-13T17:34:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.400+0000 7fc97dc4a740 1 Processor -- start 2024-09-13T17:34:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.400+0000 7fc97dc4a740 1 -- start start 2024-09-13T17:34:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac9278e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac9284580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac9284ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac9297a40 con 0x55aac927ce10 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac927cb90 con 0x55aac927d730 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac927dfe0 con 0x55aac9287fd0 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac9278e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac9278e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36184/0 (socket says 172.21.15.44:36184) 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 -- 172.21.15.44:0/755657822 learned_addr learned my addr 172.21.15.44:0/755657822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc978cc1640 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac9284580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 -- 172.21.15.44:0/755657822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 msgr2=0x55aac9284ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac9284ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 -- 172.21.15.44:0/755657822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 msgr2=0x55aac9284580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac9284580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 -- 172.21.15.44:0/755657822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac9181790 con 0x55aac927ce10 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.401+0000 7fc9794c2640 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac9278e20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc96c000f80 tx=0x7fc96c00a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eff18e03f78d650 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc95f7fe640 1 -- 172.21.15.44:0/755657822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c019040 con 0x55aac927ce10 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc95f7fe640 1 -- 172.21.15.44:0/755657822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc96c013e90 con 0x55aac927ce10 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc95f7fe640 1 -- 172.21.15.44:0/755657822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c021380 con 0x55aac927ce10 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 -- 172.21.15.44:0/755657822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 msgr2=0x55aac9278e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac9278e20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc96c000f80 tx=0x7fc96c00a4d0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 -- 172.21.15.44:0/755657822 shutdown_connections 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac9284ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac9284580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/755657822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac9278e20 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 -- 172.21.15.44:0/755657822 >> 172.21.15.44:0/755657822 conn(0x55aac9279380 msgr2=0x55aac927ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 -- 172.21.15.44:0/755657822 shutdown_connections 2024-09-13T17:34:22.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 -- 172.21.15.44:0/755657822 wait complete. 2024-09-13T17:34:22.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.402+0000 7fc97dc4a740 1 -- start start 2024-09-13T17:34:22.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 Processor -- start 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac928edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac928b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac928bda0 con 0x55aac927ce10 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac928bf10 con 0x55aac927d730 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac928c0d0 con 0x55aac9287fd0 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc97dc4a740 10 client.0 _ll_register_callbacks cb 0x7fc96c00abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac928edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac928edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36200/0 (socket says 172.21.15.44:36200) 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 -- 172.21.15.44:0/2804402659 learned_addr learned my addr 172.21.15.44:0/2804402659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc978cc1640 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac928b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 -- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 msgr2=0x55aac928b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 -- 172.21.15.44:0/2804402659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 msgr2=0x55aac928b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac928b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.403+0000 7fc9794c2640 1 -- 172.21.15.44:0/2804402659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c01c040 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc9794c2640 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac928edb0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc96c011fa0 tx=0x7fc96c011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8fa439e091f873 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc953fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc953fff640 1 -- 172.21.15.44:0/2804402659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c019040 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc953fff640 1 -- 172.21.15.44:0/2804402659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc96c003b80 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc968031df0 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc953fff640 1 -- 172.21.15.44:0/2804402659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc96c003d40 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 10 client.17949 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aac91a6ea0 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc953fff640 1 -- 172.21.15.44:0/2804402659 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc968031df0 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc953fff640 1 -- 172.21.15.44:0/2804402659 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc96c0044b0 con 0x55aac927ce10 2024-09-13T17:34:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 10 client.17949 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 20 client.17949 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac92ef3e0 0x55aac92f1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 4 client.17949 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.404+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aac8f38e30 con 0x55aac92ef3e0 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc978cc1640 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac92ef3e0 0x55aac92f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc978cc1640 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac92ef3e0 0x55aac92f1800 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb061fd2d217b45 server_cookie=5572b19a640ac468 in_seq=0 out_seq=0 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc953fff640 10 client.17949 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc953fff640 1 -- 172.21.15.44:0/2804402659 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aac8f38e30 con 0x55aac92ef3e0 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc953fff640 10 client.17949 handle_command_reply: tid=0 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 client.17949 shutdown 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 msgr2=0x55aac928edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac928edb0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc96c011fa0 tx=0x7fc96c011fd0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 shutdown_connections 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55aac92ef3e0 0x55aac92f1800 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aac9287fd0 0x55aac928b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aac927d730 0x55aac928b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc953fff640 0 client.17949 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 --2- 172.21.15.44:0/2804402659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aac927ce10 0x55aac928edb0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.405+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 >> 172.21.15.44:0/2804402659 conn(0x55aac9279380 msgr2=0x55aac9284580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 shutdown_connections 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97dc4a740 1 -- 172.21.15.44:0/2804402659 wait complete. 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97dc4a740 20 client.17949 trim_cache size 0 max 16384 2024-09-13T17:34:22.417 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:34:22.418 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:34:22.418 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:34:22.418 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:34:22.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9640a1200 msgr2=0x7fc9640a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9640a1200 0x7fc9640a36c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc960098950 tx=0x7fc96006bad0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 msgr2=0x7fc974196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 0x7fc974196690 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc96000d950 tx=0x7fc96000de10 comp rx=0 tx=0).stop 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 shutdown_connections 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc9640a1200 0x7fc9640a36c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97410a350 0x7fc97419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc974101800 0x7fc974196690 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 --2- 172.21.15.44:0/840200963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974100e50 0x7fc974196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 >> 172.21.15.44:0/840200963 conn(0x7fc9740fa930 msgr2=0x7fc9740fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 shutdown_connections 2024-09-13T17:34:22.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.406+0000 7fc97af25640 1 -- 172.21.15.44:0/840200963 wait complete. 2024-09-13T17:34:22.421 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:34:22.421 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 -- 172.21.15.44:0/1333042990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc100940 msgr2=0x7feccc104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 --2- 172.21.15.44:0/1333042990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc100940 0x7feccc104980 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009930 tx=0x7fecb402a440 comp rx=0 tx=0).stop 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 -- 172.21.15.44:0/1333042990 shutdown_connections 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 --2- 172.21.15.44:0/1333042990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc1050e0 0x7feccc10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 --2- 172.21.15.44:0/1333042990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc100940 0x7feccc104980 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 --2- 172.21.15.44:0/1333042990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feccc0fff90 0x7feccc100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 -- 172.21.15.44:0/1333042990 >> 172.21.15.44:0/1333042990 conn(0x7feccc0fba20 msgr2=0x7feccc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 -- 172.21.15.44:0/1333042990 shutdown_connections 2024-09-13T17:34:22.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.527+0000 7fecd10d7640 1 -- 172.21.15.44:0/1333042990 wait complete. 2024-09-13T17:34:22.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.528+0000 7fecd10d7640 1 Processor -- start 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.528+0000 7fecd10d7640 1 -- start start 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feccc0fff90 0x7feccc196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc1050e0 0x7feccc196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 0x7feccc19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc19b520 con 0x7feccc19ab30 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc19b690 con 0x7feccc0fff90 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc19b800 con 0x7feccc1050e0 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 0x7feccc19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 0x7feccc19afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36226/0 (socket says 172.21.15.44:36226) 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 -- 172.21.15.44:0/864850164 learned_addr learned my addr 172.21.15.44:0/864850164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecca59c640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc1050e0 0x7feccc196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccad9d640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feccc0fff90 0x7feccc196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 -- 172.21.15.44:0/864850164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc1050e0 msgr2=0x7feccc196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc1050e0 0x7feccc196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 -- 172.21.15.44:0/864850164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feccc0fff90 msgr2=0x7feccc196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feccc0fff90 0x7feccc196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 -- 172.21.15.44:0/864850164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4009560 con 0x7feccc19ab30 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7feccb59e640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 0x7feccc19afe0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c970 tx=0x7fecc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4007bb0 con 0x7feccc19ab30 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc1a2de0 con 0x7feccc19ab30 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecc4007d10 con 0x7feccc19ab30 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.529+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4018660 con 0x7feccc19ab30 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.530+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc1a3210 con 0x7feccc19ab30 2024-09-13T17:34:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.531+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc101460 con 0x7feccc19ab30 2024-09-13T17:34:22.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.532+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fecc400f450 con 0x7feccc19ab30 2024-09-13T17:34:22.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.532+0000 7fecc3fff640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fecb80a1200 0x7fecb80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.532+0000 7feccad9d640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fecb80a1200 0x7fecb80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.532+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fecc40cd030 con 0x7feccc19ab30 2024-09-13T17:34:22.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.533+0000 7feccad9d640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fecb80a1200 0x7fecb80a36c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005fd0 tx=0x7fecbc0098b0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.536+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fecc4096110 con 0x7feccc19ab30 2024-09-13T17:34:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:22 smithi044 ceph-mon[31175]: pgmap v236: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-13T17:34:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:22 smithi070 ceph-mon[34502]: pgmap v236: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-13T17:34:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.744+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feccc197600 con 0x7feccc19ab30 2024-09-13T17:34:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.744+0000 7fecc3fff640 1 -- 172.21.15.44:0/864850164 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fecc4099730 con 0x7feccc19ab30 2024-09-13T17:34:22.746 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:34:22.746 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:34:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:34:22.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fecb80a1200 msgr2=0x7fecb80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fecb80a1200 0x7fecb80a36c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005fd0 tx=0x7fecbc0098b0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 msgr2=0x7feccc19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 0x7feccc19afe0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c970 tx=0x7fecc400ce30 comp rx=0 tx=0).stop 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 shutdown_connections 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fecb80a1200 0x7fecb80a36c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc1050e0 0x7feccc196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc19ab30 0x7feccc19afe0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 --2- 172.21.15.44:0/864850164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feccc0fff90 0x7feccc196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.746+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 >> 172.21.15.44:0/864850164 conn(0x7feccc0fba20 msgr2=0x7feccc0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.747+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 shutdown_connections 2024-09-13T17:34:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.747+0000 7fecd10d7640 1 -- 172.21.15.44:0/864850164 wait complete. 2024-09-13T17:34:22.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:34:22.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:22 smithi100 ceph-mon[35055]: pgmap v236: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-13T17:34:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 -- 172.21.15.44:0/1423671365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef010a330 msgr2=0x7fcef010c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 --2- 172.21.15.44:0/1423671365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef010a330 0x7fcef010c7a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009ca0 tx=0x7fcee802a950 comp rx=0 tx=0).stop 2024-09-13T17:34:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 -- 172.21.15.44:0/1423671365 shutdown_connections 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 --2- 172.21.15.44:0/1423671365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef010a330 0x7fcef010c7a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 --2- 172.21.15.44:0/1423671365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0101880 0x7fcef0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 --2- 172.21.15.44:0/1423671365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef0100f60 0x7fcef0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 -- 172.21.15.44:0/1423671365 >> 172.21.15.44:0/1423671365 conn(0x7fcef0075c60 msgr2=0x7fcef00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 -- 172.21.15.44:0/1423671365 shutdown_connections 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 -- 172.21.15.44:0/1423671365 wait complete. 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 Processor -- start 2024-09-13T17:34:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.863+0000 7fcef52d7640 1 -- start start 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fcef52d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0100f60 0x7fcef01960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fcef52d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 0x7fcef0196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fcef52d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef010a330 0x7fcef019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fcef52d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019b020 con 0x7fcef0101880 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fcef52d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019b190 con 0x7fcef0100f60 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fcef52d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019b350 con 0x7fcef010a330 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 0x7fcef0196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 0x7fcef0196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36244/0 (socket says 172.21.15.44:36244) 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 -- 172.21.15.44:0/2494153006 learned_addr learned my addr 172.21.15.44:0/2494153006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceef7fe640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef010a330 0x7fcef019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceeeffd640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0100f60 0x7fcef01960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 -- 172.21.15.44:0/2494153006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef010a330 msgr2=0x7fcef019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef010a330 0x7fcef019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 -- 172.21.15.44:0/2494153006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0100f60 msgr2=0x7fcef01960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0100f60 0x7fcef01960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 -- 172.21.15.44:0/2494153006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8009900 con 0x7fcef0101880 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.864+0000 7fceee7fc640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 0x7fcef0196630 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fcee400d950 tx=0x7fcee400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.865+0000 7fced7fff640 1 -- 172.21.15.44:0/2494153006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4009a30 con 0x7fcef0101880 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.865+0000 7fced7fff640 1 -- 172.21.15.44:0/2494153006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcee40044b0 con 0x7fcef0101880 2024-09-13T17:34:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.865+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef019b630 con 0x7fcef0101880 2024-09-13T17:34:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.865+0000 7fced7fff640 1 -- 172.21.15.44:0/2494153006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee40047a0 con 0x7fcef0101880 2024-09-13T17:34:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.865+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef01a2eb0 con 0x7fcef0101880 2024-09-13T17:34:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.867+0000 7fced7fff640 1 -- 172.21.15.44:0/2494153006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcee4004020 con 0x7fcef0101880 2024-09-13T17:34:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.867+0000 7fcef7ffc740 1 Processor -- start 2024-09-13T17:34:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.867+0000 7fcef7ffc740 1 -- start start 2024-09-13T17:34:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3f90 0x564da00f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00f5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da004dcc0 con 0x564da00e3670 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da004b450 con 0x564da00f2d60 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0046d40 con 0x564da00e3f90 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00e3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36248/0 (socket says 172.21.15.44:36248) 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 -- 172.21.15.44:0/4070572418 learned_addr learned my addr 172.21.15.44:0/4070572418 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceee7fc640 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceeeffd640 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3f90 0x564da00f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fced7fff640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fced80a1200 0x7fced80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 -- 172.21.15.44:0/4070572418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3f90 msgr2=0x564da00f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3f90 0x564da00f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 -- 172.21.15.44:0/4070572418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 msgr2=0x564da00f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 -- 172.21.15.44:0/4070572418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564da000e620 con 0x564da00e3670 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fced7fff640 1 -- 172.21.15.44:0/2494153006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fcee40cc9f0 con 0x7fcef0101880 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fced80a1200 0x7fced80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.868+0000 7fceef7fe640 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00e3a50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fcee80011c0 tx=0x7fcee802ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae0247eb77b5d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fced57fa640 1 -- 172.21.15.44:0/4070572418 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8003c30 con 0x564da00e3670 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fced57fa640 1 -- 172.21.15.44:0/4070572418 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcee804c040 con 0x564da00e3670 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fced57fa640 1 -- 172.21.15.44:0/4070572418 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8040030 con 0x564da00e3670 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 -- 172.21.15.44:0/4070572418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 msgr2=0x564da00e3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fceef7fe640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fced80a1200 0x7fced80a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcee804e000 tx=0x7fcee804b000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00e3a50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fcee80011c0 tx=0x7fcee802ff30 comp rx=0 tx=0).stop 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 -- 172.21.15.44:0/4070572418 shutdown_connections 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3f90 0x564da00f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/4070572418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00e3a50 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 -- 172.21.15.44:0/4070572418 >> 172.21.15.44:0/4070572418 conn(0x564da00dfd00 msgr2=0x564da00e00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 -- 172.21.15.44:0/4070572418 shutdown_connections 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.869+0000 7fcef7ffc740 1 -- 172.21.15.44:0/4070572418 wait complete. 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 -- start start 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 Processor -- start 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3f90 0x564da00ef750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00efc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0005180 con 0x564da00e3670 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9fd1f430 con 0x564da00e3f90 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9ffca5e0 con 0x564da00f2d60 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fcef7ffc740 10 client.0 _ll_register_callbacks cb 0x564da0036aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36250/0 (socket says 172.21.15.44:36250) 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 -- 172.21.15.44:0/3635614181 learned_addr learned my addr 172.21.15.44:0/3635614181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceee7fc640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3f90 0x564da00ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceef7fe640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 -- 172.21.15.44:0/3635614181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 msgr2=0x564da00efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 -- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3f90 msgr2=0x564da00ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3f90 0x564da00ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceeeffd640 1 -- 172.21.15.44:0/3635614181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef010d420 con 0x564da00e3670 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.870+0000 7fceef7fe640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fceeeffd640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00f2370 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fcee0012730 tx=0x7fcee0012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0c0c3639bd86db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fced2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fced2ff5640 1 -- 172.21.15.44:0/3635614181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0017070 con 0x564da00e3670 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fced2ff5640 1 -- 172.21.15.44:0/3635614181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fcee0004260 con 0x564da00e3670 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564da0070b80 con 0x564da00e3670 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fced2ff5640 1 -- 172.21.15.44:0/3635614181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee0004420 con 0x564da00e3670 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 10 client.17967 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564da00453c0 con 0x564da00e3670 2024-09-13T17:34:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fced2ff5640 1 -- 172.21.15.44:0/3635614181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564da0070b80 con 0x564da00e3670 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fced2ff5640 1 -- 172.21.15.44:0/3635614181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcee0003b80 con 0x564da00e3670 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 10 client.17967 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 10 client.17967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 20 client.17967 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da0102710 0x564da0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 4 client.17967 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.871+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564da00391f0 con 0x564da0102710 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.872+0000 7fceee7fc640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da0102710 0x564da0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.872+0000 7fceee7fc640 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da0102710 0x564da0104b00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24f95473c64a2d1c server_cookie=46e2a3c7f753ff1f in_seq=0 out_seq=0 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.872+0000 7fced2ff5640 10 client.17967 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fced2ff5640 1 -- 172.21.15.44:0/3635614181 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564da00391f0 con 0x564da0102710 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fced2ff5640 10 client.17967 handle_command_reply: tid=0 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 client.17967 shutdown 2024-09-13T17:34:22.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 msgr2=0x564da00f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00f2370 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fcee0012730 tx=0x7fcee0012bf0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 shutdown_connections 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da0102710 0x564da0104b00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fced2ff5640 0 client.17967 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3f90 0x564da00ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/3635614181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3670 0x564da00f2370 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 >> 172.21.15.44:0/3635614181 conn(0x564da00dfd00 msgr2=0x564da00e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 shutdown_connections 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 1 -- 172.21.15.44:0/3635614181 wait complete. 2024-09-13T17:34:22.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.873+0000 7fcef7ffc740 20 client.17967 trim_cache size 0 max 16384 2024-09-13T17:34:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 Processor -- start 2024-09-13T17:34:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 -- start start 2024-09-13T17:34:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3670 0x564da00df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da00391f0 con 0x564da00e3f90 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0070b80 con 0x564da00e3670 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0101020 con 0x564da00f2d60 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00f9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36266/0 (socket says 172.21.15.44:36266) 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 -- 172.21.15.44:0/691703623 learned_addr learned my addr 172.21.15.44:0/691703623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceeeffd640 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3670 0x564da00df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 -- 172.21.15.44:0/691703623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 msgr2=0x564da00f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 -- 172.21.15.44:0/691703623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3670 msgr2=0x564da00df6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3670 0x564da00df6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.884+0000 7fceee7fc640 1 -- 172.21.15.44:0/691703623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d9ffe7770 con 0x564da00e3f90 2024-09-13T17:34:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fceee7fc640 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00f9a40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fcee4099490 tx=0x7fcee40ceec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f588b5586e548313 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fced5ffb640 1 -- 172.21.15.44:0/691703623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee40990f0 con 0x564da00e3f90 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fced5ffb640 1 -- 172.21.15.44:0/691703623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcee4009de0 con 0x564da00e3f90 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fced5ffb640 1 -- 172.21.15.44:0/691703623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee409d650 con 0x564da00e3f90 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 -- 172.21.15.44:0/691703623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 msgr2=0x564da00f9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00f9a40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fcee4099490 tx=0x7fcee40ceec0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 -- 172.21.15.44:0/691703623 shutdown_connections 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00f2d60 0x564da00f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00f9a40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/691703623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00e3670 0x564da00df6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 -- 172.21.15.44:0/691703623 >> 172.21.15.44:0/691703623 conn(0x564da00dfd00 msgr2=0x564da00ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 -- 172.21.15.44:0/691703623 shutdown_connections 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.885+0000 7fcef7ffc740 1 -- 172.21.15.44:0/691703623 wait complete. 2024-09-13T17:34:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 -- start start 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 Processor -- start 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3670 0x564da00ec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0158c80 con 0x564da00e3f90 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0158df0 con 0x564da00f2d60 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da0158fb0 con 0x564da00e3670 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fcef7ffc740 10 client.0 _ll_register_callbacks cb 0x7fcee0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00ecb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36280/0 (socket says 172.21.15.44:36280) 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 -- 172.21.15.44:0/2881734690 learned_addr learned my addr 172.21.15.44:0/2881734690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceeeffd640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3670 0x564da00ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceef7fe640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 -- 172.21.15.44:0/2881734690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3670 msgr2=0x564da00ec600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3670 0x564da00ec600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 -- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 msgr2=0x564da00ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00ed080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 -- 172.21.15.44:0/2881734690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee40bee70 con 0x564da00e3f90 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fceee7fc640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00ecb40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fcee4096200 tx=0x7fcee4099390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c974deef0a4bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.886+0000 7fced2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fced2ff5640 1 -- 172.21.15.44:0/2881734690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4099790 con 0x564da00e3f90 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fced2ff5640 1 -- 172.21.15.44:0/2881734690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fcee4099e90 con 0x564da00e3f90 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcee804c040 con 0x564da00e3f90 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fced2ff5640 1 -- 172.21.15.44:0/2881734690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee40042d0 con 0x564da00e3f90 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 10 client.17973 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564da000ce80 con 0x564da00e3f90 2024-09-13T17:34:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fced2ff5640 1 -- 172.21.15.44:0/2881734690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcee804c040 con 0x564da00e3f90 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fced2ff5640 1 -- 172.21.15.44:0/2881734690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcee4095ea0 con 0x564da00e3f90 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 10 client.17973 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 10 client.17973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 20 client.17973 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da015a600 0x564da015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 4 client.17973 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d9fd3a4e0 con 0x564da015a600 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.887+0000 7fceeeffd640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da015a600 0x564da015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fced2ff5640 10 client.17973 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fceeeffd640 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da015a600 0x564da015ca20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee45c1ec9269da68 server_cookie=e71e40c9c9e995f3 in_seq=0 out_seq=0 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fced2ff5640 1 -- 172.21.15.44:0/2881734690 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d9fd3a4e0 con 0x564da015a600 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fced2ff5640 10 client.17973 handle_command_reply: tid=0 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 client.17973 shutdown 2024-09-13T17:34:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 msgr2=0x564da00ecb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00ecb40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fcee4096200 tx=0x7fcee4099390 comp rx=0 tx=0).stop 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 shutdown_connections 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x564da015a600 0x564da015ca20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564da00f2d60 0x564da00ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564da00e3f90 0x564da00ecb40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 --2- 172.21.15.44:0/2881734690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da00e3670 0x564da00ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fced2ff5640 0 client.17973 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 >> 172.21.15.44:0/2881734690 conn(0x564da00dfd00 msgr2=0x564da00ea7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 shutdown_connections 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 1 -- 172.21.15.44:0/2881734690 wait complete. 2024-09-13T17:34:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.888+0000 7fcef7ffc740 20 client.17973 trim_cache size 0 max 16384 2024-09-13T17:34:22.897 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:34:22.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fced80a1200 msgr2=0x7fced80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fced80a1200 0x7fced80a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcee804e000 tx=0x7fcee804b000 comp rx=0 tx=0).stop 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 msgr2=0x7fcef0196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 0x7fcef0196630 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fcee400d950 tx=0x7fcee400de10 comp rx=0 tx=0).stop 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 shutdown_connections 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fced80a1200 0x7fced80a36c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef010a330 0x7fcef019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcef0101880 0x7fcef0196630 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 --2- 172.21.15.44:0/2494153006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0100f60 0x7fcef01960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 >> 172.21.15.44:0/2494153006 conn(0x7fcef0075c60 msgr2=0x7fcef00fb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 shutdown_connections 2024-09-13T17:34:22.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:22.889+0000 7fcef52d7640 1 -- 172.21.15.44:0/2494153006 wait complete. 2024-09-13T17:34:22.900 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:34:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 -- 172.21.15.44:0/3779630633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 msgr2=0x7f71b0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 --2- 172.21.15.44:0/3779630633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b0109ce0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009900 tx=0x7f71a402a410 comp rx=0 tx=0).stop 2024-09-13T17:34:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 -- 172.21.15.44:0/3779630633 shutdown_connections 2024-09-13T17:34:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 --2- 172.21.15.44:0/3779630633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71b010a350 0x7f71b010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 --2- 172.21.15.44:0/3779630633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b0109ce0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 --2- 172.21.15.44:0/3779630633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b0100e50 0x7f71b0101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 -- 172.21.15.44:0/3779630633 >> 172.21.15.44:0/3779630633 conn(0x7f71b00fa930 msgr2=0x7f71b00fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 -- 172.21.15.44:0/3779630633 shutdown_connections 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.003+0000 7f71b571f640 1 -- 172.21.15.44:0/3779630633 wait complete. 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 Processor -- start 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 -- start start 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b0100e50 0x7f71b00feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b00ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71b010a350 0x7f71b00ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01a4570 con 0x7f71b0101800 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01a46e0 con 0x7f71b010a350 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71b571f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01a48a0 con 0x7f71b0100e50 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71ae7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b00ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71ae7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b00ff070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36300/0 (socket says 172.21.15.44:36300) 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71ae7fc640 1 -- 172.21.15.44:0/1915775650 learned_addr learned my addr 172.21.15.44:0/1915775650 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71aeffd640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b0100e50 0x7f71b00feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.004+0000 7f71af7fe640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71b010a350 0x7f71b00ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71ae7fc640 1 -- 172.21.15.44:0/1915775650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b0100e50 msgr2=0x7f71b00feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b0100e50 0x7f71b00feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71ae7fc640 1 -- 172.21.15.44:0/1915775650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71b010a350 msgr2=0x7f71b00ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71b010a350 0x7f71b00ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71ae7fc640 1 -- 172.21.15.44:0/1915775650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009560 con 0x7f71b0101800 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b00ff070 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f71a4006010 tx=0x7f71a4005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f7197fff640 1 -- 172.21.15.44:0/1915775650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4004c50 con 0x7f71b0101800 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b01a4b20 con 0x7f71b0101800 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f7197fff640 1 -- 172.21.15.44:0/1915775650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a4004db0 con 0x7f71b0101800 2024-09-13T17:34:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f7197fff640 1 -- 172.21.15.44:0/1915775650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a403c930 con 0x7f71b0101800 2024-09-13T17:34:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.005+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b01a4fd0 con 0x7f71b0101800 2024-09-13T17:34:23.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f7197fff640 1 -- 172.21.15.44:0/1915775650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f71a4003c80 con 0x7f71b0101800 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 Processor -- start 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 -- start start 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bec2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267becd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267becfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267be2ccc0 con 0x56267bec2790 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267be2a450 con 0x56267becd950 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71b843d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267be25d40 con 0x56267bec30b0 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bec2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bec2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36308/0 (socket says 172.21.15.44:36308) 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 -- 172.21.15.44:0/2599879241 learned_addr learned my addr 172.21.15.44:0/2599879241 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267becd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267becfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 -- 172.21.15.44:0/2599879241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 msgr2=0x56267becd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267becd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 -- 172.21.15.44:0/2599879241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 msgr2=0x56267becfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f7197fff640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f71980a1200 0x7f71980a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267becfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71af7fe640 1 -- 172.21.15.44:0/2599879241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267bded620 con 0x56267bec2790 2024-09-13T17:34:23.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267becfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.008+0000 7f7197fff640 1 -- 172.21.15.44:0/1915775650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f71a40ede30 con 0x7f71b0101800 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71af7fe640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f71980a1200 0x7f71980a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71af7fe640 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bec2b70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009f70 tx=0x7f71a8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d913bdcbd9d7b020 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71957fa640 1 -- 172.21.15.44:0/2599879241 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8012b30 con 0x56267bec2790 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71957fa640 1 -- 172.21.15.44:0/2599879241 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a801ce30 con 0x56267bec2790 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71957fa640 1 -- 172.21.15.44:0/2599879241 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8011d70 con 0x56267bec2790 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71b843d740 1 -- 172.21.15.44:0/2599879241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 msgr2=0x56267bec2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.009+0000 7f71af7fe640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f71980a1200 0x7f71980a36c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71a801f000 tx=0x7f71a8022000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bec2b70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009f70 tx=0x7f71a8010800 comp rx=0 tx=0).stop 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- 172.21.15.44:0/2599879241 shutdown_connections 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267becd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267becfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- 172.21.15.44:0/2599879241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bec2b70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- 172.21.15.44:0/2599879241 >> 172.21.15.44:0/2599879241 conn(0x56267bebed00 msgr2=0x56267bebf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- 172.21.15.44:0/2599879241 shutdown_connections 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- 172.21.15.44:0/2599879241 wait complete. 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- start start 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 Processor -- start 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec2790 0x56267beccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 0x56267beca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267beca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bde4180 con 0x56267bec30b0 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bafe430 con 0x56267becd950 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bda95e0 con 0x56267bec2790 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71b843d740 10 client.0 _ll_register_callbacks cb 0x56267be15aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71ae7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 0x56267beca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71ae7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 0x56267beca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36324/0 (socket says 172.21.15.44:36324) 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71ae7fc640 1 -- 172.21.15.44:0/2737496589 learned_addr learned my addr 172.21.15.44:0/2737496589 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.010+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec2790 0x56267beccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71af7fe640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267beca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71ae7fc640 1 -- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec2790 msgr2=0x56267beccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec2790 0x56267beccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71ae7fc640 1 -- 172.21.15.44:0/2737496589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 msgr2=0x56267beca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267beca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71ae7fc640 1 -- 172.21.15.44:0/2737496589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a8009bd0 con 0x56267bec30b0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 0x56267beca260 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000c00 tx=0x7f71a40ba6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c152cf5a420e54ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f7192ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f7192ff5640 1 -- 172.21.15.44:0/2737496589 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40bac80 con 0x56267bec30b0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f7192ff5640 1 -- 172.21.15.44:0/2737496589 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f71a40bea00 con 0x56267bec30b0 2024-09-13T17:34:23.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56267be4fb80 con 0x56267bec30b0 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f7192ff5640 1 -- 172.21.15.44:0/2737496589 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a40c3050 con 0x56267bec30b0 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71b843d740 10 client.17985 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56267be243c0 con 0x56267bec30b0 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f7192ff5640 1 -- 172.21.15.44:0/2737496589 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56267be4fb80 con 0x56267bec30b0 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f7192ff5640 1 -- 172.21.15.44:0/2737496589 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71a40bbbe0 con 0x56267bec30b0 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71b843d740 10 client.17985 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71b843d740 10 client.17985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.011+0000 7f71b843d740 20 client.17985 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.012+0000 7f71b843d740 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bedce20 0x56267bedf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.012+0000 7f71b843d740 4 client.17985 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.012+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56267be181f0 con 0x56267bedce20 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.012+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bedce20 0x56267bedf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.012+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bedce20 0x56267bedf240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f006491e193f4503 server_cookie=cfeb8ec5a429bb5a in_seq=0 out_seq=0 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.012+0000 7f7192ff5640 10 client.17985 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f7192ff5640 1 -- 172.21.15.44:0/2737496589 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56267be181f0 con 0x56267bedce20 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f7192ff5640 10 client.17985 handle_command_reply: tid=0 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 client.17985 shutdown 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 msgr2=0x56267beca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 0x56267beca260 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000c00 tx=0x7f71a40ba6a0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 shutdown_connections 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bedce20 0x56267bedf240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267beca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f7192ff5640 0 client.17985 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec30b0 0x56267beca260 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 --2- 172.21.15.44:0/2737496589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec2790 0x56267beccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 >> 172.21.15.44:0/2737496589 conn(0x56267bebed00 msgr2=0x56267bebdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 shutdown_connections 2024-09-13T17:34:23.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 1 -- 172.21.15.44:0/2737496589 wait complete. 2024-09-13T17:34:23.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.013+0000 7f71b843d740 20 client.17985 trim_cache size 0 max 16384 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.024+0000 7f71b843d740 1 Processor -- start 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.024+0000 7f71b843d740 1 -- start start 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71b843d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bebe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71b843d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267bec30b0 0x56267bec5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71b843d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267becd950 0x56267bec5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71b843d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bec8790 con 0x56267bec2790 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71b843d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bec1f20 con 0x56267bec30b0 2024-09-13T17:34:23.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71b843d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bedf570 con 0x56267becd950 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bebe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bebe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36328/0 (socket says 172.21.15.44:36328) 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 -- 172.21.15.44:0/2321927950 learned_addr learned my addr 172.21.15.44:0/2321927950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71af7fe640 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267becd950 0x56267bec5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 -- 172.21.15.44:0/2321927950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267becd950 msgr2=0x56267bec5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267becd950 0x56267bec5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 -- 172.21.15.44:0/2321927950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267bec30b0 msgr2=0x56267bec5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267bec30b0 0x56267bec5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 -- 172.21.15.44:0/2321927950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267bdc6770 con 0x56267bec2790 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71af7fe640 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267becd950 0x56267bec5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f71aeffd640 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bebe6a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009b60 tx=0x7f71a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c949945ed64d7db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f7195ffb640 1 -- 172.21.15.44:0/2321927950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0017070 con 0x56267bec2790 2024-09-13T17:34:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f7195ffb640 1 -- 172.21.15.44:0/2321927950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a0018040 con 0x56267bec2790 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.025+0000 7f7195ffb640 1 -- 172.21.15.44:0/2321927950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0004b20 con 0x56267bec2790 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- 172.21.15.44:0/2321927950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 msgr2=0x56267bebe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bebe6a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009b60 tx=0x7f71a0005f90 comp rx=0 tx=0).stop 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- 172.21.15.44:0/2321927950 shutdown_connections 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267becd950 0x56267bec5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267bec30b0 0x56267bec5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- 172.21.15.44:0/2321927950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bebe6a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- 172.21.15.44:0/2321927950 >> 172.21.15.44:0/2321927950 conn(0x56267bebed00 msgr2=0x56267bed90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- 172.21.15.44:0/2321927950 shutdown_connections 2024-09-13T17:34:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- 172.21.15.44:0/2321927950 wait complete. 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- start start 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 Processor -- start 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bed3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267bed35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267bed3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bf332f0 con 0x56267bec2790 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.026+0000 7f71b843d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bf33430 con 0x56267becd950 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71b843d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267bf33570 con 0x56267bec30b0 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71b843d740 10 client.0 _ll_register_callbacks cb 0x7f71a8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bed3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bed3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36330/0 (socket says 172.21.15.44:36330) 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 -- 172.21.15.44:0/1140661229 learned_addr learned my addr 172.21.15.44:0/1140661229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71af7fe640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267bed3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267bed35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 -- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 msgr2=0x56267bed35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267bed35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 -- 172.21.15.44:0/1140661229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 msgr2=0x56267bed3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267bed3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 -- 172.21.15.44:0/1140661229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a0009db0 con 0x56267bec2790 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267bed35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:23.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71aeffd640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bed3080 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f71a001e670 tx=0x7f71a001e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc18cfb2117835d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f7192ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f7192ff5640 1 -- 172.21.15.44:0/1140661229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0017040 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f7192ff5640 1 -- 172.21.15.44:0/1140661229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f71a0030640 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71a801ce30 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f7192ff5640 1 -- 172.21.15.44:0/1140661229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a001e8c0 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.027+0000 7f71b843d740 10 client.17991 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56267bdebe80 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f7192ff5640 1 -- 172.21.15.44:0/1140661229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a801ce30 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f7192ff5640 1 -- 172.21.15.44:0/1140661229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71a0024020 con 0x56267bec2790 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 10 client.17991 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 10 client.17991 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 20 client.17991 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bf34b00 0x56267bf36f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 4 client.17991 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:34:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56267bb194e0 con 0x56267bf34b00 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.028+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bf34b00 0x56267bf36f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f7192ff5640 10 client.17991 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71ae7fc640 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bf34b00 0x56267bf36f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b182207d6640600c server_cookie=8639aa6fcee4a7f4 in_seq=0 out_seq=0 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f7192ff5640 1 -- 172.21.15.44:0/1140661229 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56267bb194e0 con 0x56267bf34b00 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f7192ff5640 10 client.17991 handle_command_reply: tid=0 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 client.17991 shutdown 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 msgr2=0x56267bed3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bed3080 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f71a001e670 tx=0x7f71a001e6a0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 shutdown_connections 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x56267bf34b00 0x56267bf36f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56267bec30b0 0x56267bed35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56267becd950 0x56267bed3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f7192ff5640 0 client.17991 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 --2- 172.21.15.44:0/1140661229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56267bec2790 0x56267bed3080 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 >> 172.21.15.44:0/1140661229 conn(0x56267bebed00 msgr2=0x56267bec9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 shutdown_connections 2024-09-13T17:34:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 1 -- 172.21.15.44:0/1140661229 wait complete. 2024-09-13T17:34:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b843d740 20 client.17991 trim_cache size 0 max 16384 2024-09-13T17:34:23.040 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:34:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f71980a1200 msgr2=0x7f71980a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b571f640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f71980a1200 0x7f71980a36c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71a801f000 tx=0x7f71a8022000 comp rx=0 tx=0).stop 2024-09-13T17:34:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 msgr2=0x7f71b00ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.029+0000 7f71b571f640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b00ff070 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f71a4006010 tx=0x7f71a4005620 comp rx=0 tx=0).stop 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 shutdown_connections 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f71980a1200 0x7f71980a36c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71b010a350 0x7f71b00ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b0101800 0x7f71b00ff070 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 --2- 172.21.15.44:0/1915775650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b0100e50 0x7f71b00feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 >> 172.21.15.44:0/1915775650 conn(0x7f71b00fa930 msgr2=0x7f71b00fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 shutdown_connections 2024-09-13T17:34:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.030+0000 7f71b571f640 1 -- 172.21.15.44:0/1915775650 wait complete. 2024-09-13T17:34:23.042 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:34:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 -- 172.21.15.44:0/1639683277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 msgr2=0x7f63f40ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/1639683277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f40ff260 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009930 tx=0x7f63e402a420 comp rx=0 tx=0).stop 2024-09-13T17:34:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 -- 172.21.15.44:0/1639683277 shutdown_connections 2024-09-13T17:34:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/1639683277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f40ff830 0x7f63f4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/1639683277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63f4103fd0 0x7f63f410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/1639683277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f40ff260 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 -- 172.21.15.44:0/1639683277 >> 172.21.15.44:0/1639683277 conn(0x7f63f40fa930 msgr2=0x7f63f40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 -- 172.21.15.44:0/1639683277 shutdown_connections 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.146+0000 7f63fa8aa640 1 -- 172.21.15.44:0/1639683277 wait complete. 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 Processor -- start 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 -- start start 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f4196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63f4103fd0 0x7f63f4196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f419ab60 0x7f63f419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419b550 con 0x7f63f40fee80 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419b6c0 con 0x7f63f4103fd0 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63fa8aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419b830 con 0x7f63f419ab60 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f4196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f4196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36344/0 (socket says 172.21.15.44:36344) 2024-09-13T17:34:23.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 -- 172.21.15.44:0/2752560754 learned_addr learned my addr 172.21.15.44:0/2752560754 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f8e47640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f419ab60 0x7f63f419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63ebfff640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63f4103fd0 0x7f63f4196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 -- 172.21.15.44:0/2752560754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f419ab60 msgr2=0x7f63f419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f419ab60 0x7f63f419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 -- 172.21.15.44:0/2752560754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63f4103fd0 msgr2=0x7f63f4196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63f4103fd0 0x7f63f4196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.147+0000 7f63f3fff640 1 -- 172.21.15.44:0/2752560754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4009560 con 0x7f63f40fee80 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.148+0000 7f63f3fff640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f4196230 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f63e402a940 tx=0x7f63e4005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.148+0000 7f63f1ffb640 1 -- 172.21.15.44:0/2752560754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4004b50 con 0x7f63f40fee80 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.148+0000 7f63f1ffb640 1 -- 172.21.15.44:0/2752560754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63e4004cb0 con 0x7f63f40fee80 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.148+0000 7f63f1ffb640 1 -- 172.21.15.44:0/2752560754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e403c890 con 0x7f63f40fee80 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.148+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f41a2e10 con 0x7f63f40fee80 2024-09-13T17:34:23.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.148+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f41a31d0 con 0x7f63f40fee80 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.150+0000 7f63f1ffb640 1 -- 172.21.15.44:0/2752560754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f63e40035f0 con 0x7f63f40fee80 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.150+0000 7f63fd5ce740 1 Processor -- start 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.150+0000 7f63fd5ce740 1 -- start start 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.150+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.150+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e5d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.150+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e5fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9dbccc0 con 0x557de9e52790 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9dba450 con 0x557de9e5d950 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9db5d40 con 0x557de9e530b0 2024-09-13T17:34:23.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e52b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36360/0 (socket says 172.21.15.44:36360) 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 -- 172.21.15.44:0/554967351 learned_addr learned my addr 172.21.15.44:0/554967351 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f3fff640 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e5d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63ebfff640 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e5fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f1ffb640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63e00a1280 0x7f63e00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 -- 172.21.15.44:0/554967351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 msgr2=0x557de9e5d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e5d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 -- 172.21.15.44:0/554967351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 msgr2=0x557de9e5fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e5fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 -- 172.21.15.44:0/554967351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de9d7d620 con 0x557de9e52790 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f3fff640 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e5d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63e00a1280 0x7f63e00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f1ffb640 1 -- 172.21.15.44:0/2752560754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f63e40ede60 con 0x7f63f40fee80 2024-09-13T17:34:23.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e52b70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011590 tx=0x7f63ec011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1af59ff77c8ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63ea7fc640 1 -- 172.21.15.44:0/554967351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec01bb90 con 0x557de9e52790 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63ea7fc640 1 -- 172.21.15.44:0/554967351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63ec01f070 con 0x557de9e52790 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63f8e47640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63e00a1280 0x7f63e00a3740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63ec022000 tx=0x7f63ec023000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.151+0000 7f63ea7fc640 1 -- 172.21.15.44:0/554967351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec025d50 con 0x557de9e52790 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 -- 172.21.15.44:0/554967351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 msgr2=0x557de9e52b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e52b70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011590 tx=0x7f63ec011a50 comp rx=0 tx=0).stop 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 -- 172.21.15.44:0/554967351 shutdown_connections 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e5d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e5fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/554967351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e52b70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 -- 172.21.15.44:0/554967351 >> 172.21.15.44:0/554967351 conn(0x557de9e4ed00 msgr2=0x557de9e4f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 -- 172.21.15.44:0/554967351 shutdown_connections 2024-09-13T17:34:23.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 -- 172.21.15.44:0/554967351 wait complete. 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.152+0000 7f63fd5ce740 1 -- start start 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 Processor -- start 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e5ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e530b0 0x557de9e5d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e5d950 0x557de9e5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9d74180 con 0x557de9e52790 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9a8e430 con 0x557de9e530b0 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9d395e0 con 0x557de9e5d950 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63fd5ce740 10 client.0 _ll_register_callbacks cb 0x557de9da5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e5ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e5ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36372/0 (socket says 172.21.15.44:36372) 2024-09-13T17:34:23.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 -- 172.21.15.44:0/1082387324 learned_addr learned my addr 172.21.15.44:0/1082387324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63ebfff640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e530b0 0x557de9e5d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f8e47640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e5d950 0x557de9e5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 -- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e5d950 msgr2=0x557de9e5a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e5d950 0x557de9e5a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 -- 172.21.15.44:0/1082387324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e530b0 msgr2=0x557de9e5d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e530b0 0x557de9e5d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.153+0000 7f63f3fff640 1 -- 172.21.15.44:0/1082387324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec009c50 con 0x557de9e52790 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63f3fff640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e5ce70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f63e40eb840 tx=0x7f63e40baa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9389c0de2385bd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63d37fe640 1 -- 172.21.15.44:0/1082387324 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40ba2f0 con 0x557de9e52790 2024-09-13T17:34:23.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63d37fe640 1 -- 172.21.15.44:0/1082387324 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f63e40bb300 con 0x557de9e52790 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de9ddfb80 con 0x557de9e52790 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63d37fe640 1 -- 172.21.15.44:0/1082387324 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e40bac00 con 0x557de9e52790 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 10 client.18003 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557de9db43c0 con 0x557de9e52790 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63d37fe640 1 -- 172.21.15.44:0/1082387324 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557de9ddfb80 con 0x557de9e52790 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63d37fe640 1 -- 172.21.15.44:0/1082387324 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63e40c2bf0 con 0x557de9e52790 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 10 client.18003 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 20 client.18003 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9e6cda0 0x557de9e6f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 4 client.18003 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.154+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557de9da81f0 con 0x557de9e6cda0 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.155+0000 7f63ebfff640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9e6cda0 0x557de9e6f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.155+0000 7f63d37fe640 10 client.18003 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.155+0000 7f63ebfff640 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9e6cda0 0x557de9e6f190 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a89a95b9d766c10 server_cookie=5fafafba0fb6a429 in_seq=0 out_seq=0 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63d37fe640 1 -- 172.21.15.44:0/1082387324 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557de9da81f0 con 0x557de9e6cda0 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63d37fe640 10 client.18003 handle_command_reply: tid=0 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 client.18003 shutdown 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 msgr2=0x557de9e5ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e5ce70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f63e40eb840 tx=0x7f63e40baa90 comp rx=0 tx=0).stop 2024-09-13T17:34:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 shutdown_connections 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9e6cda0 0x557de9e6f190 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e5d950 0x557de9e5a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e530b0 0x557de9e5d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/1082387324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e5ce70 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63d37fe640 0 client.18003 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 >> 172.21.15.44:0/1082387324 conn(0x557de9e4ed00 msgr2=0x557de9e4deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 shutdown_connections 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 1 -- 172.21.15.44:0/1082387324 wait complete. 2024-09-13T17:34:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.156+0000 7f63fd5ce740 20 client.18003 trim_cache size 0 max 16384 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.167+0000 7f63fd5ce740 1 Processor -- start 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.167+0000 7f63fd5ce740 1 -- start start 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.167+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e55a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.167+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e4f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e64620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9ddfb80 con 0x557de9e52790 2024-09-13T17:34:23.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9e4a6b0 con 0x557de9e5d950 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9e69f60 con 0x557de9e530b0 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e55a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e55a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36384/0 (socket says 172.21.15.44:36384) 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 -- 172.21.15.44:0/3058067950 learned_addr learned my addr 172.21.15.44:0/3058067950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63ebfff640 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e4f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 -- 172.21.15.44:0/3058067950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 msgr2=0x557de9e4f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e4f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 -- 172.21.15.44:0/3058067950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 msgr2=0x557de9e64620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e64620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 -- 172.21.15.44:0/3058067950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de9d56770 con 0x557de9e52790 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63ebfff640 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63f3fff640 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e55a70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f63e40ba900 tx=0x7f63e4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2175bffeed4960af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63eaffd640 1 -- 172.21.15.44:0/3058067950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40f3e70 con 0x557de9e52790 2024-09-13T17:34:23.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63eaffd640 1 -- 172.21.15.44:0/3058067950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63e40bb300 con 0x557de9e52790 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.168+0000 7f63eaffd640 1 -- 172.21.15.44:0/3058067950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40bea30 con 0x557de9e52790 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 -- 172.21.15.44:0/3058067950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 msgr2=0x557de9e55a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e55a70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f63e40ba900 tx=0x7f63e4004e20 comp rx=0 tx=0).stop 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 -- 172.21.15.44:0/3058067950 shutdown_connections 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e530b0 0x557de9e4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e64620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/3058067950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e52790 0x557de9e55a70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 -- 172.21.15.44:0/3058067950 >> 172.21.15.44:0/3058067950 conn(0x557de9e4ed00 msgr2=0x557de9e55360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 -- 172.21.15.44:0/3058067950 shutdown_connections 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 -- 172.21.15.44:0/3058067950 wait complete. 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 -- start start 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 Processor -- start 2024-09-13T17:34:23.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e52790 0x557de9e63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 0x557de9e63750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.169+0000 7f63fd5ce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9ec3300 con 0x557de9e530b0 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9ec3470 con 0x557de9e5d950 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9ec3630 con 0x557de9e52790 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 10 client.0 _ll_register_callbacks cb 0x7f63ec009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 0x557de9e63750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 0x557de9e63750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36390/0 (socket says 172.21.15.44:36390) 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 -- 172.21.15.44:0/2357940361 learned_addr learned my addr 172.21.15.44:0/2357940361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63f3fff640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e52790 0x557de9e63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63f8e47640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 -- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e52790 msgr2=0x557de9e63210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e52790 0x557de9e63210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 -- 172.21.15.44:0/2357940361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 msgr2=0x557de9e63c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e63c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 -- 172.21.15.44:0/2357940361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557de9db43c0 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63f8e47640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e63c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63ebfff640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 0x557de9e63750 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0058c0 tx=0x7f63dc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e16d8682358cab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63d37fe640 1 -- 172.21.15.44:0/2357940361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc017040 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63d37fe640 1 -- 172.21.15.44:0/2357940361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f63dc020450 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63ec01f070 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63d37fe640 1 -- 172.21.15.44:0/2357940361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63dc020610 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 10 client.18009 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.170+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557de9d7be80 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63d37fe640 1 -- 172.21.15.44:0/2357940361 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63ec01f070 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63d37fe640 1 -- 172.21.15.44:0/2357940361 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63dc01f400 con 0x557de9e530b0 2024-09-13T17:34:23.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63fd5ce740 10 client.18009 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63fd5ce740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63fd5ce740 20 client.18009 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9ec4e70 0x557de9ec7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63fd5ce740 4 client.18009 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557de9aa94e0 con 0x557de9ec4e70 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63f3fff640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9ec4e70 0x557de9ec7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63f3fff640 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9ec4e70 0x557de9ec7290 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ce1f4c899be5cd server_cookie=be0b2ef42845804e in_seq=0 out_seq=0 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.171+0000 7f63d37fe640 10 client.18009 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63d37fe640 1 -- 172.21.15.44:0/2357940361 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557de9aa94e0 con 0x557de9ec4e70 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63d37fe640 10 client.18009 handle_command_reply: tid=0 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 client.18009 shutdown 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 msgr2=0x557de9e63750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 0x557de9e63750 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0058c0 tx=0x7f63dc012820 comp rx=0 tx=0).stop 2024-09-13T17:34:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 shutdown_connections 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557de9ec4e70 0x557de9ec7290 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557de9e5d950 0x557de9e63c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de9e530b0 0x557de9e63750 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63d37fe640 0 client.18009 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 --2- 172.21.15.44:0/2357940361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557de9e52790 0x557de9e63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 >> 172.21.15.44:0/2357940361 conn(0x557de9e4ed00 msgr2=0x557de9e541f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 shutdown_connections 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 1 -- 172.21.15.44:0/2357940361 wait complete. 2024-09-13T17:34:23.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fd5ce740 20 client.18009 trim_cache size 0 max 16384 2024-09-13T17:34:23.181 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:34:23.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63e00a1280 msgr2=0x7f63e00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63e00a1280 0x7f63e00a3740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63ec022000 tx=0x7f63ec023000 comp rx=0 tx=0).stop 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 msgr2=0x7f63f4196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.172+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f4196230 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f63e402a940 tx=0x7f63e4005530 comp rx=0 tx=0).stop 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 shutdown_connections 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f63e00a1280 0x7f63e00a3740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f419ab60 0x7f63f419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63f4103fd0 0x7f63f4196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 --2- 172.21.15.44:0/2752560754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63f40fee80 0x7f63f4196230 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 >> 172.21.15.44:0/2752560754 conn(0x7f63f40fa930 msgr2=0x7f63f40fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 shutdown_connections 2024-09-13T17:34:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:23.173+0000 7f63fa8aa640 1 -- 172.21.15.44:0/2752560754 wait complete. 2024-09-13T17:34:23.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:23 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/864850164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:34:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:23 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/864850164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:34:23.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:23 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/864850164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:34:24.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:34:24.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:34:24.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 -- 172.21.15.44:0/3438895287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128100e50 msgr2=0x7f3128101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 --2- 172.21.15.44:0/3438895287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128100e50 0x7f3128101230 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3118009900 tx=0x7f311802a410 comp rx=0 tx=0).stop 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 -- 172.21.15.44:0/3438895287 shutdown_connections 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 --2- 172.21.15.44:0/3438895287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3128101800 0x7f3128109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 --2- 172.21.15.44:0/3438895287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f312810a350 0x7f312810c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 --2- 172.21.15.44:0/3438895287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128100e50 0x7f3128101230 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 -- 172.21.15.44:0/3438895287 >> 172.21.15.44:0/3438895287 conn(0x7f31280fa930 msgr2=0x7f31280fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 -- 172.21.15.44:0/3438895287 shutdown_connections 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 -- 172.21.15.44:0/3438895287 wait complete. 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 Processor -- start 2024-09-13T17:34:24.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.307+0000 7f312e976640 1 -- start start 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312e976640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3128100e50 0x7f3128196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312e976640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 0x7f3128196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312e976640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312810a350 0x7f312819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312e976640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312819b060 con 0x7f3128101800 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312e976640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312819b1d0 con 0x7f3128100e50 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312e976640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312819b390 con 0x7f312810a350 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 0x7f3128196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f3127fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3128100e50 0x7f3128196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 0x7f3128196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36410/0 (socket says 172.21.15.44:36410) 2024-09-13T17:34:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 -- 172.21.15.44:0/513100642 learned_addr learned my addr 172.21.15.44:0/513100642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f312cf13640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312810a350 0x7f312819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 -- 172.21.15.44:0/513100642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312810a350 msgr2=0x7f312819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312810a350 0x7f312819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 -- 172.21.15.44:0/513100642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3128100e50 msgr2=0x7f3128196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3128100e50 0x7f3128196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 -- 172.21.15.44:0/513100642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009560 con 0x7f3128101800 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.308+0000 7f31277fe640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 0x7f3128196670 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f311c00d770 tx=0x7f311c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.309+0000 7f31257fa640 1 -- 172.21.15.44:0/513100642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c009c30 con 0x7f3128101800 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.309+0000 7f31257fa640 1 -- 172.21.15.44:0/513100642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f311c00de80 con 0x7f3128101800 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.309+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312819b670 con 0x7f3128101800 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.309+0000 7f31257fa640 1 -- 172.21.15.44:0/513100642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c004d40 con 0x7f3128101800 2024-09-13T17:34:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.309+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31281a2ef0 con 0x7f3128101800 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.311+0000 7f31257fa640 1 -- 172.21.15.44:0/513100642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f311c004ea0 con 0x7f3128101800 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.311+0000 7f313169c740 1 Processor -- start 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.311+0000 7f313169c740 1 -- start start 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f313169c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f313169c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc72850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f313169c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc75180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f313169c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cbcec60 con 0x55e32cc63fa0 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f313169c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cbcc3f0 con 0x55e32cc63680 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f313169c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cbc7ce0 con 0x55e32cc72d90 2024-09-13T17:34:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f31257fa640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31100a1200 0x7f31100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc72850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc72850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36418/0 (socket says 172.21.15.44:36418) 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 -- 172.21.15.44:0/1278760636 learned_addr learned my addr 172.21.15.44:0/1278760636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f312cf13640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31100a1200 0x7f31100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f312cf13640 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f31257fa640 1 -- 172.21.15.44:0/513100642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f311c0cca70 con 0x7f3128101800 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f31277fe640 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc75180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 -- 172.21.15.44:0/1278760636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 msgr2=0x55e32cc75180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc75180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 -- 172.21.15.44:0/1278760636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 msgr2=0x55e32cc63a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc63a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 -- 172.21.15.44:0/1278760636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e32cb83970 con 0x55e32cc63fa0 2024-09-13T17:34:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f3127fff640 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc72850 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f311802a930 tx=0x7f3118005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447d961a901e4003 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.312+0000 7f312cf13640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31100a1200 0x7f31100a36c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31200098b0 tx=0x7f3120009840 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f310e7fc640 1 -- 172.21.15.44:0/1278760636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118004230 con 0x55e32cc63fa0 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f310e7fc640 1 -- 172.21.15.44:0/1278760636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3118031df0 con 0x55e32cc63fa0 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f310e7fc640 1 -- 172.21.15.44:0/1278760636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118003bf0 con 0x55e32cc63fa0 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 -- 172.21.15.44:0/1278760636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 msgr2=0x55e32cc72850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc72850 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f311802a930 tx=0x7f3118005c00 comp rx=0 tx=0).stop 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 -- 172.21.15.44:0/1278760636 shutdown_connections 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc75180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc72850 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 --2- 172.21.15.44:0/1278760636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc63a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 -- 172.21.15.44:0/1278760636 >> 172.21.15.44:0/1278760636 conn(0x55e32cc5fd50 msgr2=0x55e32cc60130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 -- 172.21.15.44:0/1278760636 shutdown_connections 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.313+0000 7f313169c740 1 -- 172.21.15.44:0/1278760636 wait complete. 2024-09-13T17:34:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 -- start start 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 Processor -- start 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 0x55e32cc727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc6fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cb85f80 con 0x55e32cc63680 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cb85c20 con 0x55e32cc63fa0 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cb52ab0 con 0x55e32cc72d90 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f313169c740 10 client.0 _ll_register_callbacks cb 0x55e32cbc1350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f3127fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 0x55e32cc727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f3127fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 0x55e32cc727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36422/0 (socket says 172.21.15.44:36422) 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.314+0000 7f3127fff640 1 -- 172.21.15.44:0/3954534605 learned_addr learned my addr 172.21.15.44:0/3954534605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f31277fe640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc6fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f312cf13640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f3127fff640 1 -- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 msgr2=0x55e32cc70050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f3127fff640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc70050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f3127fff640 1 -- 172.21.15.44:0/3954534605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 msgr2=0x55e32cc6fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f3127fff640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc6fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f3127fff640 1 -- 172.21.15.44:0/3954534605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312810d420 con 0x55e32cc63680 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f3127fff640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 0x55e32cc727a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3118031860 tx=0x7f311802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67df5fa6d4858c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f31077fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f31077fe640 1 -- 172.21.15.44:0/3954534605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311803d650 con 0x55e32cc63680 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f31077fe640 1 -- 172.21.15.44:0/3954534605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f311804c070 con 0x55e32cc63680 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e32c8bb590 con 0x55e32cc63680 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f31077fe640 1 -- 172.21.15.44:0/3954534605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f311803d7b0 con 0x55e32cc63680 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f313169c740 10 client.18021 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.315+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e32cbc6360 con 0x55e32cc63680 2024-09-13T17:34:24.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f31077fe640 1 -- 172.21.15.44:0/3954534605 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e32c8bb590 con 0x55e32cc63680 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f31077fe640 1 -- 172.21.15.44:0/3954534605 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3118043020 con 0x55e32cc63680 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f313169c740 10 client.18021 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f313169c740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f313169c740 20 client.18021 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f313169c740 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32cc827d0 0x55e32cc84bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f313169c740 4 client.18021 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e32c8cc860 con 0x55e32cc827d0 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f31277fe640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32cc827d0 0x55e32cc84bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f31277fe640 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32cc827d0 0x55e32cc84bc0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4542e343bfa289a7 server_cookie=6e465e7ae2b1f9ea in_seq=0 out_seq=0 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.316+0000 7f31077fe640 10 client.18021 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f31077fe640 1 -- 172.21.15.44:0/3954534605 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e32c8cc860 con 0x55e32cc827d0 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f31077fe640 10 client.18021 handle_command_reply: tid=0 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 client.18021 shutdown 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 msgr2=0x55e32cc727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 0x55e32cc727a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3118031860 tx=0x7f311802a410 comp rx=0 tx=0).stop 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 shutdown_connections 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32cc827d0 0x55e32cc84bc0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f31077fe640 0 client.18021 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc6fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 --2- 172.21.15.44:0/3954534605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63680 0x55e32cc727a0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 >> 172.21.15.44:0/3954534605 conn(0x55e32cc5fd50 msgr2=0x55e32cc5f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 shutdown_connections 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.317+0000 7f313169c740 1 -- 172.21.15.44:0/3954534605 wait complete. 2024-09-13T17:34:24.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.318+0000 7f313169c740 20 client.18021 trim_cache size 0 max 16384 2024-09-13T17:34:24.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 Processor -- start 2024-09-13T17:34:24.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 -- start start 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc6f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc6f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cc89a50 con 0x55e32cc63fa0 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cc64d70 con 0x55e32cc63680 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f313169c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cc7f210 con 0x55e32cc72d90 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f31277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc6f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f31277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc6f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36438/0 (socket says 172.21.15.44:36438) 2024-09-13T17:34:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f31277fe640 1 -- 172.21.15.44:0/114237980 learned_addr learned my addr 172.21.15.44:0/114237980 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f31277fe640 1 -- 172.21.15.44:0/114237980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 msgr2=0x55e32cc6f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.332+0000 7f31277fe640 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc6f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f31277fe640 1 -- 172.21.15.44:0/114237980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 msgr2=0x55e32cc722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f31277fe640 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f31277fe640 1 -- 172.21.15.44:0/114237980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e32cb681c0 con 0x55e32cc63fa0 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f31277fe640 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc6f2e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f311c0962d0 tx=0x7f311c0954c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78b5035b8534903 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f310effd640 1 -- 172.21.15.44:0/114237980 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b2170 con 0x55e32cc63fa0 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f310effd640 1 -- 172.21.15.44:0/114237980 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f311c0a1c30 con 0x55e32cc63fa0 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f310effd640 1 -- 172.21.15.44:0/114237980 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b90b0 con 0x55e32cc63fa0 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 -- 172.21.15.44:0/114237980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 msgr2=0x55e32cc6f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc6f2e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f311c0962d0 tx=0x7f311c0954c0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 -- 172.21.15.44:0/114237980 shutdown_connections 2024-09-13T17:34:24.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc72d90 0x55e32cc6f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc63fa0 0x55e32cc6f2e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 --2- 172.21.15.44:0/114237980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63680 0x55e32cc722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 -- 172.21.15.44:0/114237980 >> 172.21.15.44:0/114237980 conn(0x55e32cc5fd50 msgr2=0x55e32cc6e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 -- 172.21.15.44:0/114237980 shutdown_connections 2024-09-13T17:34:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.333+0000 7f313169c740 1 -- 172.21.15.44:0/114237980 wait complete. 2024-09-13T17:34:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 -- start start 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 Processor -- start 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc63680 0x55e32cc79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 0x55e32cc77a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cc77f40 con 0x55e32cc72d90 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cc780b0 con 0x55e32cc63fa0 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cc78270 con 0x55e32cc63680 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f313169c740 10 client.0 _ll_register_callbacks cb 0x7f31200126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f312cf13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 0x55e32cc77a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f312cf13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 0x55e32cc77a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36446/0 (socket says 172.21.15.44:36446) 2024-09-13T17:34:24.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.334+0000 7f312cf13640 1 -- 172.21.15.44:0/1899152644 learned_addr learned my addr 172.21.15.44:0/1899152644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f3127fff640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc63680 0x55e32cc79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f31277fe640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f312cf13640 1 -- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc63680 msgr2=0x55e32cc79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f312cf13640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc63680 0x55e32cc79a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f312cf13640 1 -- 172.21.15.44:0/1899152644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 msgr2=0x55e32cc79f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f312cf13640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc79f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f312cf13640 1 -- 172.21.15.44:0/1899152644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31281a2ef0 con 0x55e32cc72d90 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f312cf13640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 0x55e32cc77a00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3120012300 tx=0x7f3120009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da076451ce7c65d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f31077fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f31077fe640 1 -- 172.21.15.44:0/1899152644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120010e60 con 0x55e32cc72d90 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f31077fe640 1 -- 172.21.15.44:0/1899152644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f3120003d40 con 0x55e32cc72d90 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3118031df0 con 0x55e32cc72d90 2024-09-13T17:34:24.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f31077fe640 1 -- 172.21.15.44:0/1899152644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3120004b20 con 0x55e32cc72d90 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f313169c740 10 client.18027 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e32cb8d990 con 0x55e32cc72d90 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.335+0000 7f31077fe640 1 -- 172.21.15.44:0/1899152644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3118031df0 con 0x55e32cc72d90 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f31077fe640 1 -- 172.21.15.44:0/1899152644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31200295e0 con 0x55e32cc72d90 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f313169c740 10 client.18027 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f313169c740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f313169c740 20 client.18027 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f313169c740 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32ccdace0 0x55e32ccdd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f313169c740 4 client.18027 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e32c91feb0 con 0x55e32ccdace0 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f3127fff640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32ccdace0 0x55e32ccdd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f31077fe640 10 client.18027 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.336+0000 7f3127fff640 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32ccdace0 0x55e32ccdd100 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9780660cff588db4 server_cookie=b02beebf67e1c3c1 in_seq=0 out_seq=0 2024-09-13T17:34:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.337+0000 7f31077fe640 1 -- 172.21.15.44:0/1899152644 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e32c91feb0 con 0x55e32ccdace0 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.337+0000 7f31077fe640 10 client.18027 handle_command_reply: tid=0 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.337+0000 7f313169c740 1 client.18027 shutdown 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.337+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 msgr2=0x55e32cc77a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.337+0000 7f313169c740 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 0x55e32cc77a00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3120012300 tx=0x7f3120009d90 comp rx=0 tx=0).stop 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 shutdown_connections 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e32ccdace0 0x55e32ccdd100 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e32cc72d90 0x55e32cc77a00 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e32cc63fa0 0x55e32cc79f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 --2- 172.21.15.44:0/1899152644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e32cc63680 0x55e32cc79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 >> 172.21.15.44:0/1899152644 conn(0x55e32cc5fd50 msgr2=0x55e32cc6f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f31077fe640 0 client.18027 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 shutdown_connections 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 1 -- 172.21.15.44:0/1899152644 wait complete. 2024-09-13T17:34:24.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f313169c740 20 client.18027 trim_cache size 0 max 16384 2024-09-13T17:34:24.345 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:34:24.345 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:34:24.345 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "ba559051-337d-4997-b880-671e28c30749", 2024-09-13T17:34:24.345 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:34:24.345 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:34:24.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31100a1200 msgr2=0x7f31100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31100a1200 0x7f31100a36c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31200098b0 tx=0x7f3120009840 comp rx=0 tx=0).stop 2024-09-13T17:34:24.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 msgr2=0x7f3128196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 0x7f3128196670 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f311c00d770 tx=0x7f311c00dc30 comp rx=0 tx=0).stop 2024-09-13T17:34:24.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 shutdown_connections 2024-09-13T17:34:24.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f31100a1200 0x7f31100a36c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312810a350 0x7f312819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3128101800 0x7f3128196670 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 --2- 172.21.15.44:0/513100642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3128100e50 0x7f3128196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 >> 172.21.15.44:0/513100642 conn(0x7f31280fa930 msgr2=0x7f31280fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.338+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 shutdown_connections 2024-09-13T17:34:24.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.339+0000 7f312e976640 1 -- 172.21.15.44:0/513100642 wait complete. 2024-09-13T17:34:24.350 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:34:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 -- 172.21.15.44:0/2287037726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 msgr2=0x7fdfe810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 --2- 172.21.15.44:0/2287037726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810bb00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009cd0 tx=0x7fdfe002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 -- 172.21.15.44:0/2287037726 shutdown_connections 2024-09-13T17:34:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 --2- 172.21.15.44:0/2287037726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810bb00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 --2- 172.21.15.44:0/2287037726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfe80ff810 0x7fdfe8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 --2- 172.21.15.44:0/2287037726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe80fee60 0x7fdfe80ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 -- 172.21.15.44:0/2287037726 >> 172.21.15.44:0/2287037726 conn(0x7fdfe8075c60 msgr2=0x7fdfe80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 -- 172.21.15.44:0/2287037726 shutdown_connections 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.454+0000 7fdfee041640 1 -- 172.21.15.44:0/2287037726 wait complete. 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 Processor -- start 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 -- start start 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe80fee60 0x7fdfe810e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfe810f350 0x7fdfe81a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe81a4910 con 0x7fdfe8103fb0 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe81a4a80 con 0x7fdfe810f350 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfee041640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe81a4c40 con 0x7fdfe80fee60 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36460/0 (socket says 172.21.15.44:36460) 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2679955689 learned_addr learned my addr 172.21.15.44:0/2679955689 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe80fee60 0x7fdfe810e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfe810f350 0x7fdfe81a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2679955689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe80fee60 msgr2=0x7fdfe810e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe80fee60 0x7fdfe810e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2679955689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfe810f350 msgr2=0x7fdfe81a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfe810f350 0x7fdfe81a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.455+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2679955689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009900 con 0x7fdfe8103fb0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.456+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810ee10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c970 tx=0x7fdfdc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.456+0000 7fdfe4ff9640 1 -- 172.21.15.44:0/2679955689 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc004440 con 0x7fdfe8103fb0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.456+0000 7fdfe4ff9640 1 -- 172.21.15.44:0/2679955689 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfdc007cf0 con 0x7fdfe8103fb0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.456+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe81a4f20 con 0x7fdfe8103fb0 2024-09-13T17:34:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.456+0000 7fdfe4ff9640 1 -- 172.21.15.44:0/2679955689 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00f450 con 0x7fdfe8103fb0 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.456+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe81a53f0 con 0x7fdfe8103fb0 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 Processor -- start 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdfe4ff9640 1 -- 172.21.15.44:0/2679955689 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdfdc007860 con 0x7fdfe8103fb0 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 -- start start 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fb21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fbca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fbf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9f1cf20 con 0x55e2a9fb2730 2024-09-13T17:34:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9f1a6b0 con 0x55e2a9fb1e10 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.458+0000 7fdff0e42740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9f15fa0 con 0x55e2a9fbcfd0 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fbca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fbca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36476/0 (socket says 172.21.15.44:36476) 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3098828976 learned_addr learned my addr 172.21.15.44:0/3098828976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fb21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fbf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3098828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 msgr2=0x55e2a9fbf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fbf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3098828976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 msgr2=0x55e2a9fb21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fb21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3098828976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a9ed0840 con 0x55e2a9fb2730 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe4ff9640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdfd00a1280 0x7fdfd00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fbca90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800b760 tx=0x7fdfd800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee397828452469e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdfd00a1280 0x7fdfd00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfe4ff9640 1 -- 172.21.15.44:0/2679955689 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fdfdc0ccfe0 con 0x7fdfe8103fb0 2024-09-13T17:34:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfcdffb640 1 -- 172.21.15.44:0/3098828976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8003dd0 con 0x55e2a9fb2730 2024-09-13T17:34:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfcdffb640 1 -- 172.21.15.44:0/3098828976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfd800be70 con 0x55e2a9fb2730 2024-09-13T17:34:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.459+0000 7fdfcdffb640 1 -- 172.21.15.44:0/3098828976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004e40 con 0x55e2a9fb2730 2024-09-13T17:34:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 -- 172.21.15.44:0/3098828976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 msgr2=0x55e2a9fbca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdfd00a1280 0x7fdfd00a3740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009ca0 tx=0x7fdfe0005c50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fbca90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800b760 tx=0x7fdfd800bc20 comp rx=0 tx=0).stop 2024-09-13T17:34:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 -- 172.21.15.44:0/3098828976 shutdown_connections 2024-09-13T17:34:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fbf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fbca90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3098828976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fb21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 -- 172.21.15.44:0/3098828976 >> 172.21.15.44:0/3098828976 conn(0x55e2a9fae380 msgr2=0x55e2a9fae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 -- 172.21.15.44:0/3098828976 shutdown_connections 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 -- 172.21.15.44:0/3098828976 wait complete. 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.460+0000 7fdff0e42740 1 -- start start 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 Processor -- start 2024-09-13T17:34:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9eb72a0 con 0x55e2a9fb1e10 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9bee430 con 0x55e2a9fb2730 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9e99680 con 0x55e2a9fbcfd0 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdff0e42740 10 client.0 _ll_register_callbacks cb 0x55e2a9f0f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36482/0 (socket says 172.21.15.44:36482) 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3789863421 learned_addr learned my addr 172.21.15.44:0/3789863421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 msgr2=0x55e2a9fba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fba220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3789863421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb2730 msgr2=0x55e2a9fb9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.461+0000 7fdfe77fe640 1 -- 172.21.15.44:0/3789863421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd8004370 con 0x55e2a9fb1e10 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc9e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80039c0 tx=0x7fdfd8003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2da5af8cfb61cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/3789863421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004890 con 0x55e2a9fb1e10 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/3789863421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fdfd8011650 con 0x55e2a9fb1e10 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2a9c094e0 con 0x55e2a9fb1e10 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/3789863421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfd8011810 con 0x55e2a9fb1e10 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 10 client.18039 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e2a9f14620 con 0x55e2a9fb1e10 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/3789863421 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e2a9c094e0 con 0x55e2a9fb1e10 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/3789863421 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfd80259d0 con 0x55e2a9fb1e10 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 10 client.18039 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 20 client.18039 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2a9fccd60 0x55e2a9fcf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 4 client.18039 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.462+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2a9c1a7e0 con 0x55e2a9fccd60 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.463+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2a9fccd60 0x55e2a9fcf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.463+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2a9fccd60 0x55e2a9fcf150 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197d6ea28de20ed server_cookie=9d4443fde4890e48 in_seq=0 out_seq=0 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.463+0000 7fdfc6ffd640 10 client.18039 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/3789863421 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e2a9c1a7e0 con 0x55e2a9fccd60 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdfc6ffd640 10 client.18039 handle_command_reply: tid=0 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 client.18039 shutdown 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 msgr2=0x55e2a9fbc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc9e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80039c0 tx=0x7fdfd8003e00 comp rx=0 tx=0).stop 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 shutdown_connections 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2a9fccd60 0x55e2a9fcf150 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3789863421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc9e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 >> 172.21.15.44:0/3789863421 conn(0x55e2a9fae380 msgr2=0x55e2a9fb40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdfc6ffd640 0 client.18039 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 shutdown_connections 2024-09-13T17:34:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 1 -- 172.21.15.44:0/3789863421 wait complete. 2024-09-13T17:34:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.464+0000 7fdff0e42740 20 client.18039 trim_cache size 0 max 16384 2024-09-13T17:34:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.478+0000 7fdff0e42740 1 Processor -- start 2024-09-13T17:34:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.478+0000 7fdff0e42740 1 -- start start 2024-09-13T17:34:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fb5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdff0e42740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9fc94f0 con 0x55e2a9fb2730 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdff0e42740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9fbb150 con 0x55e2a9fbcfd0 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdff0e42740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a9fb2e00 con 0x55e2a9fb1e10 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36494/0 (socket says 172.21.15.44:36494) 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/3396737304 learned_addr learned my addr 172.21.15.44:0/3396737304 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fb5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/3396737304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fb1e10 msgr2=0x55e2a9fbc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/3396737304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fbcfd0 msgr2=0x55e2a9fb5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fb5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/3396737304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a9eb6790 con 0x55e2a9fb2730 2024-09-13T17:34:24.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fb5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb4ef0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc099d10 tx=0x7fdfdc00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6984e4018268d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfce7fc640 1 -- 172.21.15.44:0/3396737304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0998c0 con 0x55e2a9fb2730 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfce7fc640 1 -- 172.21.15.44:0/3396737304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfdc0996e0 con 0x55e2a9fb2730 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.479+0000 7fdfce7fc640 1 -- 172.21.15.44:0/3396737304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc09dbb0 con 0x55e2a9fb2730 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 -- 172.21.15.44:0/3396737304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 msgr2=0x55e2a9fb4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb4ef0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc099d10 tx=0x7fdfdc00af70 comp rx=0 tx=0).stop 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 -- 172.21.15.44:0/3396737304 shutdown_connections 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fbcfd0 0x55e2a9fb5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fb4ef0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 --2- 172.21.15.44:0/3396737304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 -- 172.21.15.44:0/3396737304 >> 172.21.15.44:0/3396737304 conn(0x55e2a9fae380 msgr2=0x55e2a9fae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 -- 172.21.15.44:0/3396737304 shutdown_connections 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 -- 172.21.15.44:0/3396737304 wait complete. 2024-09-13T17:34:24.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.480+0000 7fdff0e42740 1 -- start start 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 Processor -- start 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fc0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2aa0221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa022730 con 0x55e2a9fb2730 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa022870 con 0x55e2a9fb1e10 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa0229e0 con 0x55e2a9fbcfd0 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdff0e42740 10 client.0 _ll_register_callbacks cb 0x7fdfe0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fc0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fc0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36498/0 (socket says 172.21.15.44:36498) 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2101070596 learned_addr learned my addr 172.21.15.44:0/2101070596 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe7fff640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2aa0221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 msgr2=0x55e2aa0221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2aa0221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2101070596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 msgr2=0x55e2a9fbff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 -- 172.21.15.44:0/2101070596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe81a53f0 con 0x55e2a9fb2730 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfe6ffd640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fc0440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc000c00 tx=0x7fdfdc099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629ffe12e1449641 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.481+0000 7fdfc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/2101070596 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc007b10 con 0x55e2a9fb2730 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/2101070596 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fdfdc09a480 con 0x55e2a9fb2730 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfd800be70 con 0x55e2a9fb2730 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/2101070596 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfdc0a2200 con 0x55e2a9fb2730 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 10 client.18045 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e2a9edbea0 con 0x55e2a9fb2730 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/2101070596 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfd800be70 con 0x55e2a9fb2730 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/2101070596 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfdc096490 con 0x55e2a9fb2730 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 10 client.18045 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 20 client.18045 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2aa023c80 0x55e2aa0260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 4 client.18045 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e2a9c6de30 con 0x55e2aa023c80 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.482+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2aa023c80 0x55e2aa0260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdfc6ffd640 10 client.18045 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdfe77fe640 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2aa023c80 0x55e2aa0260a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a23de668932ce5 server_cookie=1076063ea7501c1a in_seq=0 out_seq=0 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdfc6ffd640 1 -- 172.21.15.44:0/2101070596 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e2a9c6de30 con 0x55e2aa023c80 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdfc6ffd640 10 client.18045 handle_command_reply: tid=0 2024-09-13T17:34:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 client.18045 shutdown 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 msgr2=0x55e2a9fc0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fc0440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc000c00 tx=0x7fdfdc099930 comp rx=0 tx=0).stop 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 shutdown_connections 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e2aa023c80 0x55e2aa0260a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2a9fbcfd0 0x55e2aa0221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2a9fb2730 0x55e2a9fc0440 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdfc6ffd640 0 client.18045 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 --2- 172.21.15.44:0/2101070596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e2a9fb1e10 0x55e2a9fbff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 >> 172.21.15.44:0/2101070596 conn(0x55e2a9fae380 msgr2=0x55e2a9fae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 shutdown_connections 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 1 -- 172.21.15.44:0/2101070596 wait complete. 2024-09-13T17:34:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.483+0000 7fdff0e42740 20 client.18045 trim_cache size 0 max 16384 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: "ba559051-337d-4997-b880-671e28c30749": { 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "ba559051-337d-4997-b880-671e28c30749", 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:34:24.493 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:34:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdfd00a1280 msgr2=0x7fdfd00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdfd00a1280 0x7fdfd00a3740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009ca0 tx=0x7fdfe0005c50 comp rx=0 tx=0).stop 2024-09-13T17:34:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 msgr2=0x7fdfe810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810ee10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c970 tx=0x7fdfdc00ce30 comp rx=0 tx=0).stop 2024-09-13T17:34:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 shutdown_connections 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdfd00a1280 0x7fdfd00a3740 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfe810f350 0x7fdfe81a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8103fb0 0x7fdfe810ee10 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 --2- 172.21.15.44:0/2679955689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe80fee60 0x7fdfe810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 >> 172.21.15.44:0/2679955689 conn(0x7fdfe8075c60 msgr2=0x7fdfe8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 shutdown_connections 2024-09-13T17:34:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:24.484+0000 7fdfee041640 1 -- 172.21.15.44:0/2679955689 wait complete. 2024-09-13T17:34:24.495 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:24 smithi044 ceph-mon[31175]: pgmap v237: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-13T17:34:24.496 INFO:tasks.cephfs.filesystem:scrub status for tag:ba559051-337d-4997-b880-671e28c30749 - {'path': '/', 'tag': 'ba559051-337d-4997-b880-671e28c30749', 'options': 'recursive,force'} 2024-09-13T17:34:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:24 smithi070 ceph-mon[34502]: pgmap v237: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-13T17:34:24.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:24 smithi100 ceph-mon[35055]: pgmap v237: 129 pgs: 129 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-13T17:34:25.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:25 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:34:25.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:25 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:25.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:25 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:34:25.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:25 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:25.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:25 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:34:25.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:25 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:25.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:25 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:34:25.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:25 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:25 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:34:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:25 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:25 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:34:25.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:25 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:26 smithi044 ceph-mon[31175]: pgmap v238: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-13T17:34:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:26 smithi070 ceph-mon[34502]: pgmap v238: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-13T17:34:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:26 smithi100 ceph-mon[35055]: pgmap v238: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-13T17:34:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:28 smithi044 ceph-mon[31175]: pgmap v239: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-13T17:34:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:28 smithi070 ceph-mon[34502]: pgmap v239: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-13T17:34:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:28 smithi100 ceph-mon[35055]: pgmap v239: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-13T17:34:29.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:29 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:34:29.675 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:29 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:34:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:29 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:34:30.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:30 smithi044 ceph-mon[31175]: pgmap v240: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-13T17:34:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:30 smithi070 ceph-mon[34502]: pgmap v240: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-13T17:34:30.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:30 smithi100 ceph-mon[35055]: pgmap v240: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-13T17:34:32.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:32 smithi044 ceph-mon[31175]: pgmap v241: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:34:32.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:32 smithi070 ceph-mon[34502]: pgmap v241: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:34:32.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:32 smithi100 ceph-mon[35055]: pgmap v241: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:34:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 -- 172.21.15.44:0/1893875793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ff360 msgr2=0x7f17980ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 --2- 172.21.15.44:0/1893875793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ff360 0x7f17980ff7a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1780009900 tx=0x7f178002a410 comp rx=0 tx=0).stop 2024-09-13T17:34:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 -- 172.21.15.44:0/1893875793 shutdown_connections 2024-09-13T17:34:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 --2- 172.21.15.44:0/1893875793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17980ffce0 0x7f179810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 --2- 172.21.15.44:0/1893875793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ff360 0x7f17980ff7a0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 --2- 172.21.15.44:0/1893875793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798103bc0 0x7f1798103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 -- 172.21.15.44:0/1893875793 >> 172.21.15.44:0/1893875793 conn(0x7f1798075340 msgr2=0x7f17980fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 -- 172.21.15.44:0/1893875793 shutdown_connections 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.809+0000 7f179e7c7640 1 -- 172.21.15.44:0/1893875793 wait complete. 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 Processor -- start 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 -- start start 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17980ff360 0x7f1798194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 0x7f17981953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1798103bc0 0x7f17981958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981a4650 con 0x7f17980ffce0 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981a47c0 con 0x7f1798103bc0 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179e7c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981a4980 con 0x7f17980ff360 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f178ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 0x7f17981953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f178ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 0x7f17981953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55828/0 (socket says 172.21.15.44:55828) 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f178ffff640 1 -- 172.21.15.44:0/3821769130 learned_addr learned my addr 172.21.15.44:0/3821769130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f1797fff640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17980ff360 0x7f1798194e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.810+0000 7f179cd64640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1798103bc0 0x7f17981958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f178ffff640 1 -- 172.21.15.44:0/3821769130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17980ff360 msgr2=0x7f1798194e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f178ffff640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17980ff360 0x7f1798194e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f178ffff640 1 -- 172.21.15.44:0/3821769130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1798103bc0 msgr2=0x7f17981958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f178ffff640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1798103bc0 0x7f17981958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f178ffff640 1 -- 172.21.15.44:0/3821769130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009560 con 0x7f17980ffce0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f178ffff640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 0x7f17981953b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1780006010 tx=0x7f1780005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f1795ffb640 1 -- 172.21.15.44:0/3821769130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780004bd0 con 0x7f17980ffce0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f1795ffb640 1 -- 172.21.15.44:0/3821769130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1780004d30 con 0x7f17980ffce0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17981a4c00 con 0x7f17980ffce0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f1795ffb640 1 -- 172.21.15.44:0/3821769130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178003c890 con 0x7f17980ffce0 2024-09-13T17:34:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.811+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17981a50b0 con 0x7f17980ffce0 2024-09-13T17:34:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.813+0000 7f1795ffb640 1 -- 172.21.15.44:0/3821769130 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f17800035f0 con 0x7f17980ffce0 2024-09-13T17:34:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 Processor -- start 2024-09-13T17:34:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 -- start start 2024-09-13T17:34:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844e6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 0x5592844f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844f14f0 0x5592844f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559284451c30 con 0x5592844e6c50 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928444f3c0 con 0x5592844f14f0 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f17a15c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928444acb0 con 0x5592844e6330 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f1797fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 0x5592844f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f1795ffb640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f17840a1200 0x7f17840a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f1797fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 0x5592844f0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55844/0 (socket says 172.21.15.44:55844) 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f1797fff640 1 -- 172.21.15.44:0/260163109 learned_addr learned my addr 172.21.15.44:0/260163109 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f179cd64640 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f178ffff640 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844f14f0 0x5592844f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.814+0000 7f179cd64640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f17840a1200 0x7f17840a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1797fff640 1 -- 172.21.15.44:0/260163109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 msgr2=0x5592844e6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1797fff640 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844e6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1797fff640 1 -- 172.21.15.44:0/260163109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844f14f0 msgr2=0x5592844f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1797fff640 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844f14f0 0x5592844f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1797fff640 1 -- 172.21.15.44:0/260163109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592843f1910 con 0x5592844e6c50 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1795ffb640 1 -- 172.21.15.44:0/3821769130 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f17800ede30 con 0x7f17980ffce0 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f1797fff640 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 0x5592844f0fb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d20 tx=0x7f1788011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2705a8672cd00e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f179cd64640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f17840a1200 0x7f17840a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1790009fd0 tx=0x7f1790009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f178e7fc640 1 -- 172.21.15.44:0/260163109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788019630 con 0x5592844e6c50 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f178e7fc640 1 -- 172.21.15.44:0/260163109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1788019c20 con 0x5592844e6c50 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.815+0000 7f178e7fc640 1 -- 172.21.15.44:0/260163109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800eaf0 con 0x5592844e6c50 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 -- 172.21.15.44:0/260163109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 msgr2=0x5592844f0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 0x5592844f0fb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d20 tx=0x7f1788011810 comp rx=0 tx=0).stop 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 -- 172.21.15.44:0/260163109 shutdown_connections 2024-09-13T17:34:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844f14f0 0x5592844f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6c50 0x5592844f0fb0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 --2- 172.21.15.44:0/260163109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844e6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 -- 172.21.15.44:0/260163109 >> 172.21.15.44:0/260163109 conn(0x5592844e28c0 msgr2=0x5592844e2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 -- 172.21.15.44:0/260163109 shutdown_connections 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 -- 172.21.15.44:0/260163109 wait complete. 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 -- start start 2024-09-13T17:34:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.816+0000 7f17a15c2740 1 Processor -- start 2024-09-13T17:34:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844ee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 0x5592844ee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559284408e80 con 0x5592844f14f0 2024-09-13T17:34:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559284408b20 con 0x5592844e6c50 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592843ce260 con 0x5592844e6330 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17a15c2740 10 client.0 _ll_register_callbacks cb 0x559284403df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 0x5592844ee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 0x5592844ee7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55856/0 (socket says 172.21.15.44:55856) 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 -- 172.21.15.44:0/2717800806 learned_addr learned my addr 172.21.15.44:0/2717800806 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f1797fff640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f178ffff640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 -- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 msgr2=0x5592844f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 -- 172.21.15.44:0/2717800806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 msgr2=0x5592844ee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844ee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 -- 172.21.15.44:0/2717800806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1788009950 con 0x5592844f14f0 2024-09-13T17:34:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f179cd64640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 0x5592844ee7d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f179000ffd0 tx=0x7f1790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8acc5626af6160 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.817+0000 7f17777fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17777fe640 1 -- 172.21.15.44:0/2717800806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790004820 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17777fe640 1 -- 172.21.15.44:0/2717800806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f1790020480 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559284473680 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17777fe640 1 -- 172.21.15.44:0/2717800806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1790020640 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 10 client.18057 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559284449330 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17777fe640 1 -- 172.21.15.44:0/2717800806 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559284473680 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17777fe640 1 -- 172.21.15.44:0/2717800806 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f179001f730 con 0x5592844f14f0 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 10 client.18057 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 10 client.18057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 20 client.18057 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559284500f50 0x559284503370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 4 client.18057 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.818+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55928443e5d0 con 0x559284500f50 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.819+0000 7f1797fff640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559284500f50 0x559284503370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.819+0000 7f1797fff640 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559284500f50 0x559284503370 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68cf7509f4dda6a5 server_cookie=e3b6c306d8a46f17 in_seq=0 out_seq=0 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.819+0000 7f17777fe640 10 client.18057 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17777fe640 1 -- 172.21.15.44:0/2717800806 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55928443e5d0 con 0x559284500f50 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17777fe640 10 client.18057 handle_command_reply: tid=0 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 client.18057 shutdown 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 msgr2=0x5592844ee7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 0x5592844ee7d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f179000ffd0 tx=0x7f1790004a40 comp rx=0 tx=0).stop 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 shutdown_connections 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559284500f50 0x559284503370 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844f14f0 0x5592844ee7d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844ee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 --2- 172.21.15.44:0/2717800806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17777fe640 0 client.18057 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 >> 172.21.15.44:0/2717800806 conn(0x5592844e28c0 msgr2=0x5592844e1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 shutdown_connections 2024-09-13T17:34:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 1 -- 172.21.15.44:0/2717800806 wait complete. 2024-09-13T17:34:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.820+0000 7f17a15c2740 20 client.18057 trim_cache size 0 max 16384 2024-09-13T17:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 Processor -- start 2024-09-13T17:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 -- start start 2024-09-13T17:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 0x5592844f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592844ff7e0 con 0x5592844e28c0 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592844e3550 con 0x5592844e6c50 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f17a15c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592844f0b60 con 0x5592844e6330 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 0x5592844f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 0x5592844f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55872/0 (socket says 172.21.15.44:55872) 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 -- 172.21.15.44:0/741893682 learned_addr learned my addr 172.21.15.44:0/741893682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f178ffff640 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 -- 172.21.15.44:0/741893682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 msgr2=0x5592844f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f179cd64640 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 -- 172.21.15.44:0/741893682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 msgr2=0x5592844eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f1797fff640 1 -- 172.21.15.44:0/741893682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592843eb300 con 0x5592844e28c0 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.833+0000 7f179cd64640 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f1797fff640 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 0x5592844f81b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1788009cf0 tx=0x7f1788005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74900dc568dde6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f178effd640 1 -- 172.21.15.44:0/741893682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788029070 con 0x5592844e28c0 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f178effd640 1 -- 172.21.15.44:0/741893682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f178800e3b0 con 0x5592844e28c0 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f178effd640 1 -- 172.21.15.44:0/741893682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788019430 con 0x5592844e28c0 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 -- 172.21.15.44:0/741893682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 msgr2=0x5592844f81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 0x5592844f81b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1788009cf0 tx=0x7f1788005d30 comp rx=0 tx=0).stop 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 -- 172.21.15.44:0/741893682 shutdown_connections 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e6330 0x5592844f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 --2- 172.21.15.44:0/741893682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e28c0 0x5592844f81b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 -- 172.21.15.44:0/741893682 >> 172.21.15.44:0/741893682 conn(0x5592844f1050 msgr2=0x5592844dda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 -- 172.21.15.44:0/741893682 shutdown_connections 2024-09-13T17:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.834+0000 7f17a15c2740 1 -- 172.21.15.44:0/741893682 wait complete. 2024-09-13T17:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 -- start start 2024-09-13T17:34:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 Processor -- start 2024-09-13T17:34:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e28c0 0x5592844f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 0x5592844f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592844f61e0 con 0x5592844e6330 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592844f6350 con 0x5592844e6c50 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592844f6510 con 0x5592844e28c0 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f17a15c2740 10 client.0 _ll_register_callbacks cb 0x7f1788009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f178ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 0x5592844f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f178ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 0x5592844f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55880/0 (socket says 172.21.15.44:55880) 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f178ffff640 1 -- 172.21.15.44:0/3863753299 learned_addr learned my addr 172.21.15.44:0/3863753299 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.835+0000 7f179cd64640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f1797fff640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e28c0 0x5592844f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f178ffff640 1 -- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e28c0 msgr2=0x5592844f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f178ffff640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e28c0 0x5592844f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f178ffff640 1 -- 172.21.15.44:0/3863753299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 msgr2=0x5592844f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f178ffff640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f178ffff640 1 -- 172.21.15.44:0/3863753299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592843f1910 con 0x5592844e6330 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f178ffff640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 0x5592844f5760 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f17800b76f0 tx=0x7f1780089f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75f8d7cc708545c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17777fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17777fe640 1 -- 172.21.15.44:0/3863753299 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800bac80 con 0x5592844e6330 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17777fe640 1 -- 172.21.15.44:0/3863753299 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f17800bea00 con 0x5592844e6330 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1788019c20 con 0x5592844e6330 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17777fe640 1 -- 172.21.15.44:0/3863753299 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17800c3050 con 0x5592844e6330 2024-09-13T17:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17a15c2740 10 client.18063 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592844108e0 con 0x5592844e6330 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17777fe640 1 -- 172.21.15.44:0/3863753299 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1788019c20 con 0x5592844e6330 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.836+0000 7f17777fe640 1 -- 172.21.15.44:0/3863753299 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1780080be0 con 0x5592844e6330 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17a15c2740 10 client.18063 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17a15c2740 10 client.18063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17a15c2740 20 client.18063 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17a15c2740 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5592844f14f0 0x55928455b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17a15c2740 4 client.18063 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55928413e300 con 0x5592844f14f0 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f1797fff640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5592844f14f0 0x55928455b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f17777fe640 10 client.18063 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.837+0000 7f1797fff640 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5592844f14f0 0x55928455b560 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c471d6cf89d7a079 server_cookie=ea9b3a39bfae03e7 in_seq=0 out_seq=0 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17777fe640 1 -- 172.21.15.44:0/3863753299 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55928413e300 con 0x5592844f14f0 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17777fe640 10 client.18063 handle_command_reply: tid=0 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 client.18063 shutdown 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 msgr2=0x5592844f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 0x5592844f5760 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f17800b76f0 tx=0x7f1780089f40 comp rx=0 tx=0).stop 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 shutdown_connections 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5592844f14f0 0x55928455b560 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592844e6c50 0x5592844f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17777fe640 0 client.18063 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592844e6330 0x5592844f5760 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.849+0000 7f17a15c2740 1 --2- 172.21.15.44:0/3863753299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592844e28c0 0x5592844f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 >> 172.21.15.44:0/3863753299 conn(0x5592844f1050 msgr2=0x5592844ec9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 shutdown_connections 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f17a15c2740 1 -- 172.21.15.44:0/3863753299 wait complete. 2024-09-13T17:34:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f17a15c2740 20 client.18063 trim_cache size 0 max 16384 2024-09-13T17:34:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:34:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:34:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:34:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f17840a1200 msgr2=0x7f17840a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f17840a1200 0x7f17840a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1790009fd0 tx=0x7f1790009500 comp rx=0 tx=0).stop 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 msgr2=0x7f17981953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 0x7f17981953b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1780006010 tx=0x7f1780005620 comp rx=0 tx=0).stop 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 shutdown_connections 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f17840a1200 0x7f17840a36c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1798103bc0 0x7f17981958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17980ffce0 0x7f17981953b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 --2- 172.21.15.44:0/3821769130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17980ff360 0x7f1798194e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.850+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 >> 172.21.15.44:0/3821769130 conn(0x7f1798075340 msgr2=0x7f17980fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.851+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 shutdown_connections 2024-09-13T17:34:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:32.851+0000 7f179e7c7640 1 -- 172.21.15.44:0/3821769130 wait complete. 2024-09-13T17:34:34.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:34 smithi044 ceph-mon[31175]: pgmap v242: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:34:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:34 smithi070 ceph-mon[34502]: pgmap v242: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:34:34.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:34 smithi100 ceph-mon[35055]: pgmap v242: 129 pgs: 129 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:34:35.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:35 smithi070 ceph-mon[34502]: pgmap v243: 129 pgs: 129 active+clean; 1.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 22 op/s 2024-09-13T17:34:35.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:35 smithi100 ceph-mon[35055]: pgmap v243: 129 pgs: 129 active+clean; 1.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 22 op/s 2024-09-13T17:34:35.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:35 smithi044 ceph-mon[31175]: pgmap v243: 129 pgs: 129 active+clean; 1.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 22 op/s 2024-09-13T17:34:37.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:37 smithi044 ceph-mon[31175]: pgmap v244: 129 pgs: 129 active+clean; 1.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:37 smithi070 ceph-mon[34502]: pgmap v244: 129 pgs: 129 active+clean; 1.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:37 smithi100 ceph-mon[35055]: pgmap v244: 129 pgs: 129 active+clean; 1.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:34:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:39 smithi044 ceph-mon[31175]: pgmap v245: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:34:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:39 smithi070 ceph-mon[34502]: pgmap v245: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:34:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:39 smithi100 ceph-mon[35055]: pgmap v245: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:34:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:41 smithi044 ceph-mon[31175]: pgmap v246: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:34:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:41 smithi070 ceph-mon[34502]: pgmap v246: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:34:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:41 smithi100 ceph-mon[35055]: pgmap v246: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:34:43.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:43 smithi044 ceph-mon[31175]: pgmap v247: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:34:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:43 smithi070 ceph-mon[34502]: pgmap v247: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:34:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:43 smithi100 ceph-mon[35055]: pgmap v247: 129 pgs: 129 active+clean; 2.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:34:45.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:45 smithi044 ceph-mon[31175]: pgmap v248: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:34:45.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:45 smithi070 ceph-mon[34502]: pgmap v248: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:34:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 -- 172.21.15.44:0/1781249125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc00fdf90 msgr2=0x7f3fc00fe370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/1781249125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc00fdf90 0x7f3fc00fe370 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009900 tx=0x7f3fb002a400 comp rx=0 tx=0).stop 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 -- 172.21.15.44:0/1781249125 shutdown_connections 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/1781249125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc00fe940 0x7f3fc01049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/1781249125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc01050b0 0x7f3fc010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/1781249125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc00fdf90 0x7f3fc00fe370 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 -- 172.21.15.44:0/1781249125 >> 172.21.15.44:0/1781249125 conn(0x7f3fc00f9bd0 msgr2=0x7f3fc00fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 -- 172.21.15.44:0/1781249125 shutdown_connections 2024-09-13T17:34:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.948+0000 7f3fc49e2640 1 -- 172.21.15.44:0/1781249125 wait complete. 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.949+0000 7f3fc49e2640 1 Processor -- start 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.949+0000 7f3fc49e2640 1 -- start start 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.949+0000 7f3fc49e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc00fdf90 0x7f3fc0196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.949+0000 7f3fc49e2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 0x7f3fc01966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fc49e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc019aa70 0x7f3fc019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fc49e2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc019b460 con 0x7f3fc01050b0 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fc49e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc019b5d0 con 0x7f3fc00fdf90 2024-09-13T17:34:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fc49e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc019b740 con 0x7f3fc019aa70 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 0x7f3fc01966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 0x7f3fc01966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47270/0 (socket says 172.21.15.44:47270) 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/3454558331 learned_addr learned my addr 172.21.15.44:0/3454558331 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc00fdf90 0x7f3fc0196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc019aa70 0x7f3fc019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/3454558331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc019aa70 msgr2=0x7f3fc019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc019aa70 0x7f3fc019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/3454558331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc00fdf90 msgr2=0x7f3fc0196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc00fdf90 0x7f3fc0196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/3454558331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0009560 con 0x7f3fc01050b0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 0x7f3fc01966d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400d750 tx=0x7f3fb400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3faf7fe640 1 -- 172.21.15.44:0/3454558331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4009c30 con 0x7f3fc01050b0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc01a2d20 con 0x7f3fc01050b0 2024-09-13T17:34:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.950+0000 7f3faf7fe640 1 -- 172.21.15.44:0/3454558331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fb400de60 con 0x7f3fc01050b0 2024-09-13T17:34:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.951+0000 7f3faf7fe640 1 -- 172.21.15.44:0/3454558331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4004cf0 con 0x7f3fc01050b0 2024-09-13T17:34:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.951+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc01a31d0 con 0x7f3fc01050b0 2024-09-13T17:34:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3faf7fe640 1 -- 172.21.15.44:0/3454558331 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3fb4004e50 con 0x7f3fc01050b0 2024-09-13T17:34:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 Processor -- start 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 -- start start 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d772e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d773d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d773fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7699c30 con 0x5573d773d900 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d76973c0 con 0x5573d772e230 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.954+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7692cb0 con 0x5573d772eb50 2024-09-13T17:34:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d773fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3faf7fe640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fa80a1200 0x7f3fa80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d773fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47276/0 (socket says 172.21.15.44:47276) 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/1585316530 learned_addr learned my addr 172.21.15.44:0/1585316530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d772e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d773d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fa80a1200 0x7f3fa80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3faf7fe640 1 -- 172.21.15.44:0/3454558331 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f3fb40cc960 con 0x7f3fc01050b0 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/1585316530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 msgr2=0x5573d773d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d773d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/1585316530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 msgr2=0x5573d772e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d772e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/1585316530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d7639910 con 0x5573d773d900 2024-09-13T17:34:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d773fcf0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40994b0 tx=0x7f3fb4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4ceaf2b4785e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fa80a1200 0x7f3fa80a36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009990 tx=0x7f3fb80096a0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3facff9640 1 -- 172.21.15.44:0/1585316530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40997b0 con 0x5573d773d900 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3facff9640 1 -- 172.21.15.44:0/1585316530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fb409d530 con 0x5573d773d900 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.955+0000 7f3facff9640 1 -- 172.21.15.44:0/1585316530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4095ec0 con 0x5573d773d900 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 -- 172.21.15.44:0/1585316530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 msgr2=0x5573d773fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d773fcf0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40994b0 tx=0x7f3fb4005780 comp rx=0 tx=0).stop 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 -- 172.21.15.44:0/1585316530 shutdown_connections 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d773d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d773fcf0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/1585316530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d772e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 -- 172.21.15.44:0/1585316530 >> 172.21.15.44:0/1585316530 conn(0x5573d772a8c0 msgr2=0x5573d772aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 -- 172.21.15.44:0/1585316530 shutdown_connections 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.956+0000 7f3fc77d9740 1 -- 172.21.15.44:0/1585316530 wait complete. 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 -- start start 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 Processor -- start 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d773d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 0x5573d773a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d773ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7650e80 con 0x5573d772eb50 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7650b20 con 0x5573d772e230 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7616260 con 0x5573d773d900 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fc77d9740 10 client.0 _ll_register_callbacks cb 0x5573d764bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 0x5573d773a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 0x5573d773a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47284/0 (socket says 172.21.15.44:47284) 2024-09-13T17:34:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/2357254637 learned_addr learned my addr 172.21.15.44:0/2357254637 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d773d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d773ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 msgr2=0x5573d773ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d773ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/2357254637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 msgr2=0x5573d773d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d773d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.957+0000 7f3fbdd9b640 1 -- 172.21.15.44:0/2357254637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc01a31d0 con 0x5573d772eb50 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 0x5573d773a660 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4096220 tx=0x7f3fb4095b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce170b155d71fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/2357254637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40a1b80 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/2357254637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f3fb4097b50 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573d76bb680 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/2357254637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb40997b0 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 10 client.18075 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5573d7691330 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/2357254637 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5573d76bb680 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/2357254637 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fb40b3070 con 0x5573d772eb50 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 10 client.18075 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 10 client.18075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 20 client.18075 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d774ceb0 0x5573d774f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 4 client.18075 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.958+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573d76865d0 con 0x5573d774ceb0 2024-09-13T17:34:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.959+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d774ceb0 0x5573d774f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.959+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d774ceb0 0x5573d774f2a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f77ed825736cdee server_cookie=1d04c5d2549c45c4 in_seq=0 out_seq=0 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.959+0000 7f3fa5ffb640 10 client.18075 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/2357254637 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5573d76865d0 con 0x5573d774ceb0 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fa5ffb640 10 client.18075 handle_command_reply: tid=0 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 client.18075 shutdown 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 msgr2=0x5573d773a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 0x5573d773a660 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4096220 tx=0x7f3fb4095b70 comp rx=0 tx=0).stop 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 shutdown_connections 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d774ceb0 0x5573d774f2a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d773ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772eb50 0x5573d773a660 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2357254637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d773d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 >> 172.21.15.44:0/2357254637 conn(0x5573d772a8c0 msgr2=0x5573d77305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fa5ffb640 0 client.18075 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 shutdown_connections 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2357254637 wait complete. 2024-09-13T17:34:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.960+0000 7f3fc77d9740 20 client.18075 trim_cache size 0 max 16384 2024-09-13T17:34:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 Processor -- start 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 -- start start 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 0x5573d772a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772eb50 0x5573d7749020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d7749560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d76bb680 con 0x5573d772e230 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7730d90 con 0x5573d772eb50 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7738b70 con 0x5573d773d900 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fbe59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 0x5573d772a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fbe59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 0x5573d772a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47296/0 (socket says 172.21.15.44:47296) 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fbe59c640 1 -- 172.21.15.44:0/2570288484 learned_addr learned my addr 172.21.15.44:0/2570288484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772eb50 0x5573d7749020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.973+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d7749560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbe59c640 1 -- 172.21.15.44:0/2570288484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 msgr2=0x5573d7749560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d7749560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbe59c640 1 -- 172.21.15.44:0/2570288484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772eb50 msgr2=0x5573d7749020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772eb50 0x5573d7749020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbe59c640 1 -- 172.21.15.44:0/2570288484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d7633300 con 0x5573d772e230 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772eb50 0x5573d7749020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d7749560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 0x5573d772a300 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006010 tx=0x7f3fb0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b9de95d2a42de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fad7fa640 1 -- 172.21.15.44:0/2570288484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0043040 con 0x5573d772e230 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fad7fa640 1 -- 172.21.15.44:0/2570288484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fb00452c0 con 0x5573d772e230 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fad7fa640 1 -- 172.21.15.44:0/2570288484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0031bb0 con 0x5573d772e230 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2570288484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 msgr2=0x5573d772a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.974+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 0x5573d772a300 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006010 tx=0x7f3fb0005c00 comp rx=0 tx=0).stop 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2570288484 shutdown_connections 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d773d900 0x5573d7749560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772eb50 0x5573d7749020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/2570288484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d772e230 0x5573d772a300 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2570288484 >> 172.21.15.44:0/2570288484 conn(0x5573d772a8c0 msgr2=0x5573d7735920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2570288484 shutdown_connections 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- 172.21.15.44:0/2570288484 wait complete. 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- start start 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 Processor -- start 2024-09-13T17:34:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d77419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d7741f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d779d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d779d820 con 0x5573d773d900 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d779d990 con 0x5573d772e230 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d779db50 con 0x5573d772eb50 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fc77d9740 10 client.0 _ll_register_callbacks cb 0x7f3fb4099580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fbed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d779d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fbed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d779d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47312/0 (socket says 172.21.15.44:47312) 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fbed9d640 1 -- 172.21.15.44:0/3764374925 learned_addr learned my addr 172.21.15.44:0/3764374925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.975+0000 7f3fbdd9b640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d7741f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d77419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbed9d640 1 -- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 msgr2=0x5573d7741f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d7741f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbed9d640 1 -- 172.21.15.44:0/3764374925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 msgr2=0x5573d77419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d77419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbed9d640 1 -- 172.21.15.44:0/3764374925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc010d3e0 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fbed9d640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d779d2e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8012fe0 tx=0x7f3fb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdf91b3962fa8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/3764374925 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8004b20 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/3764374925 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f3fb8004c80 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fb409d530 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/3764374925 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb8004e40 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fc77d9740 10 client.18081 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.976+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5573d76588e0 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/3764374925 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fb409d530 con 0x5573d773d900 2024-09-13T17:34:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/3764374925 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fb80053f0 con 0x5573d773d900 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fc77d9740 10 client.18081 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fc77d9740 10 client.18081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fc77d9740 20 client.18081 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d77a53d0 0x5573d77a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fc77d9740 4 client.18081 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573d7386300 con 0x5573d77a53d0 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d77a53d0 0x5573d77a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fbe59c640 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d77a53d0 0x5573d77a77f0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12ef3e31e625b24 server_cookie=b590d9e41b616f56 in_seq=0 out_seq=0 2024-09-13T17:34:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.977+0000 7f3fa5ffb640 10 client.18081 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fa5ffb640 1 -- 172.21.15.44:0/3764374925 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573d7386300 con 0x5573d77a53d0 2024-09-13T17:34:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fa5ffb640 10 client.18081 handle_command_reply: tid=0 2024-09-13T17:34:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 client.18081 shutdown 2024-09-13T17:34:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 msgr2=0x5573d779d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d779d2e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8012fe0 tx=0x7f3fb8004a40 comp rx=0 tx=0).stop 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 shutdown_connections 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5573d77a53d0 0x5573d77a77f0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573d772eb50 0x5573d7741f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fa5ffb640 0 client.18081 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573d773d900 0x5573d779d2e0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 --2- 172.21.15.44:0/3764374925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5573d772e230 0x5573d77419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 >> 172.21.15.44:0/3764374925 conn(0x5573d772a8c0 msgr2=0x5573d7735920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 shutdown_connections 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 1 -- 172.21.15.44:0/3764374925 wait complete. 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.990+0000 7f3fc77d9740 20 client.18081 trim_cache size 0 max 16384 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:34:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:34:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fa80a1200 msgr2=0x7f3fa80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fa80a1200 0x7f3fa80a36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009990 tx=0x7f3fb80096a0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 msgr2=0x7f3fc01966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 0x7f3fc01966d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400d750 tx=0x7f3fb400dc10 comp rx=0 tx=0).stop 2024-09-13T17:34:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 shutdown_connections 2024-09-13T17:34:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f3fa80a1200 0x7f3fa80a36c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc019aa70 0x7f3fc019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fc01050b0 0x7f3fc01966d0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 --2- 172.21.15.44:0/3454558331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc00fdf90 0x7f3fc0196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 >> 172.21.15.44:0/3454558331 conn(0x7f3fc00f9bd0 msgr2=0x7f3fc00fac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 shutdown_connections 2024-09-13T17:34:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:45.991+0000 7f3fc49e2640 1 -- 172.21.15.44:0/3454558331 wait complete. 2024-09-13T17:34:46.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:45 smithi100 ceph-mon[35055]: pgmap v248: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:34:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:47 smithi044 ceph-mon[31175]: pgmap v249: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:34:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:47 smithi070 ceph-mon[34502]: pgmap v249: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:34:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:47 smithi100 ceph-mon[35055]: pgmap v249: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:34:49.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:49 smithi044 ceph-mon[31175]: pgmap v250: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-13T17:34:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:49 smithi070 ceph-mon[34502]: pgmap v250: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-13T17:34:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:49 smithi100 ceph-mon[35055]: pgmap v250: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-13T17:34:51.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:51 smithi044 ceph-mon[31175]: pgmap v251: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-13T17:34:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:51 smithi070 ceph-mon[34502]: pgmap v251: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-13T17:34:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:51 smithi100 ceph-mon[35055]: pgmap v251: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-13T17:34:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:53 smithi044 ceph-mon[31175]: pgmap v252: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-13T17:34:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:53 smithi070 ceph-mon[34502]: pgmap v252: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-13T17:34:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:53 smithi100 ceph-mon[35055]: pgmap v252: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-13T17:34:54.497 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:34:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 -- 172.21.15.44:0/2584593219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d8103fd0 msgr2=0x7fb1d810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 --2- 172.21.15.44:0/2584593219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d8103fd0 0x7fb1d810bb20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009cd0 tx=0x7fb1d002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 -- 172.21.15.44:0/2584593219 shutdown_connections 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 --2- 172.21.15.44:0/2584593219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d8103fd0 0x7fb1d810bb20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 --2- 172.21.15.44:0/2584593219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d80ff830 0x7fb1d8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 --2- 172.21.15.44:0/2584593219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d80fee80 0x7fb1d80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 -- 172.21.15.44:0/2584593219 >> 172.21.15.44:0/2584593219 conn(0x7fb1d80fa930 msgr2=0x7fb1d80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 -- 172.21.15.44:0/2584593219 shutdown_connections 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.614+0000 7fb1df220640 1 -- 172.21.15.44:0/2584593219 wait complete. 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 Processor -- start 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 -- start start 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d80fee80 0x7fb1d81961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d8103fd0 0x7fb1d8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 0x7fb1d819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819b500 con 0x7fb1d819ab10 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819b670 con 0x7fb1d80fee80 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1df220640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819b7e0 con 0x7fb1d8103fd0 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1dd7bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 0x7fb1d819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1dd7bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 0x7fb1d819afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55778/0 (socket says 172.21.15.44:55778) 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.615+0000 7fb1dd7bd640 1 -- 172.21.15.44:0/1495412739 learned_addr learned my addr 172.21.15.44:0/1495412739 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d80fee80 0x7fb1d81961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d8103fd0 0x7fb1d8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dd7bd640 1 -- 172.21.15.44:0/1495412739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d8103fd0 msgr2=0x7fb1d8196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dd7bd640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d8103fd0 0x7fb1d8196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dd7bd640 1 -- 172.21.15.44:0/1495412739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d80fee80 msgr2=0x7fb1d81961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dd7bd640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d80fee80 0x7fb1d81961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dd7bd640 1 -- 172.21.15.44:0/1495412739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0009900 con 0x7fb1d819ab10 2024-09-13T17:34:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1dd7bd640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 0x7fb1d819afc0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0004af0 tx=0x7fb1d0030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1d5ffb640 1 -- 172.21.15.44:0/1495412739 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0031b80 con 0x7fb1d819ab10 2024-09-13T17:34:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1d5ffb640 1 -- 172.21.15.44:0/1495412739 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1d0003e00 con 0x7fb1d819ab10 2024-09-13T17:34:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d81a2dc0 con 0x7fb1d819ab10 2024-09-13T17:34:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1d5ffb640 1 -- 172.21.15.44:0/1495412739 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0045410 con 0x7fb1d819ab10 2024-09-13T17:34:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.616+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d81a3270 con 0x7fb1d819ab10 2024-09-13T17:34:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.619+0000 7fb1e2021740 1 Processor -- start 2024-09-13T17:34:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.619+0000 7fb1d5ffb640 1 -- 172.21.15.44:0/1495412739 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb1d000a040 con 0x7fb1d819ab10 2024-09-13T17:34:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 -- start start 2024-09-13T17:34:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb8b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb99ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdaf5f20 con 0x55abbdb8acd0 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdaf36b0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1e2021740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdaeefa0 con 0x55abbdb9a3e0 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb8b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb8b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55784/0 (socket says 172.21.15.44:55784) 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 -- 172.21.15.44:0/1694960693 learned_addr learned my addr 172.21.15.44:0/1694960693 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb99ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d5ffb640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1c40a1280 0x7fb1c40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 -- 172.21.15.44:0/1694960693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 msgr2=0x55abbdb9c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 -- 172.21.15.44:0/1694960693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 msgr2=0x55abbdb99ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb99ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1d7fff640 1 -- 172.21.15.44:0/1694960693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbdaa9840 con 0x55abbdb8acd0 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.620+0000 7fb1dd7bd640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1c40a1280 0x7fb1c40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb8b0b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009fc0 tx=0x7fb1c0009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5faa2dc5ca106 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1d5ffb640 1 -- 172.21.15.44:0/1495412739 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb1d00ee4e0 con 0x7fb1d819ab10 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1ceffd640 1 -- 172.21.15.44:0/1694960693 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0004050 con 0x55abbdb8acd0 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1ceffd640 1 -- 172.21.15.44:0/1694960693 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1c00035f0 con 0x55abbdb8acd0 2024-09-13T17:34:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1ceffd640 1 -- 172.21.15.44:0/1694960693 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00116b0 con 0x55abbdb8acd0 2024-09-13T17:34:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1e2021740 1 -- 172.21.15.44:0/1694960693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 msgr2=0x55abbdb8b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1dd7bd640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1c40a1280 0x7fb1c40a3740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00baf20 tx=0x7fb1d0037600 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.621+0000 7fb1e2021740 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb8b0b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009fc0 tx=0x7fb1c0009700 comp rx=0 tx=0).stop 2024-09-13T17:34:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 -- 172.21.15.44:0/1694960693 shutdown_connections 2024-09-13T17:34:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb99ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 --2- 172.21.15.44:0/1694960693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb8b0b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 -- 172.21.15.44:0/1694960693 >> 172.21.15.44:0/1694960693 conn(0x55abbdb87380 msgr2=0x55abbdb87760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 -- 172.21.15.44:0/1694960693 shutdown_connections 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 -- 172.21.15.44:0/1694960693 wait complete. 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 -- start start 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 Processor -- start 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.622+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb9a3e0 0x55abbdb97740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1e2021740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbda902a0 con 0x55abbdb8acd0 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1e2021740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbd7c7430 con 0x55abbdb9a3e0 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1e2021740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbda72680 con 0x55abbdb8b5f0 2024-09-13T17:34:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1e2021740 10 client.0 _ll_register_callbacks cb 0x55abbdae8610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb99e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55790/0 (socket says 172.21.15.44:55790) 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2069545650 learned_addr learned my addr 172.21.15.44:0/2069545650 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb9a3e0 0x55abbdb97740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 msgr2=0x55abbdb970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2069545650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb9a3e0 msgr2=0x55abbdb97740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb9a3e0 0x55abbdb97740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2069545650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c0009c40 con 0x55abbdb8acd0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb99e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0003f70 tx=0x7fb1c0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fe2a28570e5326 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2069545650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00044a0 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2069545650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb1c0005050 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.623+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abbd7e24e0 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2069545650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c0005210 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 10 client.18093 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55abbdaed620 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2069545650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55abbd7e24e0 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2069545650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb1c0011470 con 0x55abbdb8acd0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 10 client.18093 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 10 client.18093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 20 client.18093 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdbaa1c0 0x55abbdbac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 4 client.18093 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abbd7f37e0 con 0x55abbdbaa1c0 2024-09-13T17:34:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.624+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdbaa1c0 0x55abbdbac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.625+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdbaa1c0 0x55abbdbac5e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c9769239ebb26c server_cookie=3576f95355fa86c7 in_seq=0 out_seq=0 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.625+0000 7fb1b7fff640 10 client.18093 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.625+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2069545650 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55abbd7f37e0 con 0x55abbdbaa1c0 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.625+0000 7fb1b7fff640 10 client.18093 handle_command_reply: tid=0 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 client.18093 shutdown 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 msgr2=0x55abbdb99e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb99e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0003f70 tx=0x7fb1c0005900 comp rx=0 tx=0).stop 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 shutdown_connections 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdbaa1c0 0x55abbdbac5e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb9a3e0 0x55abbdb97740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2069545650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdb99e40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1b7fff640 0 client.18093 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 >> 172.21.15.44:0/2069545650 conn(0x55abbdb87380 msgr2=0x55abbdb86510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 shutdown_connections 2024-09-13T17:34:54.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 1 -- 172.21.15.44:0/2069545650 wait complete. 2024-09-13T17:34:54.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.626+0000 7fb1e2021740 20 client.18093 trim_cache size 0 max 16384 2024-09-13T17:34:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 Processor -- start 2024-09-13T17:34:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 -- start start 2024-09-13T17:34:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdba5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbd7e24e0 con 0x55abbdb8acd0 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdba9ff0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.640+0000 7fb1e2021740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdb8be20 con 0x55abbdb9a3e0 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdba5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdba5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55804/0 (socket says 172.21.15.44:55804) 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2430838833 learned_addr learned my addr 172.21.15.44:0/2430838833 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2430838833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 msgr2=0x55abbdb923a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2430838833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 msgr2=0x55abbdb9c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2430838833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbda8f790 con 0x55abbdb8acd0 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdba5c30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0011880 tx=0x7fb1c00119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f7a600bf8c4bad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1cf7fe640 1 -- 172.21.15.44:0/2430838833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0011670 con 0x55abbdb8acd0 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1cf7fe640 1 -- 172.21.15.44:0/2430838833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1c0013d70 con 0x55abbdb8acd0 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.641+0000 7fb1cf7fe640 1 -- 172.21.15.44:0/2430838833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0005580 con 0x55abbdb8acd0 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 -- 172.21.15.44:0/2430838833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 msgr2=0x55abbdba5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdba5c30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0011880 tx=0x7fb1c00119e0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 -- 172.21.15.44:0/2430838833 shutdown_connections 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb9a3e0 0x55abbdb923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2430838833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb8acd0 0x55abbdba5c30 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 -- 172.21.15.44:0/2430838833 >> 172.21.15.44:0/2430838833 conn(0x55abbdb87380 msgr2=0x55abbdb87760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 -- 172.21.15.44:0/2430838833 shutdown_connections 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 -- 172.21.15.44:0/2430838833 wait complete. 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 -- start start 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.642+0000 7fb1e2021740 1 Processor -- start 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8acd0 0x55abbdb9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdbff750 con 0x55abbdb9a3e0 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdbff890 con 0x55abbdb8acd0 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdbffa50 con 0x55abbdb8b5f0 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1e2021740 10 client.0 _ll_register_callbacks cb 0x7fb1c800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55814/0 (socket says 172.21.15.44:55814) 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1d7fff640 1 -- 172.21.15.44:0/2746811408 learned_addr learned my addr 172.21.15.44:0/2746811408 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8acd0 0x55abbdb9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 -- 172.21.15.44:0/2746811408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8acd0 msgr2=0x55abbdb9d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8acd0 0x55abbdb9d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 -- 172.21.15.44:0/2746811408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 msgr2=0x55abbdb9dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 -- 172.21.15.44:0/2746811408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abbdaed620 con 0x55abbdb8b5f0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1dcfbc640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9d690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55abbdbace90 tx=0x7fb1c801c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c1daf885167f573 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.643+0000 7fb1b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2746811408 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8017040 con 0x55abbdb8b5f0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2746811408 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb1c8003ca0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1c00035f0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2746811408 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c80044e0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 10 client.27370 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55abbdab4ea0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2746811408 <== mon.1 v2:172.21.15.100:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1c00035f0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2746811408 <== mon.1 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb1c80243e0 con 0x55abbdb8b5f0 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 10 client.27370 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 10 client.27370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 20 client.27370 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdc01250 0x55abbdc03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 4 client.27370 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55abbd846e30 con 0x55abbdc01250 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.644+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdc01250 0x55abbdc03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1b7fff640 10 client.27370 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1d7fff640 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdc01250 0x55abbdc03670 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be310af512cb9bbf server_cookie=152b91ea6a4384a7 in_seq=0 out_seq=0 2024-09-13T17:34:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1b7fff640 1 -- 172.21.15.44:0/2746811408 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55abbd846e30 con 0x55abbdc01250 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1b7fff640 10 client.27370 handle_command_reply: tid=0 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 client.27370 shutdown 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 msgr2=0x55abbdb9d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9d690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55abbdbace90 tx=0x7fb1c801c680 comp rx=0 tx=0).stop 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 shutdown_connections 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55abbdc01250 0x55abbdc03670 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abbdb8b5f0 0x55abbdb9d690 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbdb9a3e0 0x55abbdb9dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 --2- 172.21.15.44:0/2746811408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abbdb8acd0 0x55abbdb9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1b7fff640 0 client.27370 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 >> 172.21.15.44:0/2746811408 conn(0x55abbdb87380 msgr2=0x55abbdb923a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 shutdown_connections 2024-09-13T17:34:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.645+0000 7fb1e2021740 1 -- 172.21.15.44:0/2746811408 wait complete. 2024-09-13T17:34:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1e2021740 20 client.27370 trim_cache size 0 max 16384 2024-09-13T17:34:54.657 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:34:54.657 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:34:54.657 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:34:54.657 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1c40a1280 msgr2=0x7fb1c40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1c40a1280 0x7fb1c40a3740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00baf20 tx=0x7fb1d0037600 comp rx=0 tx=0).stop 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 msgr2=0x7fb1d819afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 0x7fb1d819afc0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0004af0 tx=0x7fb1d0030e50 comp rx=0 tx=0).stop 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 shutdown_connections 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb1c40a1280 0x7fb1c40a3740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d8103fd0 0x7fb1d8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d819ab10 0x7fb1d819afc0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 --2- 172.21.15.44:0/1495412739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d80fee80 0x7fb1d81961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 >> 172.21.15.44:0/1495412739 conn(0x7fb1d80fa930 msgr2=0x7fb1d80688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 shutdown_connections 2024-09-13T17:34:54.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.646+0000 7fb1df220640 1 -- 172.21.15.44:0/1495412739 wait complete. 2024-09-13T17:34:54.660 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:34:54.660 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.760+0000 7f096455e640 1 -- 172.21.15.44:0/1227311021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 msgr2=0x7f095c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.760+0000 7f096455e640 1 --2- 172.21.15.44:0/1227311021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1012f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0958009900 tx=0x7f095802a480 comp rx=0 tx=0).stop 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.760+0000 7f096455e640 1 -- 172.21.15.44:0/1227311021 shutdown_connections 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.760+0000 7f096455e640 1 --2- 172.21.15.44:0/1227311021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 0x7f095c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.760+0000 7f096455e640 1 --2- 172.21.15.44:0/1227311021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c10a360 0x7f095c10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.760+0000 7f096455e640 1 --2- 172.21.15.44:0/1227311021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1012f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- 172.21.15.44:0/1227311021 >> 172.21.15.44:0/1227311021 conn(0x7f095c0fa900 msgr2=0x7f095c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- 172.21.15.44:0/1227311021 shutdown_connections 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- 172.21.15.44:0/1227311021 wait complete. 2024-09-13T17:34:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 Processor -- start 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- start start 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 0x7f095c196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c10a360 0x7f095c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19b120 con 0x7f095c100f10 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19b290 con 0x7f095c10a360 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f096455e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19b450 con 0x7f095c101830 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f09622fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f09622fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55830/0 (socket says 172.21.15.44:55830) 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.761+0000 7f09622fa640 1 -- 172.21.15.44:0/2457927426 learned_addr learned my addr 172.21.15.44:0/2457927426 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f0962afb640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c10a360 0x7f095c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f0961af9640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 0x7f095c196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09622fa640 1 -- 172.21.15.44:0/2457927426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 msgr2=0x7f095c196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09622fa640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 0x7f095c196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09622fa640 1 -- 172.21.15.44:0/2457927426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c10a360 msgr2=0x7f095c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09622fa640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c10a360 0x7f095c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09622fa640 1 -- 172.21.15.44:0/2457927426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958009560 con 0x7f095c100f10 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f0961af9640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 0x7f095c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09622fa640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1961f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f09580057c0 tx=0x7f0958032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958032e10 con 0x7f095c100f10 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09580046c0 con 0x7f095c100f10 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c19b6d0 con 0x7f095c100f10 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958003850 con 0x7f095c100f10 2024-09-13T17:34:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.762+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c1a2fb0 con 0x7f095c100f10 2024-09-13T17:34:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.763+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c068ed0 con 0x7f095c100f10 2024-09-13T17:34:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.765+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0958005210 con 0x7f095c100f10 2024-09-13T17:34:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.766+0000 7f09537fe640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f09480a1280 0x7f09480a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.766+0000 7f0961af9640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f09480a1280 0x7f09480a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.766+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f09580ee200 con 0x7f095c100f10 2024-09-13T17:34:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.766+0000 7f0961af9640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f09480a1280 0x7f09480a3740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f095c197790 tx=0x7f094c005e30 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.769+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f09580b7280 con 0x7f095c100f10 2024-09-13T17:34:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.977+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f095c105410 con 0x7f095c100f10 2024-09-13T17:34:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.978+0000 7f09537fe640 1 -- 172.21.15.44:0/2457927426 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f095804a090 con 0x7f095c100f10 2024-09-13T17:34:54.981 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:34:54.981 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:34:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:34:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f09480a1280 msgr2=0x7f09480a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f09480a1280 0x7f09480a3740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f095c197790 tx=0x7f094c005e30 comp rx=0 tx=0).stop 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 msgr2=0x7f095c1961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1961f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f09580057c0 tx=0x7f0958032c20 comp rx=0 tx=0).stop 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 shutdown_connections 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f09480a1280 0x7f09480a3740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f095c101830 0x7f095c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c10a360 0x7f095c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 --2- 172.21.15.44:0/2457927426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f095c100f10 0x7f095c1961f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 >> 172.21.15.44:0/2457927426 conn(0x7f095c0fa900 msgr2=0x7f095c0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 shutdown_connections 2024-09-13T17:34:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:54.980+0000 7f096455e640 1 -- 172.21.15.44:0/2457927426 wait complete. 2024-09-13T17:34:54.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 -- 172.21.15.44:0/2116909571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc100900 msgr2=0x7fb7fc104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 --2- 172.21.15.44:0/2116909571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc100900 0x7fb7fc104940 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009930 tx=0x7fb7ec02a420 comp rx=0 tx=0).stop 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 -- 172.21.15.44:0/2116909571 shutdown_connections 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 --2- 172.21.15.44:0/2116909571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1050a0 0x7fb7fc10cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 --2- 172.21.15.44:0/2116909571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc100900 0x7fb7fc104940 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 --2- 172.21.15.44:0/2116909571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 -- 172.21.15.44:0/2116909571 >> 172.21.15.44:0/2116909571 conn(0x7fb7fc0fb9e0 msgr2=0x7fb7fc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 -- 172.21.15.44:0/2116909571 shutdown_connections 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 -- 172.21.15.44:0/2116909571 wait complete. 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.097+0000 7fb802ef7640 1 Processor -- start 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 -- start start 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1050a0 0x7fb7fc1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7fc19ab60 0x7fb7fc19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19b550 con 0x7fb7fc0fff50 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19b6c0 con 0x7fb7fc19ab60 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb802ef7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19b830 con 0x7fb7fc1050a0 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55858/0 (socket says 172.21.15.44:55858) 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 -- 172.21.15.44:0/255818424 learned_addr learned my addr 172.21.15.44:0/255818424 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb801494640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7fc19ab60 0x7fb7fc19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1050a0 0x7fb7fc1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 -- 172.21.15.44:0/255818424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1050a0 msgr2=0x7fb7fc1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1050a0 0x7fb7fc1967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 -- 172.21.15.44:0/255818424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7fc19ab60 msgr2=0x7fb7fc19b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7fc19ab60 0x7fb7fc19b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 -- 172.21.15.44:0/255818424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec009560 con 0x7fb7fc0fff50 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb800c93640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc196280 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400e9f0 tx=0x7fb7e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.098+0000 7fb7f9ffb640 1 -- 172.21.15.44:0/255818424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4009700 con 0x7fb7fc0fff50 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.099+0000 7fb7f9ffb640 1 -- 172.21.15.44:0/255818424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7e400cd80 con 0x7fb7fc0fff50 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.099+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc1a2e10 con 0x7fb7fc0fff50 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.099+0000 7fb7f9ffb640 1 -- 172.21.15.44:0/255818424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40187e0 con 0x7fb7fc0fff50 2024-09-13T17:34:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.099+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a3280 con 0x7fb7fc0fff50 2024-09-13T17:34:55.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.101+0000 7fb7f9ffb640 1 -- 172.21.15.44:0/255818424 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb7e4010430 con 0x7fb7fc0fff50 2024-09-13T17:34:55.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.101+0000 7fb805c1a740 1 Processor -- start 2024-09-13T17:34:55.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.101+0000 7fb805c1a740 1 -- start start 2024-09-13T17:34:55.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d22b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb805c1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3c8ccc0 con 0x55f1c3d2d950 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb805c1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3c8a450 con 0x55f1c3d22790 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb805c1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3c85d40 con 0x55f1c3d230b0 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55864/0 (socket says 172.21.15.44:55864) 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 -- 172.21.15.44:0/214100566 learned_addr learned my addr 172.21.15.44:0/214100566 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb800c93640 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb801494640 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d22b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7f9ffb640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb7e80a1280 0x7fb7e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 -- 172.21.15.44:0/214100566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 msgr2=0x55f1c3d2d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb801494640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb7e80a1280 0x7fb7e80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 -- 172.21.15.44:0/214100566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 msgr2=0x55f1c3d22b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d22b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 -- 172.21.15.44:0/214100566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1c3c4d620 con 0x55f1c3d2d950 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7f9ffb640 1 -- 172.21.15.44:0/255818424 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb7e4014070 con 0x7fb7fc0fff50 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.102+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2fd40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005940 tx=0x7fb7ec031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ab18dbf92f262b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb801494640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb7e80a1280 0x7fb7e80a3740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005fd0 tx=0x7fb7f4004ab0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb7f2ffd640 1 -- 172.21.15.44:0/214100566 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec004020 con 0x55f1c3d2d950 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb7f2ffd640 1 -- 172.21.15.44:0/214100566 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ec031960 con 0x55f1c3d2d950 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb7f2ffd640 1 -- 172.21.15.44:0/214100566 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec003ab0 con 0x55f1c3d2d950 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 -- 172.21.15.44:0/214100566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 msgr2=0x55f1c3d2fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2fd40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005940 tx=0x7fb7ec031400 comp rx=0 tx=0).stop 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 -- 172.21.15.44:0/214100566 shutdown_connections 2024-09-13T17:34:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2fd40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 --2- 172.21.15.44:0/214100566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d22b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 -- 172.21.15.44:0/214100566 >> 172.21.15.44:0/214100566 conn(0x55f1c3d1ed00 msgr2=0x55f1c3d1f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 -- 172.21.15.44:0/214100566 shutdown_connections 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.103+0000 7fb805c1a740 1 -- 172.21.15.44:0/214100566 wait complete. 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 -- start start 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 Processor -- start 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d2ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3c44180 con 0x55f1c3d2d950 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c395e430 con 0x55f1c3d22790 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3c095e0 con 0x55f1c3d230b0 2024-09-13T17:34:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb805c1a740 10 client.0 _ll_register_callbacks cb 0x55f1c3c75aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.104+0000 7fb801494640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55882/0 (socket says 172.21.15.44:55882) 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 -- 172.21.15.44:0/22907931 learned_addr learned my addr 172.21.15.44:0/22907931 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb800c93640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d2ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 -- 172.21.15.44:0/22907931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 msgr2=0x55f1c3d2d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 -- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 msgr2=0x55f1c3d2ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d2ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 -- 172.21.15.44:0/22907931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc10d430 con 0x55f1c3d2d950 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb800c93640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d2ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb801494640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2a440 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40143d0 tx=0x7fb7f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d01419ba6f03ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb7dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb7dbfff640 1 -- 172.21.15.44:0/22907931 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4004b20 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb7dbfff640 1 -- 172.21.15.44:0/22907931 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb7f4004c80 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1c3cafb80 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb7dbfff640 1 -- 172.21.15.44:0/22907931 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7f4004e40 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb805c1a740 10 client.18117 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.105+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f1c3c843c0 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb7dbfff640 1 -- 172.21.15.44:0/22907931 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f1c3cafb80 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb7dbfff640 1 -- 172.21.15.44:0/22907931 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb7f4005590 con 0x55f1c3d2d950 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb805c1a740 10 client.18117 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb805c1a740 10 client.18117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb805c1a740 20 client.18117 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb805c1a740 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d3d2c0 0x55f1c3d3f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb805c1a740 4 client.18117 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1c3c781f0 con 0x55f1c3d3d2c0 2024-09-13T17:34:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb800c93640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d3d2c0 0x55f1c3d3f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb800c93640 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d3d2c0 0x55f1c3d3f6b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93ac63e866f73dba server_cookie=decb5965f58a7e81 in_seq=0 out_seq=0 2024-09-13T17:34:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.106+0000 7fb7dbfff640 10 client.18117 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb7dbfff640 1 -- 172.21.15.44:0/22907931 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f1c3c781f0 con 0x55f1c3d3d2c0 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb7dbfff640 10 client.18117 handle_command_reply: tid=0 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 client.18117 shutdown 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 msgr2=0x55f1c3d2a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2a440 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40143d0 tx=0x7fb7f4004a40 comp rx=0 tx=0).stop 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 shutdown_connections 2024-09-13T17:34:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d3d2c0 0x55f1c3d3f6b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d230b0 0x55f1c3d2d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d2d950 0x55f1c3d2a440 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb7dbfff640 0 client.18117 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 --2- 172.21.15.44:0/22907931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d2ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 >> 172.21.15.44:0/22907931 conn(0x55f1c3d1ed00 msgr2=0x55f1c3d24a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 shutdown_connections 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.107+0000 7fb805c1a740 1 -- 172.21.15.44:0/22907931 wait complete. 2024-09-13T17:34:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.108+0000 7fb805c1a740 20 client.18117 trim_cache size 0 max 16384 2024-09-13T17:34:55.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.118+0000 7fb805c1a740 1 Processor -- start 2024-09-13T17:34:55.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.118+0000 7fb805c1a740 1 -- start start 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d34a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d28aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d2d950 0x55f1c3d28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb805c1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3c781f0 con 0x55f1c3d230b0 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb805c1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3cafb80 con 0x55f1c3d22790 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb805c1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3d34f70 con 0x55f1c3d2d950 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d28aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d28aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55886/0 (socket says 172.21.15.44:55886) 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 -- 172.21.15.44:0/1634680457 learned_addr learned my addr 172.21.15.44:0/1634680457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb801494640 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d2d950 0x55f1c3d28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb800c93640 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d34a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 -- 172.21.15.44:0/1634680457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d2d950 msgr2=0x55f1c3d28fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d2d950 0x55f1c3d28fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 -- 172.21.15.44:0/1634680457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 msgr2=0x55f1c3d34a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d34a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 -- 172.21.15.44:0/1634680457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1c3c26770 con 0x55f1c3d230b0 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb800c93640 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d28aa0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009930 tx=0x7fb7ec02aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31aeb427c2b141d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7f37fe640 1 -- 172.21.15.44:0/1634680457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec049070 con 0x55f1c3d230b0 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7f37fe640 1 -- 172.21.15.44:0/1634680457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ec0315f0 con 0x55f1c3d230b0 2024-09-13T17:34:55.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.119+0000 7fb7f37fe640 1 -- 172.21.15.44:0/1634680457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec03cca0 con 0x55f1c3d230b0 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 -- 172.21.15.44:0/1634680457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 msgr2=0x55f1c3d28aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d28aa0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009930 tx=0x7fb7ec02aaa0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 -- 172.21.15.44:0/1634680457 shutdown_connections 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d2d950 0x55f1c3d28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d28aa0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 --2- 172.21.15.44:0/1634680457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d22790 0x55f1c3d34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 -- 172.21.15.44:0/1634680457 >> 172.21.15.44:0/1634680457 conn(0x55f1c3d1ed00 msgr2=0x55f1c3d44050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 -- 172.21.15.44:0/1634680457 shutdown_connections 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 -- 172.21.15.44:0/1634680457 wait complete. 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 -- start start 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.120+0000 7fb805c1a740 1 Processor -- start 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d22790 0x55f1c3d8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d8bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d2d950 0x55f1c3d8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3d90280 con 0x55f1c3d230b0 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3d903f0 con 0x55f1c3d2d950 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c3d905b0 con 0x55f1c3d22790 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb805c1a740 10 client.0 _ll_register_callbacks cb 0x7fb7e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d8bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d8bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55898/0 (socket says 172.21.15.44:55898) 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 -- 172.21.15.44:0/184148045 learned_addr learned my addr 172.21.15.44:0/184148045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb801494640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d2d950 0x55f1c3d8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb800c93640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d22790 0x55f1c3d8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 -- 172.21.15.44:0/184148045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d22790 msgr2=0x55f1c3d8b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d22790 0x55f1c3d8b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 -- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d2d950 msgr2=0x55f1c3d8fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d2d950 0x55f1c3d8fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 -- 172.21.15.44:0/184148045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1c3c4d620 con 0x55f1c3d230b0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7fbfff640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d8bc80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005c40 tx=0x7fb7ec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be8a89c9fa0871 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.121+0000 7fb7dbfff640 1 -- 172.21.15.44:0/184148045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec049040 con 0x55f1c3d230b0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb7dbfff640 1 -- 172.21.15.44:0/184148045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb7ec03ca20 con 0x55f1c3d230b0 2024-09-13T17:34:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7ec031960 con 0x55f1c3d230b0 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb7dbfff640 1 -- 172.21.15.44:0/184148045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ec03cbe0 con 0x55f1c3d230b0 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 10 client.18123 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f1c3c4be80 con 0x55f1c3d230b0 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb7dbfff640 1 -- 172.21.15.44:0/184148045 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ec031960 con 0x55f1c3d230b0 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb7dbfff640 1 -- 172.21.15.44:0/184148045 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb7ec055030 con 0x55f1c3d230b0 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 10 client.18123 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 10 client.18123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 20 client.18123 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d98c30 0x55f1c3d9b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 4 client.18123 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1c39794e0 con 0x55f1c3d98c30 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.122+0000 7fb800c93640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d98c30 0x55f1c3d9b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb7dbfff640 10 client.18123 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb800c93640 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d98c30 0x55f1c3d9b020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73a2a6b5a854cfd2 server_cookie=ae47e6542ba8503d in_seq=0 out_seq=0 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb7dbfff640 1 -- 172.21.15.44:0/184148045 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1c39794e0 con 0x55f1c3d98c30 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb7dbfff640 10 client.18123 handle_command_reply: tid=0 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 client.18123 shutdown 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 msgr2=0x55f1c3d8bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d8bc80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005c40 tx=0x7fb7ec005780 comp rx=0 tx=0).stop 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 shutdown_connections 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55f1c3d98c30 0x55f1c3d9b020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1c3d2d950 0x55f1c3d8fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1c3d230b0 0x55f1c3d8bc80 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb7dbfff640 0 client.18123 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 --2- 172.21.15.44:0/184148045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1c3d22790 0x55f1c3d8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 >> 172.21.15.44:0/184148045 conn(0x55f1c3d1ed00 msgr2=0x55f1c3d28a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 shutdown_connections 2024-09-13T17:34:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 1 -- 172.21.15.44:0/184148045 wait complete. 2024-09-13T17:34:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.123+0000 7fb805c1a740 20 client.18123 trim_cache size 0 max 16384 2024-09-13T17:34:55.132 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb7e80a1280 msgr2=0x7fb7e80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb7e80a1280 0x7fb7e80a3740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005fd0 tx=0x7fb7f4004ab0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 msgr2=0x7fb7fc196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc196280 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400e9f0 tx=0x7fb7e400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 shutdown_connections 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb7e80a1280 0x7fb7e80a3740 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1050a0 0x7fb7fc1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7fc19ab60 0x7fb7fc19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 --2- 172.21.15.44:0/255818424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7fc0fff50 0x7fb7fc196280 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 >> 172.21.15.44:0/255818424 conn(0x7fb7fc0fb9e0 msgr2=0x7fb7fc0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 shutdown_connections 2024-09-13T17:34:55.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.124+0000 7fb802ef7640 1 -- 172.21.15.44:0/255818424 wait complete. 2024-09-13T17:34:55.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:34:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 -- 172.21.15.44:0/139032398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8103770 msgr2=0x7fd4a810b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 --2- 172.21.15.44:0/139032398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8103770 0x7fd4a810b2c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009cd0 tx=0x7fd4a002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 -- 172.21.15.44:0/139032398 shutdown_connections 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 --2- 172.21.15.44:0/139032398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a80fefd0 0x7fd4a8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 --2- 172.21.15.44:0/139032398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8103770 0x7fd4a810b2c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 --2- 172.21.15.44:0/139032398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4a80fe620 0x7fd4a80fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 -- 172.21.15.44:0/139032398 >> 172.21.15.44:0/139032398 conn(0x7fd4a80753c0 msgr2=0x7fd4a80fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 -- 172.21.15.44:0/139032398 shutdown_connections 2024-09-13T17:34:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.239+0000 7fd4af565640 1 -- 172.21.15.44:0/139032398 wait complete. 2024-09-13T17:34:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 Processor -- start 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 -- start start 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a80fe620 0x7fd4a81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4a8103770 0x7fd4a8196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 0x7fd4a819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819b4e0 con 0x7fd4a819aaf0 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819b650 con 0x7fd4a8103770 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4af565640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819b7c0 con 0x7fd4a80fe620 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4adb02640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 0x7fd4a819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4adb02640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 0x7fd4a819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55924/0 (socket says 172.21.15.44:55924) 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.240+0000 7fd4adb02640 1 -- 172.21.15.44:0/2953063721 learned_addr learned my addr 172.21.15.44:0/2953063721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4a8103770 0x7fd4a8196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4ad301640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a80fe620 0x7fd4a81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4adb02640 1 -- 172.21.15.44:0/2953063721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a80fe620 msgr2=0x7fd4a81961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4adb02640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a80fe620 0x7fd4a81961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4adb02640 1 -- 172.21.15.44:0/2953063721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4a8103770 msgr2=0x7fd4a8196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4adb02640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4a8103770 0x7fd4a8196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4adb02640 1 -- 172.21.15.44:0/2953063721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009900 con 0x7fd4a819aaf0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4adb02640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 0x7fd4a819afa0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0004af0 tx=0x7fd4a0030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd49e7fc640 1 -- 172.21.15.44:0/2953063721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031b80 con 0x7fd4a819aaf0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd49e7fc640 1 -- 172.21.15.44:0/2953063721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4a0003e00 con 0x7fd4a819aaf0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a81a2da0 con 0x7fd4a819aaf0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd49e7fc640 1 -- 172.21.15.44:0/2953063721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0045410 con 0x7fd4a819aaf0 2024-09-13T17:34:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.241+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a81a3220 con 0x7fd4a819aaf0 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd49e7fc640 1 -- 172.21.15.44:0/2953063721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd4a000a040 con 0x7fd4a819aaf0 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 Processor -- start 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 -- start start 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 0x55c905c9da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9dfb0 0x55c905cac820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905caf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905c07cc0 con 0x55c905c9d690 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905c05450 con 0x55c905cacd60 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4b2291740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905c00d40 con 0x55c905c9dfb0 2024-09-13T17:34:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd49e7fc640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4940a1200 0x7fd4940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.244+0000 7fd4acb00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 0x55c905c9da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 0x55c905c9da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55930/0 (socket says 172.21.15.44:55930) 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 -- 172.21.15.44:0/2443731456 learned_addr learned my addr 172.21.15.44:0/2443731456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4adb02640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4940a1200 0x7fd4940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4ad301640 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9dfb0 0x55c905cac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd49e7fc640 1 -- 172.21.15.44:0/2953063721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd4a00ee6a0 con 0x7fd4a819aaf0 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 -- 172.21.15.44:0/2443731456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9dfb0 msgr2=0x55c905cac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9dfb0 0x55c905cac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 -- 172.21.15.44:0/2443731456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 msgr2=0x55c905caf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905caf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 -- 172.21.15.44:0/2443731456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c905bc8620 con 0x55c905c9d690 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 0x55c905c9da70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd4980058c0 tx=0x7fd498013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3a135316adae38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4adb02640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4940a1200 0x7fd4940a36c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00bb480 tx=0x7fd4a002a8e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4937fe640 1 -- 172.21.15.44:0/2443731456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498009e10 con 0x55c905c9d690 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4937fe640 1 -- 172.21.15.44:0/2443731456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4980044e0 con 0x55c905c9d690 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.245+0000 7fd4937fe640 1 -- 172.21.15.44:0/2443731456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498010430 con 0x55c905c9d690 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 -- 172.21.15.44:0/2443731456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 msgr2=0x55c905c9da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 0x55c905c9da70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd4980058c0 tx=0x7fd498013830 comp rx=0 tx=0).stop 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 -- 172.21.15.44:0/2443731456 shutdown_connections 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9dfb0 0x55c905cac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905caf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2443731456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9d690 0x55c905c9da70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 -- 172.21.15.44:0/2443731456 >> 172.21.15.44:0/2443731456 conn(0x55c905c99d00 msgr2=0x55c905c9a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 -- 172.21.15.44:0/2443731456 shutdown_connections 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 -- 172.21.15.44:0/2443731456 wait complete. 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 -- start start 2024-09-13T17:34:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.246+0000 7fd4b2291740 1 Processor -- start 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9d690 0x55c905cac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905ca9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905ca9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905bbf180 con 0x55c905c9dfb0 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9058d9430 con 0x55c905cacd60 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905b845e0 con 0x55c905c9d690 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4b2291740 10 client.0 _ll_register_callbacks cb 0x55c905bf0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905ca9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905ca9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55946/0 (socket says 172.21.15.44:55946) 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 -- 172.21.15.44:0/1937463399 learned_addr learned my addr 172.21.15.44:0/1937463399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4acb00640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9d690 0x55c905cac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4acb00640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905ca9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 -- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9d690 msgr2=0x55c905cac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9d690 0x55c905cac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 -- 172.21.15.44:0/1937463399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 msgr2=0x55c905ca9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905ca9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 -- 172.21.15.44:0/1937463399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd498009970 con 0x55c905c9dfb0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4acb00640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905ca9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd4ad301640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905ca9710 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4012740 tx=0x7fd4a4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa7a12e2df234b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.247+0000 7fd490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd490ff9640 1 -- 172.21.15.44:0/1937463399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4017070 con 0x55c905c9dfb0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd490ff9640 1 -- 172.21.15.44:0/1937463399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd4a400f920 con 0x55c905c9dfb0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c905c2ab80 con 0x55c905c9dfb0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd490ff9640 1 -- 172.21.15.44:0/1937463399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a400fae0 con 0x55c905c9dfb0 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 10 client.18135 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c905bff3c0 con 0x55c905c9dfb0 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd490ff9640 1 -- 172.21.15.44:0/1937463399 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c905c2ab80 con 0x55c905c9dfb0 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd490ff9640 1 -- 172.21.15.44:0/1937463399 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4a4010b90 con 0x55c905c9dfb0 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 10 client.18135 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 10 client.18135 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 20 client.18135 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905cbc950 0x55c905cbed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 4 client.18135 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c905bf31f0 con 0x55c905cbc950 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.248+0000 7fd4acb00640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905cbc950 0x55c905cbed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.249+0000 7fd4acb00640 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905cbc950 0x55c905cbed70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22ecde5a4e40fb23 server_cookie=e3079afd848640ca in_seq=0 out_seq=0 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.249+0000 7fd490ff9640 10 client.18135 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.249+0000 7fd490ff9640 1 -- 172.21.15.44:0/1937463399 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c905bf31f0 con 0x55c905cbc950 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.249+0000 7fd490ff9640 10 client.18135 handle_command_reply: tid=0 2024-09-13T17:34:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.249+0000 7fd4b2291740 1 client.18135 shutdown 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 msgr2=0x55c905ca9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905ca9710 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4012740 tx=0x7fd4a4012c00 comp rx=0 tx=0).stop 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 shutdown_connections 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905cbc950 0x55c905cbed70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905cacd60 0x55c905ca9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905ca9710 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd490ff9640 0 client.18135 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 --2- 172.21.15.44:0/1937463399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905c9d690 0x55c905cac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 >> 172.21.15.44:0/1937463399 conn(0x55c905c99d00 msgr2=0x55c905c9fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 shutdown_connections 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 1 -- 172.21.15.44:0/1937463399 wait complete. 2024-09-13T17:34:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.250+0000 7fd4b2291740 20 client.18135 trim_cache size 0 max 16384 2024-09-13T17:34:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 Processor -- start 2024-09-13T17:34:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 -- start start 2024-09-13T17:34:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905ca47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905caf8a0 con 0x55c905c9dfb0 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905caa670 con 0x55c905c9d690 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4b2291740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905caa850 con 0x55c905cacd60 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55962/0 (socket says 172.21.15.44:55962) 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 -- 172.21.15.44:0/4172669565 learned_addr learned my addr 172.21.15.44:0/4172669565 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4acb00640 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 -- 172.21.15.44:0/4172669565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 msgr2=0x55c905ca47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905ca47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 -- 172.21.15.44:0/4172669565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 msgr2=0x55c905cb3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4ad301640 1 -- 172.21.15.44:0/4172669565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c905ba1770 con 0x55c905c9dfb0 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.261+0000 7fd4acb00640 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4ad301640 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb4020 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4001eb0 tx=0x7fd4a400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7160df245831ea98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd493fff640 1 -- 172.21.15.44:0/4172669565 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4017040 con 0x55c905c9dfb0 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd493fff640 1 -- 172.21.15.44:0/4172669565 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4a40098d0 con 0x55c905c9dfb0 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd493fff640 1 -- 172.21.15.44:0/4172669565 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4005350 con 0x55c905c9dfb0 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 -- 172.21.15.44:0/4172669565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 msgr2=0x55c905cb4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb4020 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4001eb0 tx=0x7fd4a400f810 comp rx=0 tx=0).stop 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 -- 172.21.15.44:0/4172669565 shutdown_connections 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905ca47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb4020 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 --2- 172.21.15.44:0/4172669565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 -- 172.21.15.44:0/4172669565 >> 172.21.15.44:0/4172669565 conn(0x55c905c99d00 msgr2=0x55c905ca7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 -- 172.21.15.44:0/4172669565 shutdown_connections 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.262+0000 7fd4b2291740 1 -- 172.21.15.44:0/4172669565 wait complete. 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 -- start start 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 Processor -- start 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905cb0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905d12e10 con 0x55c905c9dfb0 2024-09-13T17:34:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905d12f50 con 0x55c905c9d690 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c905d13090 con 0x55c905cacd60 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4b2291740 10 client.0 _ll_register_callbacks cb 0x7fd498009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4ad301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4ad301640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55974/0 (socket says 172.21.15.44:55974) 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4ad301640 1 -- 172.21.15.44:0/2260898390 learned_addr learned my addr 172.21.15.44:0/2260898390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.263+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905cb0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4ad301640 1 -- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 msgr2=0x55c905cb0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4ad301640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905cb0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4ad301640 1 -- 172.21.15.44:0/2260898390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 msgr2=0x55c905cb04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4ad301640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4ad301640 1 -- 172.21.15.44:0/2260898390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c905bff3c0 con 0x55c905c9dfb0 2024-09-13T17:34:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4ad301640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb0a30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4011000 tx=0x7fd4a4012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd951218555d385c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd490ff9640 1 -- 172.21.15.44:0/2260898390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4017070 con 0x55c905c9dfb0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd490ff9640 1 -- 172.21.15.44:0/2260898390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd4a4000e40 con 0x55c905c9dfb0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4980044e0 con 0x55c905c9dfb0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd490ff9640 1 -- 172.21.15.44:0/2260898390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a402f070 con 0x55c905c9dfb0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 10 client.18141 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c905bc6e80 con 0x55c905c9dfb0 2024-09-13T17:34:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd490ff9640 1 -- 172.21.15.44:0/2260898390 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4980044e0 con 0x55c905c9dfb0 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd490ff9640 1 -- 172.21.15.44:0/2260898390 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4a4003920 con 0x55c905c9dfb0 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 10 client.18141 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 10 client.18141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 20 client.18141 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905d14460 0x55c905d16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.264+0000 7fd4b2291740 4 client.18141 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9058f44e0 con 0x55c905d14460 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905d14460 0x55c905d16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4acb00640 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905d14460 0x55c905d16880 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad9e63b1bccf6516 server_cookie=6848e53df262b123 in_seq=0 out_seq=0 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd490ff9640 10 client.18141 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd490ff9640 1 -- 172.21.15.44:0/2260898390 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9058f44e0 con 0x55c905d14460 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd490ff9640 10 client.18141 handle_command_reply: tid=0 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 client.18141 shutdown 2024-09-13T17:34:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 msgr2=0x55c905cb0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb0a30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4011000 tx=0x7fd4a4012450 comp rx=0 tx=0).stop 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 shutdown_connections 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55c905d14460 0x55c905d16880 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c905cacd60 0x55c905cb0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd490ff9640 0 client.18141 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c905c9dfb0 0x55c905cb0a30 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 --2- 172.21.15.44:0/2260898390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c905c9d690 0x55c905cb04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.265+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 >> 172.21.15.44:0/2260898390 conn(0x55c905c99d00 msgr2=0x55c905ca47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 shutdown_connections 2024-09-13T17:34:55.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4b2291740 1 -- 172.21.15.44:0/2260898390 wait complete. 2024-09-13T17:34:55.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4b2291740 20 client.18141 trim_cache size 0 max 16384 2024-09-13T17:34:55.277 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:34:55.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4940a1200 msgr2=0x7fd4940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4940a1200 0x7fd4940a36c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00bb480 tx=0x7fd4a002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 msgr2=0x7fd4a819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 0x7fd4a819afa0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0004af0 tx=0x7fd4a0030e50 comp rx=0 tx=0).stop 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 shutdown_connections 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd4940a1200 0x7fd4940a36c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a819aaf0 0x7fd4a819afa0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4a8103770 0x7fd4a8196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 --2- 172.21.15.44:0/2953063721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a80fe620 0x7fd4a81961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 >> 172.21.15.44:0/2953063721 conn(0x7fd4a80753c0 msgr2=0x7fd4a810bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 shutdown_connections 2024-09-13T17:34:55.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.266+0000 7fd4af565640 1 -- 172.21.15.44:0/2953063721 wait complete. 2024-09-13T17:34:55.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:34:55.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 -- 172.21.15.44:0/341063681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad01050e0 msgr2=0x7f9ad010cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/341063681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad01050e0 0x7f9ad010cc30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009cd0 tx=0x7f9ac802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 -- 172.21.15.44:0/341063681 shutdown_connections 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/341063681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad01050e0 0x7f9ad010cc30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/341063681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad0100940 0x7f9ad0104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/341063681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad00fff90 0x7f9ad0100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 -- 172.21.15.44:0/341063681 >> 172.21.15.44:0/341063681 conn(0x7f9ad00fba20 msgr2=0x7f9ad00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 -- 172.21.15.44:0/341063681 shutdown_connections 2024-09-13T17:34:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 -- 172.21.15.44:0/341063681 wait complete. 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 Processor -- start 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.382+0000 7f9ad70f7640 1 -- start start 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 0x7f9ad01961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01050e0 0x7f9ad0196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad019ab10 0x7f9ad019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad019b500 con 0x7f9ad00fff90 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad019b670 con 0x7f9ad01050e0 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad019b7e0 con 0x7f9ad019ab10 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 0x7f9ad01961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 0x7f9ad01961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55984/0 (socket says 172.21.15.44:55984) 2024-09-13T17:34:55.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 -- 172.21.15.44:0/32160318 learned_addr learned my addr 172.21.15.44:0/32160318 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad5694640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad019ab10 0x7f9ad019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9acffff640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01050e0 0x7f9ad0196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 -- 172.21.15.44:0/32160318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad019ab10 msgr2=0x7f9ad019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad019ab10 0x7f9ad019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 -- 172.21.15.44:0/32160318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01050e0 msgr2=0x7f9ad0196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01050e0 0x7f9ad0196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 -- 172.21.15.44:0/32160318 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac8009900 con 0x7f9ad00fff90 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9acffff640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01050e0 0x7f9ad0196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 0x7f9ad01961e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400e9f0 tx=0x7f9ac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9acdffb640 1 -- 172.21.15.44:0/32160318 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4009720 con 0x7f9ad00fff90 2024-09-13T17:34:55.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.383+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad01a2dc0 con 0x7f9ad00fff90 2024-09-13T17:34:55.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.384+0000 7f9acdffb640 1 -- 172.21.15.44:0/32160318 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ac400cd80 con 0x7f9ad00fff90 2024-09-13T17:34:55.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.384+0000 7f9acdffb640 1 -- 172.21.15.44:0/32160318 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4010430 con 0x7f9ad00fff90 2024-09-13T17:34:55.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.384+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad01a3210 con 0x7f9ad00fff90 2024-09-13T17:34:55.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9acdffb640 1 -- 172.21.15.44:0/32160318 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9ac4004fb0 con 0x7f9ad00fff90 2024-09-13T17:34:55.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 Processor -- start 2024-09-13T17:34:55.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 -- start start 2024-09-13T17:34:55.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 0x55918eef1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eefc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eefed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ee5bcc0 con 0x55918eef1790 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ee59450 con 0x55918eef20b0 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ee54d40 con 0x55918eefc950 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad5694640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 0x55918eef1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad5694640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 0x55918eef1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55994/0 (socket says 172.21.15.44:55994) 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad5694640 1 -- 172.21.15.44:0/2142511947 learned_addr learned my addr 172.21.15.44:0/2142511947 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eefc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.387+0000 7f9acffff640 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eefed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9ad5694640 1 -- 172.21.15.44:0/2142511947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 msgr2=0x55918eefed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9ad5694640 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eefed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9ad5694640 1 -- 172.21.15.44:0/2142511947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 msgr2=0x55918eefc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9ad5694640 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eefc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9acdffb640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ac00a1200 0x7f9ac00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9ad5694640 1 -- 172.21.15.44:0/2142511947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918ee1c620 con 0x55918eef1790 2024-09-13T17:34:55.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9acffff640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ac00a1200 0x7f9ac00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9ad5694640 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 0x55918eef1b70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009cd0 tx=0x7f9ac8005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96153b483faeebb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9acdffb640 1 -- 172.21.15.44:0/32160318 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9ac4014070 con 0x7f9ad00fff90 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9abaffd640 1 -- 172.21.15.44:0/2142511947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80307a0 con 0x55918eef1790 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9abaffd640 1 -- 172.21.15.44:0/2142511947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ac8040070 con 0x55918eef1790 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.388+0000 7f9abaffd640 1 -- 172.21.15.44:0/2142511947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8003cd0 con 0x55918eef1790 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2142511947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 msgr2=0x55918eef1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9acffff640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ac00a1200 0x7f9ac00a36c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9abc005d80 tx=0x7f9abc005d10 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 0x55918eef1b70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009cd0 tx=0x7f9ac8005f70 comp rx=0 tx=0).stop 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2142511947 shutdown_connections 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eefed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eefc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2142511947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef1790 0x55918eef1b70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2142511947 >> 172.21.15.44:0/2142511947 conn(0x55918eeedd00 msgr2=0x55918eeee0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2142511947 shutdown_connections 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2142511947 wait complete. 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 -- start start 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.389+0000 7f9ad9e18740 1 Processor -- start 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef1790 0x55918eefbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 0x55918eef9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eef9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ee13180 con 0x55918eef20b0 2024-09-13T17:34:55.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918eb2d430 con 0x55918eef1790 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918edd85e0 con 0x55918eefc950 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 10 client.0 _ll_register_callbacks cb 0x55918ee44aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 0x55918eef9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 0x55918eef9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56010/0 (socket says 172.21.15.44:56010) 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 -- 172.21.15.44:0/3764167301 learned_addr learned my addr 172.21.15.44:0/3764167301 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef1790 0x55918eefbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 -- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 msgr2=0x55918eef9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eef9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 -- 172.21.15.44:0/3764167301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef1790 msgr2=0x55918eefbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef1790 0x55918eefbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 -- 172.21.15.44:0/3764167301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad010d470 con 0x55918eef20b0 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef1790 0x55918eefbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:55.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9acffff640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 0x55918eef9250 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0126d0 tx=0x7f9abc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d1deb6c1a56c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9aaffff640 1 -- 172.21.15.44:0/3764167301 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc01f8f0 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9aaffff640 1 -- 172.21.15.44:0/3764167301 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9abc018070 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55918ee7eb80 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.390+0000 7f9aaffff640 1 -- 172.21.15.44:0/3764167301 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9abc01fa50 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 10 client.18153 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55918ee533c0 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9aaffff640 1 -- 172.21.15.44:0/3764167301 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55918ee7eb80 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9aaffff640 1 -- 172.21.15.44:0/3764167301 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9abc025020 con 0x55918eef20b0 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 10 client.18153 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:55.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 20 client.18153 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef0c340 0x55918ef0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 4 client.18153 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55918ee471f0 con 0x55918ef0c340 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.391+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef0c340 0x55918ef0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef0c340 0x55918ef0e760 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5745953dd5b4a22 server_cookie=24496e7190bea9d4 in_seq=0 out_seq=0 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9aaffff640 10 client.18153 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9aaffff640 1 -- 172.21.15.44:0/3764167301 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55918ee471f0 con 0x55918ef0c340 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9aaffff640 10 client.18153 handle_command_reply: tid=0 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 client.18153 shutdown 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 msgr2=0x55918eef9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 0x55918eef9250 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0126d0 tx=0x7f9abc0046c0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 shutdown_connections 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef0c340 0x55918ef0e760 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eefc950 0x55918eef9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eef20b0 0x55918eef9250 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9aaffff640 0 client.18153 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/3764167301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef1790 0x55918eefbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.392+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 >> 172.21.15.44:0/3764167301 conn(0x55918eeedd00 msgr2=0x55918eef3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.393+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 shutdown_connections 2024-09-13T17:34:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.393+0000 7f9ad9e18740 1 -- 172.21.15.44:0/3764167301 wait complete. 2024-09-13T17:34:55.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.393+0000 7f9ad9e18740 20 client.18153 trim_cache size 0 max 16384 2024-09-13T17:34:55.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 Processor -- start 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 -- start start 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eeed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918eef2750 con 0x55918eefc950 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918eef1ea0 con 0x55918eef20b0 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918eef1510 con 0x55918eef1790 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad5694640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad5694640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56012/0 (socket says 172.21.15.44:56012) 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad5694640 1 -- 172.21.15.44:0/2053971634 learned_addr learned my addr 172.21.15.44:0/2053971634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9acffff640 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eeed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.404+0000 7f9ad5694640 1 -- 172.21.15.44:0/2053971634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 msgr2=0x55918eeed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9ad5694640 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eeed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9ad5694640 1 -- 172.21.15.44:0/2053971634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 msgr2=0x55918eef9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9ad5694640 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9ad5694640 1 -- 172.21.15.44:0/2053971634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918edf5770 con 0x55918eefc950 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eeed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9ad5694640 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef9740 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009ca0 tx=0x7f9ac8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c672fe5036e3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9abb7fe640 1 -- 172.21.15.44:0/2053971634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80307a0 con 0x55918eefc950 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9abb7fe640 1 -- 172.21.15.44:0/2053971634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ac8031770 con 0x55918eefc950 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.405+0000 7f9abb7fe640 1 -- 172.21.15.44:0/2053971634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80478c0 con 0x55918eefc950 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2053971634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 msgr2=0x55918eef9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef9740 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009ca0 tx=0x7f9ac8003970 comp rx=0 tx=0).stop 2024-09-13T17:34:55.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2053971634 shutdown_connections 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef9740 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/2053971634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eeed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2053971634 >> 172.21.15.44:0/2053971634 conn(0x55918eeedd00 msgr2=0x55918eef8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2053971634 shutdown_connections 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 -- 172.21.15.44:0/2053971634 wait complete. 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 -- start start 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 Processor -- start 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eef6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.406+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad9e18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ef62920 con 0x55918eefc950 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ef62a90 con 0x55918eef20b0 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad9e18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ef62c50 con 0x55918eef1790 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad9e18740 10 client.0 _ll_register_callbacks cb 0x7f9ac8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad5694640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad5694640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56018/0 (socket says 172.21.15.44:56018) 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:42114/0 (socket says 172.21.15.44:42114) 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad5694640 1 -- 172.21.15.44:0/45554357 learned_addr learned my addr 172.21.15.44:0/45554357 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:55.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eef6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 -- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 msgr2=0x55918eef6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eef6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 -- 172.21.15.44:0/45554357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 msgr2=0x55918eef6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 -- 172.21.15.44:0/45554357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55918ee1c620 con 0x55918eef20b0 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eef6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9ad5694640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9acffff640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef6640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00fe50 tx=0x7f9abc01e590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111f8c6217f50d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.407+0000 7f9aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9aaffff640 1 -- 172.21.15.44:0/45554357 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc01f830 con 0x55918eef20b0 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9aaffff640 1 -- 172.21.15.44:0/45554357 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9abc018040 con 0x55918eef20b0 2024-09-13T17:34:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ac8040070 con 0x55918eef20b0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9aaffff640 1 -- 172.21.15.44:0/45554357 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9abc01f990 con 0x55918eef20b0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 10 client.27332 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55918ee1ae80 con 0x55918eef20b0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9aaffff640 1 -- 172.21.15.44:0/45554357 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ac8040070 con 0x55918eef20b0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9aaffff640 1 -- 172.21.15.44:0/45554357 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9abc025040 con 0x55918eef20b0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 10 client.27332 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 10 client.27332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 20 client.27332 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef642d0 0x55918ef666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 4 client.27332 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.408+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55918eb484e0 con 0x55918ef642d0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef642d0 0x55918ef666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:55.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9ad4e93640 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef642d0 0x55918ef666f0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef687673a8d8c3e server_cookie=17b831e01cc464bb in_seq=0 out_seq=0 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9aaffff640 10 client.27332 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9aaffff640 1 -- 172.21.15.44:0/45554357 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55918eb484e0 con 0x55918ef642d0 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9aaffff640 10 client.27332 handle_command_reply: tid=0 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9ad9e18740 1 client.27332 shutdown 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 msgr2=0x55918eef6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.409+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef6640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00fe50 tx=0x7f9abc01e590 comp rx=0 tx=0).stop 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 shutdown_connections 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55918ef642d0 0x55918ef666f0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55918eefc950 0x55918eef6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55918eef20b0 0x55918eef6640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9aaffff640 0 client.27332 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 --2- 172.21.15.44:0/45554357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55918eef1790 0x55918eef6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 >> 172.21.15.44:0/45554357 conn(0x55918eeedd00 msgr2=0x55918eef9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 shutdown_connections 2024-09-13T17:34:55.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 1 -- 172.21.15.44:0/45554357 wait complete. 2024-09-13T17:34:55.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad9e18740 20 client.27332 trim_cache size 0 max 16384 2024-09-13T17:34:55.421 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ac00a1200 msgr2=0x7f9ac00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ac00a1200 0x7f9ac00a36c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9abc005d80 tx=0x7f9abc005d10 comp rx=0 tx=0).stop 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 msgr2=0x7f9ad01961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 0x7f9ad01961e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400e9f0 tx=0x7f9ac400edf0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 shutdown_connections 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ac00a1200 0x7f9ac00a36c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad019ab10 0x7f9ad019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01050e0 0x7f9ad0196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 --2- 172.21.15.44:0/32160318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ad00fff90 0x7f9ad01961e0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.410+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 >> 172.21.15.44:0/32160318 conn(0x7f9ad00fba20 msgr2=0x7f9ad00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:55.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.411+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 shutdown_connections 2024-09-13T17:34:55.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:55.411+0000 7f9ad70f7640 1 -- 172.21.15.44:0/32160318 wait complete. 2024-09-13T17:34:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:55 smithi044 ceph-mon[31175]: pgmap v253: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-13T17:34:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:55 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2457927426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:34:55.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:55 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:34:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:55 smithi070 ceph-mon[34502]: pgmap v253: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-13T17:34:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:55 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2457927426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:34:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:55 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:34:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:55 smithi100 ceph-mon[35055]: pgmap v253: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-13T17:34:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:55 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2457927426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:34:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:55 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:34:56.424 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:34:56.424 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 -- 172.21.15.44:0/992643879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c0ff830 msgr2=0x7f578c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 --2- 172.21.15.44:0/992643879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c0ff830 0x7f578c103870 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f577c009930 tx=0x7f577c02a420 comp rx=0 tx=0).stop 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 -- 172.21.15.44:0/992643879 shutdown_connections 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 --2- 172.21.15.44:0/992643879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c103fd0 0x7f578c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 --2- 172.21.15.44:0/992643879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c0ff830 0x7f578c103870 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 --2- 172.21.15.44:0/992643879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c0fee80 0x7f578c0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 -- 172.21.15.44:0/992643879 >> 172.21.15.44:0/992643879 conn(0x7f578c0fa930 msgr2=0x7f578c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 -- 172.21.15.44:0/992643879 shutdown_connections 2024-09-13T17:34:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.540+0000 7f57946f0640 1 -- 172.21.15.44:0/992643879 wait complete. 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 Processor -- start 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 -- start start 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c0fee80 0x7f578c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 0x7f578c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c19ac00 0x7f578c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c19b5f0 con 0x7f578c103fd0 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c19b760 con 0x7f578c0fee80 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f57946f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c19b8d0 con 0x7f578c19ac00 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f5791c8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 0x7f578c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f5791c8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 0x7f578c196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56022/0 (socket says 172.21.15.44:56022) 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f579248c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c0fee80 0x7f578c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f5791c8b640 1 -- 172.21.15.44:0/1037429263 learned_addr learned my addr 172.21.15.44:0/1037429263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.541+0000 7f5792c8d640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c19ac00 0x7f578c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f5791c8b640 1 -- 172.21.15.44:0/1037429263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c19ac00 msgr2=0x7f578c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f5791c8b640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c19ac00 0x7f578c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f5791c8b640 1 -- 172.21.15.44:0/1037429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c0fee80 msgr2=0x7f578c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f5791c8b640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c0fee80 0x7f578c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f5791c8b640 1 -- 172.21.15.44:0/1037429263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c009560 con 0x7f578c103fd0 2024-09-13T17:34:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f579248c640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c0fee80 0x7f578c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f5791c8b640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 0x7f578c196810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f577c005ab0 tx=0x7f577c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f57837fe640 1 -- 172.21.15.44:0/1037429263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c032e10 con 0x7f578c103fd0 2024-09-13T17:34:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c1a2eb0 con 0x7f578c103fd0 2024-09-13T17:34:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f57837fe640 1 -- 172.21.15.44:0/1037429263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f577c0046c0 con 0x7f578c103fd0 2024-09-13T17:34:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f57837fe640 1 -- 172.21.15.44:0/1037429263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c003850 con 0x7f578c103fd0 2024-09-13T17:34:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.542+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c1a3220 con 0x7f578c103fd0 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.544+0000 7f57837fe640 1 -- 172.21.15.44:0/1037429263 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f577c004830 con 0x7f578c103fd0 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 Processor -- start 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 -- start start 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753fae2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faefd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fa4dc60 con 0x55753fae30e0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fa4b3f0 con 0x55753faed980 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579740e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fa46ce0 con 0x55753fae27c0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579248c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f57837fe640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f57780a1200 0x7f57780a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579248c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56030/0 (socket says 172.21.15.44:56030) 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f579248c640 1 -- 172.21.15.44:0/3731874524 learned_addr learned my addr 172.21.15.44:0/3731874524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f5792c8d640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f57780a1200 0x7f57780a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f5791c8b640 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faefd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f5792c8d640 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753fae2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.545+0000 7f57837fe640 1 -- 172.21.15.44:0/1037429263 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f577c0edc30 con 0x7f578c103fd0 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f579248c640 1 -- 172.21.15.44:0/3731874524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 msgr2=0x55753fae2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f579248c640 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753fae2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f579248c640 1 -- 172.21.15.44:0/3731874524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 msgr2=0x55753faefd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f579248c640 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faefd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f579248c640 1 -- 172.21.15.44:0/3731874524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55753fa02970 con 0x55753fae30e0 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f579248c640 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faed440 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f57880104d0 tx=0x7f5788010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7a6682ddda1d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f5792c8d640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f57780a1200 0x7f57780a36c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5784009fd0 tx=0x7f5784009920 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f5780ff9640 1 -- 172.21.15.44:0/3731874524 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788011760 con 0x55753fae30e0 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f5780ff9640 1 -- 172.21.15.44:0/3731874524 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5788015070 con 0x55753fae30e0 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.546+0000 7f5780ff9640 1 -- 172.21.15.44:0/3731874524 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578801cb70 con 0x55753fae30e0 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 -- 172.21.15.44:0/3731874524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 msgr2=0x55753faed440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faed440 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f57880104d0 tx=0x7f5788010990 comp rx=0 tx=0).stop 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 -- 172.21.15.44:0/3731874524 shutdown_connections 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faefd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faed440 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 --2- 172.21.15.44:0/3731874524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753fae2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 -- 172.21.15.44:0/3731874524 >> 172.21.15.44:0/3731874524 conn(0x55753faded50 msgr2=0x55753fadf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 -- 172.21.15.44:0/3731874524 shutdown_connections 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 -- 172.21.15.44:0/3731874524 wait complete. 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 -- start start 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.547+0000 7f579740e740 1 Processor -- start 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 0x55753faed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faeae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fa04f80 con 0x55753fae27c0 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fa04c20 con 0x55753faed980 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753f9d1ab0 con 0x55753fae30e0 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 10 client.0 _ll_register_callbacks cb 0x55753fa40350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 0x55753faed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 0x55753faed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56040/0 (socket says 172.21.15.44:56040) 2024-09-13T17:34:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 -- 172.21.15.44:0/3788969527 learned_addr learned my addr 172.21.15.44:0/3788969527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f5791c8b640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f5792c8d640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faeae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 -- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 msgr2=0x55753faea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 -- 172.21.15.44:0/3788969527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 msgr2=0x55753faeae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faeae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 -- 172.21.15.44:0/3788969527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578800bcf0 con 0x55753fae27c0 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f5792c8d640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faeae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579248c640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 0x55753faed410 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f578801cae0 tx=0x7f578801cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2667f3c84c087c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f5775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f5775ffb640 1 -- 172.21.15.44:0/3788969527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788021070 con 0x55753fae27c0 2024-09-13T17:34:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f5775ffb640 1 -- 172.21.15.44:0/3788969527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f5788017070 con 0x55753fae27c0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.548+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55753f73a590 con 0x55753fae27c0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f5775ffb640 1 -- 172.21.15.44:0/3788969527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57880050d0 con 0x55753fae27c0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 10 client.18171 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55753fa45360 con 0x55753fae27c0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f5775ffb640 1 -- 172.21.15.44:0/3788969527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55753f73a590 con 0x55753fae27c0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f5775ffb640 1 -- 172.21.15.44:0/3788969527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5788003cb0 con 0x55753fae27c0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 10 client.18171 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 20 client.18171 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fafd6a0 0x55753faffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 4 client.18171 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55753f74b860 con 0x55753fafd6a0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.549+0000 7f5791c8b640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fafd6a0 0x55753faffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f5775ffb640 10 client.18171 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f5791c8b640 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fafd6a0 0x55753faffa90 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf2ee262e19fdc2 server_cookie=60ccc13cc213ffe in_seq=0 out_seq=0 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f5775ffb640 1 -- 172.21.15.44:0/3788969527 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55753f74b860 con 0x55753fafd6a0 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f5775ffb640 10 client.18171 handle_command_reply: tid=0 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 client.18171 shutdown 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 msgr2=0x55753faed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 0x55753faed410 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f578801cae0 tx=0x7f578801cb10 comp rx=0 tx=0).stop 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 shutdown_connections 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fafd6a0 0x55753faffa90 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753faeae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 --2- 172.21.15.44:0/3788969527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae27c0 0x55753faed410 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f5775ffb640 0 client.18171 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.550+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 >> 172.21.15.44:0/3788969527 conn(0x55753faded50 msgr2=0x55753fae41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.551+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 shutdown_connections 2024-09-13T17:34:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.551+0000 7f579740e740 1 -- 172.21.15.44:0/3788969527 wait complete. 2024-09-13T17:34:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.551+0000 7f579740e740 20 client.18171 trim_cache size 0 max 16384 2024-09-13T17:34:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 Processor -- start 2024-09-13T17:34:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 -- start start 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753faf4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faf4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753fae9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fae54a0 con 0x55753fae30e0 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fae2ed0 con 0x55753faed980 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579740e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fb04be0 con 0x55753fae27c0 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f5791c8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faf4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f5791c8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faf4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56054/0 (socket says 172.21.15.44:56054) 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f5791c8b640 1 -- 172.21.15.44:0/2081301828 learned_addr learned my addr 172.21.15.44:0/2081301828 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f5792c8d640 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753fae9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.565+0000 7f579248c640 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753faf4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f5791c8b640 1 -- 172.21.15.44:0/2081301828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 msgr2=0x55753faf4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f5791c8b640 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753faf4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f5791c8b640 1 -- 172.21.15.44:0/2081301828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 msgr2=0x55753fae9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f5791c8b640 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753fae9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f5791c8b640 1 -- 172.21.15.44:0/2081301828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55753f9e71c0 con 0x55753fae30e0 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f5791c8b640 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faf4b90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b7490 tx=0x7f577c02fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834fea88da1ff731 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f57817fa640 1 -- 172.21.15.44:0/2081301828 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c037300 con 0x55753fae30e0 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f57817fa640 1 -- 172.21.15.44:0/2081301828 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f577c0d2070 con 0x55753fae30e0 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f57817fa640 1 -- 172.21.15.44:0/2081301828 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0c2e80 con 0x55753fae30e0 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f579740e740 1 -- 172.21.15.44:0/2081301828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 msgr2=0x55753faf4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.566+0000 7f579740e740 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faf4b90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b7490 tx=0x7f577c02fc00 comp rx=0 tx=0).stop 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- 172.21.15.44:0/2081301828 shutdown_connections 2024-09-13T17:34:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753faed980 0x55753fae9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753fae30e0 0x55753faf4b90 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 --2- 172.21.15.44:0/2081301828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae27c0 0x55753faf4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- 172.21.15.44:0/2081301828 >> 172.21.15.44:0/2081301828 conn(0x55753faded50 msgr2=0x55753fae5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- 172.21.15.44:0/2081301828 shutdown_connections 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- 172.21.15.44:0/2081301828 wait complete. 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- start start 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 Processor -- start 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753fae27c0 0x55753faf3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faf3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 0x55753faf3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fb53b80 con 0x55753faed980 2024-09-13T17:34:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fb53cf0 con 0x55753fae27c0 2024-09-13T17:34:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753fb53eb0 con 0x55753fae30e0 2024-09-13T17:34:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579740e740 10 client.0 _ll_register_callbacks cb 0x7f578800bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f5792c8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 0x55753faf3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f5792c8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 0x55753faf3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56064/0 (socket says 172.21.15.44:56064) 2024-09-13T17:34:56.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f5792c8d640 1 -- 172.21.15.44:0/285506305 learned_addr learned my addr 172.21.15.44:0/285506305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f5791c8b640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faf3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.567+0000 7f579248c640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753fae27c0 0x55753faf3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5792c8d640 1 -- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 msgr2=0x55753faf3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5792c8d640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faf3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5792c8d640 1 -- 172.21.15.44:0/285506305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753fae27c0 msgr2=0x55753faf3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5792c8d640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753fae27c0 0x55753faf3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5792c8d640 1 -- 172.21.15.44:0/285506305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c1a3220 con 0x55753faed980 2024-09-13T17:34:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f579248c640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753fae27c0 0x55753faf3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:34:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5791c8b640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faf3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5792c8d640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 0x55753faf3b90 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5784010cf0 tx=0x7f5784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4d86986649a046 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5775ffb640 1 -- 172.21.15.44:0/285506305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578402b5f0 con 0x55753faed980 2024-09-13T17:34:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5775ffb640 1 -- 172.21.15.44:0/285506305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f578402b750 con 0x55753faed980 2024-09-13T17:34:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5788015070 con 0x55753faed980 2024-09-13T17:34:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5775ffb640 1 -- 172.21.15.44:0/285506305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f578402b910 con 0x55753faed980 2024-09-13T17:34:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f579740e740 10 client.18177 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55753fa0c990 con 0x55753faed980 2024-09-13T17:34:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5775ffb640 1 -- 172.21.15.44:0/285506305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5788015070 con 0x55753faed980 2024-09-13T17:34:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.568+0000 7f5775ffb640 1 -- 172.21.15.44:0/285506305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5784012d20 con 0x55753faed980 2024-09-13T17:34:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579740e740 10 client.18177 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579740e740 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579740e740 20 client.18177 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579740e740 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fb557c0 0x55753fb57be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579740e740 4 client.18177 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:34:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55753f79eeb0 con 0x55753fb557c0 2024-09-13T17:34:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579248c640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fb557c0 0x55753fb57be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f5775ffb640 10 client.18177 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.569+0000 7f579248c640 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fb557c0 0x55753fb57be0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11110685a542863f server_cookie=e0d4b47355d6baff in_seq=0 out_seq=0 2024-09-13T17:34:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f5775ffb640 1 -- 172.21.15.44:0/285506305 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55753f79eeb0 con 0x55753fb557c0 2024-09-13T17:34:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f5775ffb640 10 client.18177 handle_command_reply: tid=0 2024-09-13T17:34:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 client.18177 shutdown 2024-09-13T17:34:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 msgr2=0x55753faf3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 0x55753faf3b90 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5784010cf0 tx=0x7f5784004a40 comp rx=0 tx=0).stop 2024-09-13T17:34:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 shutdown_connections 2024-09-13T17:34:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55753fb557c0 0x55753fb57be0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55753fae30e0 0x55753faf3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55753faed980 0x55753faf3b90 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 --2- 172.21.15.44:0/285506305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55753fae27c0 0x55753faf3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f5775ffb640 0 client.18177 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 >> 172.21.15.44:0/285506305 conn(0x55753faded50 msgr2=0x55753fae41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 shutdown_connections 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 1 -- 172.21.15.44:0/285506305 wait complete. 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.570+0000 7f579740e740 20 client.18177 trim_cache size 0 max 16384 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "de35a4d8-136e-44e4-81ab-972fbca6776f", 2024-09-13T17:34:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:34:56.587 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f57780a1200 msgr2=0x7f57780a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f57780a1200 0x7f57780a36c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5784009fd0 tx=0x7f5784009920 comp rx=0 tx=0).stop 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 msgr2=0x7f578c196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 0x7f578c196810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f577c005ab0 tx=0x7f577c032c20 comp rx=0 tx=0).stop 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 shutdown_connections 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f57780a1200 0x7f57780a36c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c19ac00 0x7f578c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c103fd0 0x7f578c196810 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 --2- 172.21.15.44:0/1037429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578c0fee80 0x7f578c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 >> 172.21.15.44:0/1037429263 conn(0x7f578c0fa930 msgr2=0x7f578c102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 shutdown_connections 2024-09-13T17:34:56.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.571+0000 7f57946f0640 1 -- 172.21.15.44:0/1037429263 wait complete. 2024-09-13T17:34:56.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:34:56.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 -- 172.21.15.44:0/3920349423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50340ff830 msgr2=0x7f5034103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 --2- 172.21.15.44:0/3920349423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50340ff830 0x7f5034103870 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5028009930 tx=0x7f502802a420 comp rx=0 tx=0).stop 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 -- 172.21.15.44:0/3920349423 shutdown_connections 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 --2- 172.21.15.44:0/3920349423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5034103fd0 0x7f503410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 --2- 172.21.15.44:0/3920349423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50340ff830 0x7f5034103870 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 --2- 172.21.15.44:0/3920349423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50340fee80 0x7f50340ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 -- 172.21.15.44:0/3920349423 >> 172.21.15.44:0/3920349423 conn(0x7f50340fa8d0 msgr2=0x7f50340fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 -- 172.21.15.44:0/3920349423 shutdown_connections 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.691+0000 7f503c583640 1 -- 172.21.15.44:0/3920349423 wait complete. 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 Processor -- start 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 -- start start 2024-09-13T17:34:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50340fee80 0x7f50341961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5034103fd0 0x7f5034196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 0x7f503419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503419b510 con 0x7f503419ab20 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503419b680 con 0x7f5034103fd0 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503c583640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503419b7f0 con 0x7f50340fee80 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503ab20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 0x7f503419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503ab20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 0x7f503419afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56082/0 (socket says 172.21.15.44:56082) 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503ab20640 1 -- 172.21.15.44:0/1323304425 learned_addr learned my addr 172.21.15.44:0/1323304425 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.692+0000 7f503a31f640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50340fee80 0x7f50341961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f5039b1e640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5034103fd0 0x7f5034196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503ab20640 1 -- 172.21.15.44:0/1323304425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50340fee80 msgr2=0x7f50341961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503ab20640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50340fee80 0x7f50341961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503ab20640 1 -- 172.21.15.44:0/1323304425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5034103fd0 msgr2=0x7f5034196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503ab20640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5034103fd0 0x7f5034196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503ab20640 1 -- 172.21.15.44:0/1323304425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028009560 con 0x7f503419ab20 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503ab20640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 0x7f503419afd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f502c00eb10 tx=0x7f502c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f50277fe640 1 -- 172.21.15.44:0/1323304425 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c004b20 con 0x7f503419ab20 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f50277fe640 1 -- 172.21.15.44:0/1323304425 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f502c004c80 con 0x7f503419ab20 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50341a2dd0 con 0x7f503419ab20 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f50277fe640 1 -- 172.21.15.44:0/1323304425 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c005a10 con 0x7f503419ab20 2024-09-13T17:34:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.693+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50341a3200 con 0x7f503419ab20 2024-09-13T17:34:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f50277fe640 1 -- 172.21.15.44:0/1323304425 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f502c005020 con 0x7f503419ab20 2024-09-13T17:34:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 Processor -- start 2024-09-13T17:34:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 -- start start 2024-09-13T17:34:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 0x55d0c90b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9022f20 con 0x55d0c90b7e10 2024-09-13T17:34:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c90206b0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503f2ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c901bfa0 con 0x55d0c90b8730 2024-09-13T17:34:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 0x55d0c90b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 0x55d0c90b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56088/0 (socket says 172.21.15.44:56088) 2024-09-13T17:34:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f5039b1e640 1 -- 172.21.15.44:0/115236744 learned_addr learned my addr 172.21.15.44:0/115236744 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.696+0000 7f503a31f640 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f50277fe640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f50200a1200 0x7f50200a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 -- 172.21.15.44:0/115236744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 msgr2=0x55d0c90c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 -- 172.21.15.44:0/115236744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 msgr2=0x55d0c90c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 -- 172.21.15.44:0/115236744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0c8fd6840 con 0x55d0c90b7e10 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f503ab20640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f50200a1200 0x7f50200a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f50277fe640 1 -- 172.21.15.44:0/1323304425 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f502c014070 con 0x7f503419ab20 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5039b1e640 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 0x55d0c90b81f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f5028000c00 tx=0x7f5028005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a061be706179f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5024ff9640 1 -- 172.21.15.44:0/115236744 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028044070 con 0x55d0c90b7e10 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5024ff9640 1 -- 172.21.15.44:0/115236744 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5028005d50 con 0x55d0c90b7e10 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.697+0000 7f5024ff9640 1 -- 172.21.15.44:0/115236744 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028003850 con 0x55d0c90b7e10 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 -- 172.21.15.44:0/115236744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 msgr2=0x55d0c90b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503ab20640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f50200a1200 0x7f50200a36c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f502c0cb050 tx=0x7f502c08fb60 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 0x55d0c90b81f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f5028000c00 tx=0x7f5028005720 comp rx=0 tx=0).stop 2024-09-13T17:34:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 -- 172.21.15.44:0/115236744 shutdown_connections 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 --2- 172.21.15.44:0/115236744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b7e10 0x55d0c90b81f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 -- 172.21.15.44:0/115236744 >> 172.21.15.44:0/115236744 conn(0x55d0c90b4380 msgr2=0x55d0c90b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 -- 172.21.15.44:0/115236744 shutdown_connections 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 -- 172.21.15.44:0/115236744 wait complete. 2024-09-13T17:34:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 -- start start 2024-09-13T17:34:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.698+0000 7f503f2ac740 1 Processor -- start 2024-09-13T17:34:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c8fbd2a0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c8cf4430 con 0x55d0c90b7e10 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c8f9f680 con 0x55d0c90b8730 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 10 client.0 _ll_register_callbacks cb 0x55d0c9015610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56104/0 (socket says 172.21.15.44:56104) 2024-09-13T17:34:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 -- 172.21.15.44:0/3221981506 learned_addr learned my addr 172.21.15.44:0/3221981506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503a31f640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 -- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 msgr2=0x55d0c90bfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90bfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 -- 172.21.15.44:0/3221981506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 msgr2=0x55d0c90c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 -- 172.21.15.44:0/3221981506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5034068730 con 0x55d0c90c2fd0 2024-09-13T17:34:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c0200 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f5028040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf0222c7ee9755b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f501dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f501dffb640 1 -- 172.21.15.44:0/3221981506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028044070 con 0x55d0c90c2fd0 2024-09-13T17:34:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f501dffb640 1 -- 172.21.15.44:0/3221981506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f50280035f0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.699+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0c8d0f4e0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f501dffb640 1 -- 172.21.15.44:0/3221981506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50280037b0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 10 client.18189 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d0c901a620 con 0x55d0c90c2fd0 2024-09-13T17:34:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f501dffb640 1 -- 172.21.15.44:0/3221981506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0c8d0f4e0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f501dffb640 1 -- 172.21.15.44:0/3221981506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5028055030 con 0x55d0c90c2fd0 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 10 client.18189 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 20 client.18189 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c90d29f0 0x55d0c90d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 4 client.18189 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0c8d207e0 con 0x55d0c90d29f0 2024-09-13T17:34:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.700+0000 7f503a31f640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c90d29f0 0x55d0c90d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f501dffb640 10 client.18189 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503a31f640 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c90d29f0 0x55d0c90d4e10 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1937e3dae496e7 server_cookie=999397306b09ee98 in_seq=0 out_seq=0 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f501dffb640 1 -- 172.21.15.44:0/3221981506 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d0c8d207e0 con 0x55d0c90d29f0 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f501dffb640 10 client.18189 handle_command_reply: tid=0 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 client.18189 shutdown 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 msgr2=0x55d0c90c0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c0200 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f5028040040 comp rx=0 tx=0).stop 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 shutdown_connections 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c90d29f0 0x55d0c90d4e10 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90bfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c0200 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3221981506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f501dffb640 0 client.18189 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 >> 172.21.15.44:0/3221981506 conn(0x55d0c90b4380 msgr2=0x55d0c90ba0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 shutdown_connections 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 1 -- 172.21.15.44:0/3221981506 wait complete. 2024-09-13T17:34:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.701+0000 7f503f2ac740 20 client.18189 trim_cache size 0 max 16384 2024-09-13T17:34:56.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 Processor -- start 2024-09-13T17:34:56.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 -- start start 2024-09-13T17:34:56.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c90b5510 con 0x55d0c90c2fd0 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c90be190 con 0x55d0c90b7e10 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503f2ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c90c0510 con 0x55d0c90b8730 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56110/0 (socket says 172.21.15.44:56110) 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 -- 172.21.15.44:0/3513089638 learned_addr learned my addr 172.21.15.44:0/3513089638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503a31f640 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 -- 172.21.15.44:0/3513089638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 msgr2=0x55d0c90b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 -- 172.21.15.44:0/3513089638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 msgr2=0x55d0c90b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 -- 172.21.15.44:0/3513089638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0c8fbc790 con 0x55d0c90c2fd0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f503a31f640 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.716+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c9d90 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5028005ec0 tx=0x7f5028031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0d68cc788462d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f50257fa640 1 -- 172.21.15.44:0/3513089638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028044040 con 0x55d0c90c2fd0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f50257fa640 1 -- 172.21.15.44:0/3513089638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5028049af0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f50257fa640 1 -- 172.21.15.44:0/3513089638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028049500 con 0x55d0c90c2fd0 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 -- 172.21.15.44:0/3513089638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 msgr2=0x55d0c90c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c9d90 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5028005ec0 tx=0x7f5028031750 comp rx=0 tx=0).stop 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 -- 172.21.15.44:0/3513089638 shutdown_connections 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b8730 0x55d0c90b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c9d90 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3513089638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90b7e10 0x55d0c90b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 -- 172.21.15.44:0/3513089638 >> 172.21.15.44:0/3513089638 conn(0x55d0c90b4380 msgr2=0x55d0c90d9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 -- 172.21.15.44:0/3513089638 shutdown_connections 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 -- 172.21.15.44:0/3513089638 wait complete. 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.717+0000 7f503f2ac740 1 -- start start 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 Processor -- start 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b7e10 0x55d0c90c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 0x55d0c90c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9128080 con 0x55d0c90b8730 2024-09-13T17:34:56.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c91281f0 con 0x55d0c90c2fd0 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c91283b0 con 0x55d0c90b7e10 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503f2ac740 10 client.0 _ll_register_callbacks cb 0x7f50300097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 0x55d0c90c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f5039b1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b7e10 0x55d0c90c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 0x55d0c90c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56122/0 (socket says 172.21.15.44:56122) 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 -- 172.21.15.44:0/3241334733 learned_addr learned my addr 172.21.15.44:0/3241334733 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 -- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b7e10 msgr2=0x55d0c90c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b7e10 0x55d0c90c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 -- 172.21.15.44:0/3241334733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 msgr2=0x55d0c90c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 -- 172.21.15.44:0/3241334733 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5030010430 con 0x55d0c90b8730 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f503a31f640 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 0x55d0c90c62b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5030009730 tx=0x7f5030010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a7c5b83eadec37 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f501dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.718+0000 7f501dffb640 1 -- 172.21.15.44:0/3241334733 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030017040 con 0x55d0c90b8730 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f501dffb640 1 -- 172.21.15.44:0/3241334733 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f50300263e0 con 0x55d0c90b8730 2024-09-13T17:34:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5028005d50 con 0x55d0c90b8730 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f501dffb640 1 -- 172.21.15.44:0/3241334733 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5030027640 con 0x55d0c90b8730 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 10 client.18195 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d0c901a620 con 0x55d0c90b8730 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f501dffb640 1 -- 172.21.15.44:0/3241334733 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5028005d50 con 0x55d0c90b8730 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f501dffb640 1 -- 172.21.15.44:0/3241334733 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f503001d040 con 0x55d0c90b8730 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 10 client.18195 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 20 client.18195 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c9129b70 0x55d0c912bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 4 client.18195 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0c8d73e30 con 0x55d0c9129b70 2024-09-13T17:34:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.719+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c9129b70 0x55d0c912bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f5039b1e640 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c9129b70 0x55d0c912bf60 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3187a776bbe9039 server_cookie=50cf03664a95c89e in_seq=0 out_seq=0 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f501dffb640 10 client.18195 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f501dffb640 1 -- 172.21.15.44:0/3241334733 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d0c8d73e30 con 0x55d0c9129b70 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f501dffb640 10 client.18195 handle_command_reply: tid=0 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 client.18195 shutdown 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 msgr2=0x55d0c90c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 0x55d0c90c62b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5030009730 tx=0x7f5030010990 comp rx=0 tx=0).stop 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 shutdown_connections 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d0c9129b70 0x55d0c912bf60 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d0c90c2fd0 0x55d0c90c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0c90b8730 0x55d0c90c62b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 --2- 172.21.15.44:0/3241334733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0c90b7e10 0x55d0c90c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 >> 172.21.15.44:0/3241334733 conn(0x55d0c90b4380 msgr2=0x55d0c90b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f501dffb640 0 client.18195 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 shutdown_connections 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 1 -- 172.21.15.44:0/3241334733 wait complete. 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.720+0000 7f503f2ac740 20 client.18195 trim_cache size 0 max 16384 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:34:56.740 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout: "de35a4d8-136e-44e4-81ab-972fbca6776f": { 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "de35a4d8-136e-44e4-81ab-972fbca6776f", 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:34:56.741 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f50200a1200 msgr2=0x7f50200a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f50200a1200 0x7f50200a36c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f502c0cb050 tx=0x7f502c08fb60 comp rx=0 tx=0).stop 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 msgr2=0x7f503419afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 0x7f503419afd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f502c00eb10 tx=0x7f502c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 shutdown_connections 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f50200a1200 0x7f50200a36c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f503419ab20 0x7f503419afd0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5034103fd0 0x7f5034196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 --2- 172.21.15.44:0/1323304425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50340fee80 0x7f50341961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 >> 172.21.15.44:0/1323304425 conn(0x7f50340fa8d0 msgr2=0x7f50340fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 shutdown_connections 2024-09-13T17:34:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:34:56.721+0000 7f503c583640 1 -- 172.21.15.44:0/1323304425 wait complete. 2024-09-13T17:34:56.743 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:56 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:34:56.743 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:56 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:34:56.743 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:56 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:34:56.744 INFO:tasks.cephfs.filesystem:scrub status for tag:de35a4d8-136e-44e4-81ab-972fbca6776f - {'path': '/', 'tag': 'de35a4d8-136e-44e4-81ab-972fbca6776f', 'options': 'recursive,force'} 2024-09-13T17:34:56.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:56 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:34:56.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:56 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:34:56.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:56 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:34:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:56 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:34:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:56 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:34:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:56 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:34:58.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:57 smithi044 ceph-mon[31175]: pgmap v254: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:34:58.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:57 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:34:58.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:57 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:58.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:57 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:34:58.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:57 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:57 smithi070 ceph-mon[34502]: pgmap v254: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:34:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:57 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:34:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:57 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:57 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:34:58.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:57 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:57 smithi100 ceph-mon[35055]: pgmap v254: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:34:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:57 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:34:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:57 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:57 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:34:58.274 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:57 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:34:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 -- 172.21.15.44:0/1168121447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 msgr2=0x7f8d4410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/1168121447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4410e910 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009900 tx=0x7f8d3c02a480 comp rx=0 tx=0).stop 2024-09-13T17:34:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 -- 172.21.15.44:0/1168121447 shutdown_connections 2024-09-13T17:34:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/1168121447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d44073200 0x7f8d4410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/1168121447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4410e910 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/1168121447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d440728e0 0x7f8d44072cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 -- 172.21.15.44:0/1168121447 >> 172.21.15.44:0/1168121447 conn(0x7f8d440fb960 msgr2=0x7f8d440fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 -- 172.21.15.44:0/1168121447 shutdown_connections 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.090+0000 7f8d4b74c640 1 -- 172.21.15.44:0/1168121447 wait complete. 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 Processor -- start 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 -- start start 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d440728e0 0x7f8d44198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d44073200 0x7f8d44198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4419d230 con 0x7f8d4410c4c0 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4419d3a0 con 0x7f8d440728e0 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d4b74c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4419d560 con 0x7f8d44073200 2024-09-13T17:34:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d49ce9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d49ce9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56128/0 (socket says 172.21.15.44:56128) 2024-09-13T17:34:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d49ce9640 1 -- 172.21.15.44:0/3843419136 learned_addr learned my addr 172.21.15.44:0/3843419136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.091+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d440728e0 0x7f8d44198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d44073200 0x7f8d44198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d49ce9640 1 -- 172.21.15.44:0/3843419136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d44073200 msgr2=0x7f8d44198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d49ce9640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d44073200 0x7f8d44198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d49ce9640 1 -- 172.21.15.44:0/3843419136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d440728e0 msgr2=0x7f8d44198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d49ce9640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d440728e0 0x7f8d44198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d49ce9640 1 -- 172.21.15.44:0/3843419136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c009560 con 0x7f8d4410c4c0 2024-09-13T17:34:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d49ce9640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4419cc30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c02aa70 tx=0x7f8d3c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d427fc640 1 -- 172.21.15.44:0/3843419136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c005d70 con 0x7f8d4410c4c0 2024-09-13T17:34:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d427fc640 1 -- 172.21.15.44:0/3843419136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d3c032e10 con 0x7f8d4410c4c0 2024-09-13T17:34:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4419d7e0 con 0x7f8d4410c4c0 2024-09-13T17:34:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d427fc640 1 -- 172.21.15.44:0/3843419136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003850 con 0x7f8d4410c4c0 2024-09-13T17:34:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.092+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4419dab0 con 0x7f8d4410c4c0 2024-09-13T17:34:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d427fc640 1 -- 172.21.15.44:0/3843419136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d3c005210 con 0x7f8d4410c4c0 2024-09-13T17:34:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 Processor -- start 2024-09-13T17:34:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 -- start start 2024-09-13T17:34:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 0x55bc0e354710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e2bfc30 con 0x55bc0e354330 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e2bd3c0 con 0x55bc0e354c50 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.095+0000 7f8d4e470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e2b8cb0 con 0x55bc0e35f4f0 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 0x55bc0e354710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 0x55bc0e354710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56136/0 (socket says 172.21.15.44:56136) 2024-09-13T17:34:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2787422942 learned_addr learned my addr 172.21.15.44:0/2787422942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d494e8640 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d427fc640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d2c0a1200 0x7f8d2c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2787422942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e35f4f0 msgr2=0x55bc0e3618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2787422942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 msgr2=0x55bc0e35efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d49ce9640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d2c0a1200 0x7f8d2c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2787422942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0e25f910 con 0x55bc0e354330 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d427fc640 1 -- 172.21.15.44:0/3843419136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8d3c0ee050 con 0x7f8d4410c4c0 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 0x55bc0e354710 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005fd0 tx=0x7f8d3800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd198bdf5c1bce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d2b7fe640 1 -- 172.21.15.44:0/2787422942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38005c50 con 0x55bc0e354330 2024-09-13T17:34:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d2b7fe640 1 -- 172.21.15.44:0/2787422942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d380054b0 con 0x55bc0e354330 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.096+0000 7f8d2b7fe640 1 -- 172.21.15.44:0/2787422942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38011440 con 0x55bc0e354330 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 -- 172.21.15.44:0/2787422942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 msgr2=0x55bc0e354710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 0x55bc0e354710 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005fd0 tx=0x7f8d3800b610 comp rx=0 tx=0).stop 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d49ce9640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d2c0a1200 0x7f8d2c0a36c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0bac10 tx=0x7f8d3c009260 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 -- 172.21.15.44:0/2787422942 shutdown_connections 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2787422942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e354330 0x55bc0e354710 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 -- 172.21.15.44:0/2787422942 >> 172.21.15.44:0/2787422942 conn(0x55bc0e3508c0 msgr2=0x55bc0e350ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 -- 172.21.15.44:0/2787422942 shutdown_connections 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.097+0000 7f8d4e470740 1 -- 172.21.15.44:0/2787422942 wait complete. 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 -- start start 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 Processor -- start 2024-09-13T17:34:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354330 0x55bc0e35eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e35c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e276e80 con 0x55bc0e35f4f0 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e276b20 con 0x55bc0e354c50 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e23c260 con 0x55bc0e354330 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d4e470740 10 client.0 _ll_register_callbacks cb 0x55bc0e271df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e35c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e35c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56144/0 (socket says 172.21.15.44:56144) 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d48ce7640 1 -- 172.21.15.44:0/3972109415 learned_addr learned my addr 172.21.15.44:0/3972109415 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.098+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354330 0x55bc0e35eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d494e8640 1 -- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354330 msgr2=0x55bc0e35eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354330 0x55bc0e35eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d494e8640 1 -- 172.21.15.44:0/3972109415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 msgr2=0x55bc0e35c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e35c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d494e8640 1 -- 172.21.15.44:0/3972109415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3800bd60 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e35c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35c1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005fd0 tx=0x7f8d34009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5038b8ec370897c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3972109415 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34004d20 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc0e2e1680 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3972109415 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8d34013910 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3972109415 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d34004e80 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d4e470740 10 client.27377 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.099+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bc0e2b7330 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3972109415 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc0e2e1680 con 0x55bc0e354c50 2024-09-13T17:34:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3972109415 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d3401b3f0 con 0x55bc0e354c50 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d4e470740 10 client.27377 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d4e470740 10 client.27377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d4e470740 20 client.27377 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e36ef40 0x55bc0e371360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d4e470740 4 client.27377 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc0e2ac5d0 con 0x55bc0e36ef40 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e36ef40 0x55bc0e371360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e36ef40 0x55bc0e371360 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d328219c92d491e server_cookie=b286d1341808d958 in_seq=0 out_seq=0 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.100+0000 7f8d28ff9640 10 client.27377 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3972109415 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bc0e2ac5d0 con 0x55bc0e36ef40 2024-09-13T17:34:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d28ff9640 10 client.27377 handle_command_reply: tid=0 2024-09-13T17:34:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 client.27377 shutdown 2024-09-13T17:34:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 msgr2=0x55bc0e35c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35c1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005fd0 tx=0x7f8d34009f90 comp rx=0 tx=0).stop 2024-09-13T17:34:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 shutdown_connections 2024-09-13T17:34:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e36ef40 0x55bc0e371360 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e35c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d28ff9640 0 client.27377 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354c50 0x55bc0e35c1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3972109415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354330 0x55bc0e35eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 >> 172.21.15.44:0/3972109415 conn(0x55bc0e3508c0 msgr2=0x55bc0e34fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 shutdown_connections 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.101+0000 7f8d4e470740 1 -- 172.21.15.44:0/3972109415 wait complete. 2024-09-13T17:34:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.102+0000 7f8d4e470740 20 client.27377 trim_cache size 0 max 16384 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.114+0000 7f8d4e470740 1 Processor -- start 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.114+0000 7f8d4e470740 1 -- start start 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e350300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e357410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e357950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d4e470740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e376530 con 0x55bc0e35f4f0 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d4e470740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e35cde0 con 0x55bc0e354330 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d4e470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e35eac0 con 0x55bc0e354c50 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e357950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e357950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56154/0 (socket says 172.21.15.44:56154) 2024-09-13T17:34:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2250957974 learned_addr learned my addr 172.21.15.44:0/2250957974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d494e8640 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e357410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e350300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2250957974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 msgr2=0x55bc0e357410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e357410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2250957974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 msgr2=0x55bc0e350300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e350300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 -- 172.21.15.44:0/2250957974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0e259300 con 0x55bc0e35f4f0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.115+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e357950 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8d380134f0 tx=0x7f8d38015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb43d268f649ee57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d2bfff640 1 -- 172.21.15.44:0/2250957974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38034070 con 0x55bc0e35f4f0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d2bfff640 1 -- 172.21.15.44:0/2250957974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d38005200 con 0x55bc0e35f4f0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d2bfff640 1 -- 172.21.15.44:0/2250957974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3802e660 con 0x55bc0e35f4f0 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 -- 172.21.15.44:0/2250957974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 msgr2=0x55bc0e357950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e357950 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8d380134f0 tx=0x7f8d38015040 comp rx=0 tx=0).stop 2024-09-13T17:34:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 -- 172.21.15.44:0/2250957974 shutdown_connections 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e357410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e357950 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 --2- 172.21.15.44:0/2250957974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e350300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 -- 172.21.15.44:0/2250957974 >> 172.21.15.44:0/2250957974 conn(0x55bc0e3508c0 msgr2=0x55bc0e36ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 -- 172.21.15.44:0/2250957974 shutdown_connections 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.116+0000 7f8d4e470740 1 -- 172.21.15.44:0/2250957974 wait complete. 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 -- start start 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 Processor -- start 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e3628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e362e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e3c6290 con 0x55bc0e35f4f0 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e3c6400 con 0x55bc0e354330 2024-09-13T17:34:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0e3c65c0 con 0x55bc0e354c50 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d4e470740 10 client.0 _ll_register_callbacks cb 0x7f8d3800b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d48ce7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56170/0 (socket says 172.21.15.44:56170) 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d494e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e362e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.117+0000 7f8d48ce7640 1 -- 172.21.15.44:0/3989314651 learned_addr learned my addr 172.21.15.44:0/3989314651 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e3628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 -- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 msgr2=0x55bc0e362e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e362e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 -- 172.21.15.44:0/3989314651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 msgr2=0x55bc0e3628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e3628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 -- 172.21.15.44:0/3989314651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d38026040 con 0x55bc0e35f4f0 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e362e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:34:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d48ce7640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3c5bc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005f70 tx=0x7f8d38005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b143ca8dee41fa11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3989314651 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38034070 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3989314651 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8d38005620 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d380054b0 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3989314651 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d3802e5e0 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d4e470740 10 client.18213 fetch_fsmap learned FSMap version 25 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.118+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bc0e27e8e0 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3989314651 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d380054b0 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3989314651 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d3803b070 con 0x55bc0e35f4f0 2024-09-13T17:34:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d4e470740 10 client.18213 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:34:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d4e470740 10 client.18213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:34:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d4e470740 20 client.18213 populate_metadata read hostname 'smithi044' 2024-09-13T17:34:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e3ceab0 0x55bc0e3d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:34:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d4e470740 4 client.18213 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:34:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc0dfac300 con 0x55bc0e3ceab0 2024-09-13T17:34:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.119+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e3ceab0 0x55bc0e3d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:34:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.120+0000 7f8d28ff9640 10 client.18213 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.120+0000 7f8d494e8640 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e3ceab0 0x55bc0e3d0ed0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406a9fceae9a25dd server_cookie=7ec6433219b40e62 in_seq=0 out_seq=0 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.132+0000 7f8d28ff9640 1 -- 172.21.15.44:0/3989314651 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc0dfac300 con 0x55bc0e3ceab0 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.132+0000 7f8d28ff9640 10 client.18213 handle_command_reply: tid=0 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.132+0000 7f8d4e470740 1 client.18213 shutdown 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 msgr2=0x55bc0e3c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3c5bc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005f70 tx=0x7f8d38005d10 comp rx=0 tx=0).stop 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 shutdown_connections 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55bc0e3ceab0 0x55bc0e3d0ed0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc0e354c50 0x55bc0e362e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc0e35f4f0 0x55bc0e3c5bc0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 --2- 172.21.15.44:0/3989314651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bc0e354330 0x55bc0e3628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d28ff9640 0 client.18213 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 >> 172.21.15.44:0/3989314651 conn(0x55bc0e3508c0 msgr2=0x55bc0e36ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 shutdown_connections 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 1 -- 172.21.15.44:0/3989314651 wait complete. 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4e470740 20 client.18213 trim_cache size 0 max 16384 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:34:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:34:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:34:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d2c0a1200 msgr2=0x7f8d2c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d2c0a1200 0x7f8d2c0a36c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0bac10 tx=0x7f8d3c009260 comp rx=0 tx=0).stop 2024-09-13T17:34:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 msgr2=0x7f8d4419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.133+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4419cc30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c02aa70 tx=0x7f8d3c032c20 comp rx=0 tx=0).stop 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 shutdown_connections 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8d2c0a1200 0x7f8d2c0a36c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d44073200 0x7f8d44198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d4410c4c0 0x7f8d4419cc30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 --2- 172.21.15.44:0/3843419136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d440728e0 0x7f8d44198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 >> 172.21.15.44:0/3843419136 conn(0x7f8d440fb960 msgr2=0x7f8d440fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 shutdown_connections 2024-09-13T17:34:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:34:59.134+0000 7f8d4b74c640 1 -- 172.21.15.44:0/3843419136 wait complete. 2024-09-13T17:35:00.076 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:59 smithi100 ceph-mon[35055]: pgmap v255: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-09-13T17:35:00.077 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:34:59 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:35:00.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:59 smithi044 ceph-mon[31175]: pgmap v255: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-09-13T17:35:00.110 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:34:59 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:35:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:59 smithi070 ceph-mon[34502]: pgmap v255: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-09-13T17:35:00.197 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:34:59 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:35:02.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:02 smithi044 ceph-mon[31175]: pgmap v256: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:35:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:02 smithi070 ceph-mon[34502]: pgmap v256: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:35:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:02 smithi100 ceph-mon[35055]: pgmap v256: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:35:04.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:04 smithi044 ceph-mon[31175]: pgmap v257: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:35:04.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:04 smithi070 ceph-mon[34502]: pgmap v257: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:35:04.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:04 smithi100 ceph-mon[35055]: pgmap v257: 129 pgs: 129 active+clean; 2.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:35:05.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:05 smithi044 ceph-mon[31175]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T17:35:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:05 smithi070 ceph-mon[34502]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T17:35:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:05 smithi100 ceph-mon[35055]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T17:35:06.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:06 smithi044 ceph-mon[31175]: pgmap v259: 129 pgs: 129 active+clean; 3.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-13T17:35:06.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:06 smithi070 ceph-mon[34502]: pgmap v259: 129 pgs: 129 active+clean; 3.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-13T17:35:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:06 smithi100 ceph-mon[35055]: pgmap v259: 129 pgs: 129 active+clean; 3.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-13T17:35:08.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:08 smithi044 ceph-mon[31175]: pgmap v260: 129 pgs: 129 active+clean; 3.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-13T17:35:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:08 smithi070 ceph-mon[34502]: pgmap v260: 129 pgs: 129 active+clean; 3.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-13T17:35:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:08 smithi100 ceph-mon[35055]: pgmap v260: 129 pgs: 129 active+clean; 3.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-13T17:35:10.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:10 smithi044 ceph-mon[31175]: pgmap v261: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-13T17:35:10.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:10 smithi070 ceph-mon[34502]: pgmap v261: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-13T17:35:10.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:10 smithi100 ceph-mon[35055]: pgmap v261: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-13T17:35:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 -- 172.21.15.44:0/3060127893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c41017a0 msgr2=0x7ff4c4109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 --2- 172.21.15.44:0/3060127893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c41017a0 0x7ff4c4109c80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009900 tx=0x7ff4b802a480 comp rx=0 tx=0).stop 2024-09-13T17:35:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 -- 172.21.15.44:0/3060127893 shutdown_connections 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 --2- 172.21.15.44:0/3060127893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c410a2a0 0x7ff4c410c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 --2- 172.21.15.44:0/3060127893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c41017a0 0x7ff4c4109c80 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 --2- 172.21.15.44:0/3060127893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 0x7ff4c4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 -- 172.21.15.44:0/3060127893 >> 172.21.15.44:0/3060127893 conn(0x7ff4c40fa8f0 msgr2=0x7ff4c40fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 -- 172.21.15.44:0/3060127893 shutdown_connections 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 -- 172.21.15.44:0/3060127893 wait complete. 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 Processor -- start 2024-09-13T17:35:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.234+0000 7ff4cc176640 1 -- start start 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 0x7ff4c40fead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c41017a0 0x7ff4c40ff010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 0x7ff4c40ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41a44e0 con 0x7ff4c410a2a0 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41a4650 con 0x7ff4c41017a0 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41a4810 con 0x7ff4c4100e80 2024-09-13T17:35:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 0x7ff4c40ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 0x7ff4c40ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43268/0 (socket says 172.21.15.44:43268) 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 -- 172.21.15.44:0/2130776103 learned_addr learned my addr 172.21.15.44:0/2130776103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 0x7ff4c40fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4c9711640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c41017a0 0x7ff4c40ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 -- 172.21.15.44:0/2130776103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 msgr2=0x7ff4c40fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 0x7ff4c40fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 -- 172.21.15.44:0/2130776103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c41017a0 msgr2=0x7ff4c40ff010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c41017a0 0x7ff4c40ff010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 -- 172.21.15.44:0/2130776103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8009560 con 0x7ff4c410a2a0 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 0x7ff4c40fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4c9711640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c41017a0 0x7ff4c40ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4ca713640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 0x7ff4c40ff550 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00eaa0 tx=0x7ff4bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/2130776103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00cdb0 con 0x7ff4c410a2a0 2024-09-13T17:35:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/2130776103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4bc005020 con 0x7ff4c410a2a0 2024-09-13T17:35:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.235+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c41a4af0 con 0x7ff4c410a2a0 2024-09-13T17:35:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.236+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/2130776103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0116d0 con 0x7ff4c410a2a0 2024-09-13T17:35:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.236+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41a5000 con 0x7ff4c410a2a0 2024-09-13T17:35:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.238+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/2130776103 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff4bc005190 con 0x7ff4c410a2a0 2024-09-13T17:35:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.238+0000 7ff4cee97740 1 Processor -- start 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 -- start start 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211adc50 0x5595211b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559521118c30 con 0x5595211b84f0 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595211163c0 con 0x5595211ad330 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4cee97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559521111cb0 con 0x5595211adc50 2024-09-13T17:35:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211ba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43284/0 (socket says 172.21.15.44:43284) 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9711640 1 -- 172.21.15.44:0/3008743757 learned_addr learned my addr 172.21.15.44:0/3008743757 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211adc50 0x5595211b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9711640 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4b6ffd640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff4b00a1280 0x7ff4b00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4ca713640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff4b00a1280 0x7ff4b00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9f12640 1 -- 172.21.15.44:0/3008743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 msgr2=0x5595211ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9f12640 1 -- 172.21.15.44:0/3008743757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 msgr2=0x5595211ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.239+0000 7ff4c9f12640 1 -- 172.21.15.44:0/3008743757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595210b8910 con 0x5595211adc50 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.240+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/2130776103 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff4bc0cd290 con 0x7ff4c410a2a0 2024-09-13T17:35:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.240+0000 7ff4ca713640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff4b00a1280 0x7ff4b00a3740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc099cd0 tx=0x7ff4bc06c0b0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.240+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211adc50 0x5595211b7fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009c30 tx=0x7ff4c00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39a947b44aa0d61f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.240+0000 7ff4affff640 1 -- 172.21.15.44:0/3008743757 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0005020 con 0x5595211adc50 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.240+0000 7ff4affff640 1 -- 172.21.15.44:0/3008743757 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c0005610 con 0x5595211adc50 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.240+0000 7ff4affff640 1 -- 172.21.15.44:0/3008743757 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0012450 con 0x5595211adc50 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 -- 172.21.15.44:0/3008743757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211adc50 msgr2=0x5595211b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211adc50 0x5595211b7fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009c30 tx=0x7ff4c00117b0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 -- 172.21.15.44:0/3008743757 shutdown_connections 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211adc50 0x5595211b7fb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 --2- 172.21.15.44:0/3008743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 -- 172.21.15.44:0/3008743757 >> 172.21.15.44:0/3008743757 conn(0x5595211a98c0 msgr2=0x5595211a9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 -- 172.21.15.44:0/3008743757 shutdown_connections 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 -- 172.21.15.44:0/3008743757 wait complete. 2024-09-13T17:35:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.241+0000 7ff4cee97740 1 -- start start 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 Processor -- start 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211ad330 0x5595211b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595210cfe80 con 0x5595211b84f0 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595210cfb20 con 0x5595211adc50 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559521095260 con 0x5595211ad330 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4cee97740 10 client.0 _ll_register_callbacks cb 0x5595210cadf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9f12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43294/0 (socket says 172.21.15.44:43294) 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9711640 1 -- 172.21.15.44:0/1858512208 learned_addr learned my addr 172.21.15.44:0/1858512208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9711640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211ad330 0x5595211b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9f12640 1 -- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211ad330 msgr2=0x5595211b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.242+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211ad330 0x5595211b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4c9f12640 1 -- 172.21.15.44:0/1858512208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 msgr2=0x5595211b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4c9f12640 1 -- 172.21.15.44:0/1858512208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c0009860 con 0x5595211adc50 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4c9711640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211ad330 0x5595211b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b50b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009c00 tx=0x7ff4c0004df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b4ab1271c5f9ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/1858512208 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0005000 con 0x5595211adc50 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/1858512208 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff4c0005160 con 0x5595211adc50 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55952113a680 con 0x5595211adc50 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/1858512208 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c0005320 con 0x5595211adc50 2024-09-13T17:35:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4cee97740 10 client.27392 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559521110330 con 0x5595211adc50 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.243+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/1858512208 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55952113a680 con 0x5595211adc50 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/1858512208 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4c0012450 con 0x5595211adc50 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4cee97740 10 client.27392 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4cee97740 10 client.27392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4cee97740 20 client.27392 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4cee97740 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5595211c7e20 0x5595211ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4cee97740 4 client.27392 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595211055d0 con 0x5595211c7e20 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4c9711640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5595211c7e20 0x5595211ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4ad7fa640 10 client.27392 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.244+0000 7ff4c9711640 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5595211c7e20 0x5595211ca210 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d8256c77a512b6 server_cookie=75e430f759068f37 in_seq=0 out_seq=0 2024-09-13T17:35:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/1858512208 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5595211055d0 con 0x5595211c7e20 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4ad7fa640 10 client.27392 handle_command_reply: tid=0 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 client.27392 shutdown 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 msgr2=0x5595211b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b50b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009c00 tx=0x7ff4c0004df0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 shutdown_connections 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5595211c7e20 0x5595211ca210 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211b84f0 0x5595211b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b50b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 --2- 172.21.15.44:0/1858512208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211ad330 0x5595211b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4ad7fa640 0 client.27392 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 >> 172.21.15.44:0/1858512208 conn(0x5595211a98c0 msgr2=0x5595211a8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 shutdown_connections 2024-09-13T17:35:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.245+0000 7ff4cee97740 1 -- 172.21.15.44:0/1858512208 wait complete. 2024-09-13T17:35:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.246+0000 7ff4cee97740 20 client.27392 trim_cache size 0 max 16384 2024-09-13T17:35:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.258+0000 7ff4cee97740 1 Processor -- start 2024-09-13T17:35:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.258+0000 7ff4cee97740 1 -- start start 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 0x5595211b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x5595211b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4cee97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595211055d0 con 0x5595211ad330 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4cee97740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952113a680 con 0x5595211adc50 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4cee97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595211baff0 con 0x5595211b84f0 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 0x5595211b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 0x5595211b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43298/0 (socket says 172.21.15.44:43298) 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 -- 172.21.15.44:0/631810092 learned_addr learned my addr 172.21.15.44:0/631810092 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 -- 172.21.15.44:0/631810092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 msgr2=0x5595211b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x5595211b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 -- 172.21.15.44:0/631810092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 msgr2=0x5595211b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 -- 172.21.15.44:0/631810092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595210b2300 con 0x5595211ad330 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4c9711640 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 0x5595211b3960 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009900 tx=0x7ff4b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4fd0ce3cee720c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4b4ff9640 1 -- 172.21.15.44:0/631810092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8004240 con 0x5595211ad330 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4b4ff9640 1 -- 172.21.15.44:0/631810092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4b8044040 con 0x5595211ad330 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.259+0000 7ff4b4ff9640 1 -- 172.21.15.44:0/631810092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8003950 con 0x5595211ad330 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 -- 172.21.15.44:0/631810092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 msgr2=0x5595211b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 0x5595211b3960 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009900 tx=0x7ff4b8009260 comp rx=0 tx=0).stop 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 -- 172.21.15.44:0/631810092 shutdown_connections 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x5595211b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211adc50 0x5595211b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 --2- 172.21.15.44:0/631810092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211ad330 0x5595211b3960 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 -- 172.21.15.44:0/631810092 >> 172.21.15.44:0/631810092 conn(0x5595211a98c0 msgr2=0x5595211b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 -- 172.21.15.44:0/631810092 shutdown_connections 2024-09-13T17:35:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 -- 172.21.15.44:0/631810092 wait complete. 2024-09-13T17:35:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 -- start start 2024-09-13T17:35:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.260+0000 7ff4cee97740 1 Processor -- start 2024-09-13T17:35:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211be460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 0x5595211be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x55952121e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952121e840 con 0x5595211adc50 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952121e980 con 0x5595211ad330 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952121eac0 con 0x5595211b84f0 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4cee97740 10 client.0 _ll_register_callbacks cb 0x7ff4c00095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 0x5595211be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 0x5595211be9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43302/0 (socket says 172.21.15.44:43302) 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 -- 172.21.15.44:0/2687649993 learned_addr learned my addr 172.21.15.44:0/2687649993 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9711640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211be460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9711640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x55952121e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 -- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 msgr2=0x55952121e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x55952121e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 -- 172.21.15.44:0/2687649993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 msgr2=0x5595211be460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211be460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.261+0000 7ff4c9f12640 1 -- 172.21.15.44:0/2687649993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c410d3e0 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4c9f12640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 0x5595211be9a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009c30 tx=0x7ff4c0012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268e7b210e322563 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/2687649993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0004b20 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/2687649993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff4c00129d0 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c0005610 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/2687649993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c0004950 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 10 client.18231 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5595210d78e0 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/2687649993 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c0005610 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/2687649993 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4c0024460 con 0x5595211adc50 2024-09-13T17:35:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 10 client.18231 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 10 client.18231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 20 client.18231 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55952121fe70 0x559521222290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 4 client.18231 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.262+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559520e05300 con 0x55952121fe70 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.263+0000 7ff4c9711640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55952121fe70 0x559521222290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.263+0000 7ff4ad7fa640 10 client.18231 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.263+0000 7ff4c9711640 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55952121fe70 0x559521222290 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d21b9250d961f7 server_cookie=5b1fcc139ca09dda in_seq=0 out_seq=0 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4ad7fa640 1 -- 172.21.15.44:0/2687649993 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559520e05300 con 0x55952121fe70 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4ad7fa640 10 client.18231 handle_command_reply: tid=0 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 client.18231 shutdown 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 msgr2=0x5595211be9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 0x5595211be9a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009c30 tx=0x7ff4c0012f90 comp rx=0 tx=0).stop 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 shutdown_connections 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55952121fe70 0x559521222290 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595211b84f0 0x55952121e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4ad7fa640 0 client.18231 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595211adc50 0x5595211be9a0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 --2- 172.21.15.44:0/2687649993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595211ad330 0x5595211be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 >> 172.21.15.44:0/2687649993 conn(0x5595211a98c0 msgr2=0x5595211b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 shutdown_connections 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.276+0000 7ff4cee97740 1 -- 172.21.15.44:0/2687649993 wait complete. 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cee97740 20 client.18231 trim_cache size 0 max 16384 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:35:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff4b00a1280 msgr2=0x7ff4b00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff4b00a1280 0x7ff4b00a3740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc099cd0 tx=0x7ff4bc06c0b0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 msgr2=0x7ff4c40ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 0x7ff4c40ff550 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00eaa0 tx=0x7ff4bc00ef60 comp rx=0 tx=0).stop 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 shutdown_connections 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff4b00a1280 0x7ff4b00a3740 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c410a2a0 0x7ff4c40ff550 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c41017a0 0x7ff4c40ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 --2- 172.21.15.44:0/2130776103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c4100e80 0x7ff4c40fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 >> 172.21.15.44:0/2130776103 conn(0x7ff4c40fa8f0 msgr2=0x7ff4c40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 shutdown_connections 2024-09-13T17:35:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:12.277+0000 7ff4cc176640 1 -- 172.21.15.44:0/2130776103 wait complete. 2024-09-13T17:35:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:12 smithi044 ceph-mon[31175]: pgmap v262: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-13T17:35:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:12 smithi044 ceph-mon[31175]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T17:35:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:12 smithi070 ceph-mon[34502]: pgmap v262: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-13T17:35:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:12 smithi070 ceph-mon[34502]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T17:35:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:12 smithi100 ceph-mon[35055]: pgmap v262: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-13T17:35:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:12 smithi100 ceph-mon[35055]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T17:35:14.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:14 smithi044 ceph-mon[31175]: pgmap v264: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 KiB/s wr, 7 op/s 2024-09-13T17:35:14.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:14 smithi070 ceph-mon[34502]: pgmap v264: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 KiB/s wr, 7 op/s 2024-09-13T17:35:14.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:14 smithi100 ceph-mon[35055]: pgmap v264: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 KiB/s wr, 7 op/s 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:375/600: open fd = 3 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:376/600: open fd = 3 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.696 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:377/600: open fd = 3 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:378/600: open fd = 3 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:379/600: open fd = 3 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.697 INFO:tasks.workunit.client.0.smithi044.stdout:380/600: open fd = 3 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:381/600: open fd = 3 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:382/600: open fd = 3 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.698 INFO:tasks.workunit.client.0.smithi044.stdout:383/600: open fd = 3 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:384/600: open fd = 3 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:385/600: open fd = 3 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:386/600: open fd = 3 2024-09-13T17:35:15.699 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:387/600: open fd = 3 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:388/600: open fd = 3 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:389/600: open fd = 3 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.700 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:390/600: open fd = 3 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:391/600: open fd = 3 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:392/600: open fd = 3 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:393/600: open fd = 3 2024-09-13T17:35:15.701 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:394/600: open fd = 3 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:395/600: open fd = 3 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:396/600: open fd = 3 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.702 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:397/600: open fd = 3 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:398/600: open fd = 3 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:399/600: open fd = 3 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:400/600: open fd = 3 2024-09-13T17:35:15.703 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:401/600: open fd = 3 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:402/600: open fd = 3 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:403/600: open fd = 3 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.704 INFO:tasks.workunit.client.0.smithi044.stdout:404/600: open fd = 3 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:405/600: open fd = 3 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:406/600: open fd = 3 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:407/600: open fd = 3 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.705 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:408/600: open fd = 3 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:409/600: open fd = 3 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:410/600: open fd = 3 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:411/600: open fd = 3 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:412/600: open fd = 3 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:413/600: open fd = 3 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:414/600: open fd = 3 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.707 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:415/600: open fd = 3 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:416/600: open fd = 3 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:417/600: open fd = 3 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:418/600: open fd = 3 2024-09-13T17:35:15.708 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:419/600: open fd = 3 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:420/600: open fd = 3 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:421/600: open fd = 3 2024-09-13T17:35:15.709 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:422/600: open fd = 3 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:423/600: open fd = 3 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:424/600: open fd = 3 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.713 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:425/600: open fd = 3 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:426/600: open fd = 3 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:427/600: open fd = 3 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.714 INFO:tasks.workunit.client.0.smithi044.stdout:428/600: open fd = 3 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:429/600: open fd = 3 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:430/600: open fd = 3 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.715 INFO:tasks.workunit.client.0.smithi044.stdout:431/600: open fd = 3 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:432/600: open fd = 3 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:433/600: open fd = 3 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:434/600: open fd = 3 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.716 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:435/600: open fd = 3 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:436/600: open fd = 3 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:35:15.717 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:35:16.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:16 smithi044 ceph-mon[31175]: pgmap v265: 129 pgs: 129 active+clean; 3.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-13T17:35:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:16 smithi070 ceph-mon[34502]: pgmap v265: 129 pgs: 129 active+clean; 3.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-13T17:35:16.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:16 smithi100 ceph-mon[35055]: pgmap v265: 129 pgs: 129 active+clean; 3.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-13T17:35:18.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:18 smithi044 ceph-mon[31175]: pgmap v266: 129 pgs: 129 active+clean; 3.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-13T17:35:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:18 smithi070 ceph-mon[34502]: pgmap v266: 129 pgs: 129 active+clean; 3.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-13T17:35:18.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:18 smithi100 ceph-mon[35055]: pgmap v266: 129 pgs: 129 active+clean; 3.6 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-13T17:35:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:20 smithi044 ceph-mon[31175]: pgmap v267: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:35:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:20 smithi070 ceph-mon[34502]: pgmap v267: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:35:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:20 smithi100 ceph-mon[35055]: pgmap v267: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:35:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:22 smithi044 ceph-mon[31175]: pgmap v268: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:35:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:22 smithi070 ceph-mon[34502]: pgmap v268: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:35:22.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:22 smithi100 ceph-mon[35055]: pgmap v268: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-13T17:35:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:24 smithi044 ceph-mon[31175]: pgmap v269: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-13T17:35:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:24 smithi070 ceph-mon[34502]: pgmap v269: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-13T17:35:24.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:24 smithi100 ceph-mon[35055]: pgmap v269: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 -- 172.21.15.44:0/386276946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 msgr2=0x7faa040ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 --2- 172.21.15.44:0/386276946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa040ff240 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009930 tx=0x7fa9f402a420 comp rx=0 tx=0).stop 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 -- 172.21.15.44:0/386276946 shutdown_connections 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 --2- 172.21.15.44:0/386276946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa04103fb0 0x7faa0410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 --2- 172.21.15.44:0/386276946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa040ff810 0x7faa04103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 --2- 172.21.15.44:0/386276946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa040ff240 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 -- 172.21.15.44:0/386276946 >> 172.21.15.44:0/386276946 conn(0x7faa040fa8d0 msgr2=0x7faa040fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 -- 172.21.15.44:0/386276946 shutdown_connections 2024-09-13T17:35:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.378+0000 7faa08bf4640 1 -- 172.21.15.44:0/386276946 wait complete. 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.379+0000 7faa08bf4640 1 Processor -- start 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.379+0000 7faa08bf4640 1 -- start start 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa08bf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa041982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa08bf4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa04103fb0 0x7faa04198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa08bf4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa0419cbf0 0x7faa0419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa08bf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419d5e0 con 0x7faa040fee60 2024-09-13T17:35:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa08bf4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419d750 con 0x7faa04103fb0 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa08bf4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419d8c0 con 0x7faa0419cbf0 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa041982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa041982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47244/0 (socket says 172.21.15.44:47244) 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 -- 172.21.15.44:0/3613764501 learned_addr learned my addr 172.21.15.44:0/3613764501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa01d9b640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa04103fb0 0x7faa04198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa02d9d640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa0419cbf0 0x7faa0419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 -- 172.21.15.44:0/3613764501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa0419cbf0 msgr2=0x7faa0419d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa0419cbf0 0x7faa0419d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 -- 172.21.15.44:0/3613764501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa04103fb0 msgr2=0x7faa04198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa04103fb0 0x7faa04198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 -- 172.21.15.44:0/3613764501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f4009560 con 0x7faa040fee60 2024-09-13T17:35:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa02d9d640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa0419cbf0 0x7faa0419d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7faa0259c640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa041982c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa9f402aa10 tx=0x7fa9f4032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.380+0000 7fa9f37fe640 1 -- 172.21.15.44:0/3613764501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005210 con 0x7faa040fee60 2024-09-13T17:35:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.381+0000 7fa9f37fe640 1 -- 172.21.15.44:0/3613764501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9f4005370 con 0x7faa040fee60 2024-09-13T17:35:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.381+0000 7fa9f37fe640 1 -- 172.21.15.44:0/3613764501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40038e0 con 0x7faa040fee60 2024-09-13T17:35:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.381+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa041a4ea0 con 0x7faa040fee60 2024-09-13T17:35:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.381+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa041a5210 con 0x7faa040fee60 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.383+0000 7fa9f37fe640 1 -- 172.21.15.44:0/3613764501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa9f4003a40 con 0x7faa040fee60 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.383+0000 7faa0b9ee740 1 Processor -- start 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.383+0000 7faa0b9ee740 1 -- start start 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2989710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f2993fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f29968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f28f4c30 con 0x5637f2989c50 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f28f23c0 con 0x5637f29944f0 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f28edcb0 con 0x5637f2989330 2024-09-13T17:35:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f2993fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7fa9f37fe640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9ec0a1200 0x7fa9ec0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f2993fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47246/0 (socket says 172.21.15.44:47246) 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 -- 172.21.15.44:0/2324072723 learned_addr learned my addr 172.21.15.44:0/2324072723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa01d9b640 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f29968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa02d9d640 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2989710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa02d9d640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9ec0a1200 0x7fa9ec0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7fa9f37fe640 1 -- 172.21.15.44:0/3613764501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa9f40ee320 con 0x7faa040fee60 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 -- 172.21.15.44:0/2324072723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 msgr2=0x5637f2989710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2989710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 -- 172.21.15.44:0/2324072723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 msgr2=0x5637f29968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f29968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa0259c640 1 -- 172.21.15.44:0/2324072723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f2894910 con 0x5637f2989c50 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.384+0000 7faa01d9b640 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f29968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.385+0000 7faa0259c640 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f2993fb0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40eb700 tx=0x7fa9f40bb080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f768627bf0ed09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.385+0000 7faa02d9d640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9ec0a1200 0x7fa9ec0a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00afd0 tx=0x7fa9fc005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.385+0000 7fa9f0ff9640 1 -- 172.21.15.44:0/2324072723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40bb880 con 0x5637f2989c50 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.385+0000 7fa9f0ff9640 1 -- 172.21.15.44:0/2324072723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9f40bb180 con 0x5637f2989c50 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.385+0000 7fa9f0ff9640 1 -- 172.21.15.44:0/2324072723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40eb920 con 0x5637f2989c50 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 -- 172.21.15.44:0/2324072723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 msgr2=0x5637f2993fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f2993fb0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40eb700 tx=0x7fa9f40bb080 comp rx=0 tx=0).stop 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 -- 172.21.15.44:0/2324072723 shutdown_connections 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f29968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f2993fb0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/2324072723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2989710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 -- 172.21.15.44:0/2324072723 >> 172.21.15.44:0/2324072723 conn(0x5637f29858c0 msgr2=0x5637f2985ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 -- 172.21.15.44:0/2324072723 shutdown_connections 2024-09-13T17:35:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 -- 172.21.15.44:0/2324072723 wait complete. 2024-09-13T17:35:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 -- start start 2024-09-13T17:35:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 Processor -- start 2024-09-13T17:35:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 0x5637f2993e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.386+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f29910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f29944f0 0x5637f2991640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f28abe80 con 0x5637f2989330 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f28abb20 con 0x5637f2989c50 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f2871260 con 0x5637f29944f0 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 10 client.0 _ll_register_callbacks cb 0x5637f28a6df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 0x5637f2993e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 0x5637f2993e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47262/0 (socket says 172.21.15.44:47262) 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 -- 172.21.15.44:0/715162265 learned_addr learned my addr 172.21.15.44:0/715162265 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa02d9d640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f29944f0 0x5637f2991640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa01d9b640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f29910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 -- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f29944f0 msgr2=0x5637f2991640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f29944f0 0x5637f2991640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 -- 172.21.15.44:0/715162265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 msgr2=0x5637f29910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f29910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 -- 172.21.15.44:0/715162265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa041a5210 con 0x5637f2989330 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa02d9d640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f29944f0 0x5637f2991640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0259c640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 0x5637f2993e60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005ab0 tx=0x7fa9f40b7b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c9edeeef20c96e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7fa9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/715162265 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40bb880 con 0x5637f2989330 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/715162265 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa9f40bd040 con 0x5637f2989330 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637f2916680 con 0x5637f2989330 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/715162265 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f4003cf0 con 0x5637f2989330 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 10 client.18243 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.387+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5637f28ec330 con 0x5637f2989330 2024-09-13T17:35:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/715162265 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637f2916680 con 0x5637f2989330 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/715162265 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9f404d070 con 0x5637f2989330 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa0b9ee740 10 client.18243 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa0b9ee740 10 client.18243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa0b9ee740 20 client.18243 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29a3880 0x5637f29a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa0b9ee740 4 client.18243 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637f28e15d0 con 0x5637f29a3880 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.388+0000 7faa01d9b640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29a3880 0x5637f29a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7fa9e9ffb640 10 client.18243 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa01d9b640 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29a3880 0x5637f29a5c70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ef41f233eb3c58 server_cookie=3a76239b5a61685 in_seq=0 out_seq=0 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/715162265 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5637f28e15d0 con 0x5637f29a3880 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7fa9e9ffb640 10 client.18243 handle_command_reply: tid=0 2024-09-13T17:35:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 client.18243 shutdown 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 msgr2=0x5637f2993e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 0x5637f2993e60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005ab0 tx=0x7fa9f40b7b40 comp rx=0 tx=0).stop 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 shutdown_connections 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29a3880 0x5637f29a5c70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f29944f0 0x5637f2991640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f29910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7fa9e9ffb640 0 client.18243 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/715162265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989330 0x5637f2993e60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.389+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 >> 172.21.15.44:0/715162265 conn(0x5637f29858c0 msgr2=0x5637f2984b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.390+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 shutdown_connections 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.390+0000 7faa0b9ee740 1 -- 172.21.15.44:0/715162265 wait complete. 2024-09-13T17:35:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.390+0000 7faa0b9ee740 20 client.18243 trim_cache size 0 max 16384 2024-09-13T17:35:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.403+0000 7faa0b9ee740 1 Processor -- start 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.403+0000 7faa0b9ee740 1 -- start start 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.403+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2993970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f299afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f299b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f29a0950 con 0x5637f2989c50 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f298f610 con 0x5637f29944f0 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f29a5fa0 con 0x5637f2989330 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa01d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f299afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2993970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa01d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f299afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47278/0 (socket says 172.21.15.44:47278) 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa01d9b640 1 -- 172.21.15.44:0/238373939 learned_addr learned my addr 172.21.15.44:0/238373939 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa02d9d640 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f299b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 -- 172.21.15.44:0/238373939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 msgr2=0x5637f299b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f299b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 -- 172.21.15.44:0/238373939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 msgr2=0x5637f299afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f299afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 -- 172.21.15.44:0/238373939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f288e300 con 0x5637f2989330 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa01d9b640 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f299afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7faa0259c640 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2993970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40c30d0 tx=0x7fa9f4003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30af0c66a5f2be5d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.404+0000 7fa9f17fa640 1 -- 172.21.15.44:0/238373939 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40bb880 con 0x5637f2989330 2024-09-13T17:35:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7fa9f17fa640 1 -- 172.21.15.44:0/238373939 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9f4003cf0 con 0x5637f2989330 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7fa9f17fa640 1 -- 172.21.15.44:0/238373939 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40ee040 con 0x5637f2989330 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 -- 172.21.15.44:0/238373939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 msgr2=0x5637f2993970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2993970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40c30d0 tx=0x7fa9f4003f90 comp rx=0 tx=0).stop 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 -- 172.21.15.44:0/238373939 shutdown_connections 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f29944f0 0x5637f299b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f2989c50 0x5637f299afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/238373939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2993970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 -- 172.21.15.44:0/238373939 >> 172.21.15.44:0/238373939 conn(0x5637f29858c0 msgr2=0x5637f298f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 -- 172.21.15.44:0/238373939 shutdown_connections 2024-09-13T17:35:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.405+0000 7faa0b9ee740 1 -- 172.21.15.44:0/238373939 wait complete. 2024-09-13T17:35:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 -- start start 2024-09-13T17:35:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 Processor -- start 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2999570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f2999ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 0x5637f29f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f29f32e0 con 0x5637f29944f0 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f29f3450 con 0x5637f2989c50 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f29f3610 con 0x5637f2989330 2024-09-13T17:35:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0b9ee740 10 client.0 _ll_register_callbacks cb 0x7fa9f40baf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 0x5637f29f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 0x5637f29f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47286/0 (socket says 172.21.15.44:47286) 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 -- 172.21.15.44:0/960703156 learned_addr learned my addr 172.21.15.44:0/960703156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa01d9b640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f2999ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa0259c640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2999570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 -- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 msgr2=0x5637f2999570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2999570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 -- 172.21.15.44:0/960703156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 msgr2=0x5637f2999ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.406+0000 7faa02d9d640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f2999ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7faa02d9d640 1 -- 172.21.15.44:0/960703156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f80043a0 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7faa02d9d640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 0x5637f29f2d20 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00fbd0 tx=0x7fa9fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea8d627e2682681 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7fa9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/960703156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00d400 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/960703156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa9fc00d560 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9f40bb180 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/960703156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9fc00d720 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7faa0b9ee740 10 client.18249 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5637f28ec330 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/960703156 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9f40bb180 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/960703156 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9fc02a8e0 con 0x5637f29944f0 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.407+0000 7faa0b9ee740 10 client.18249 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0b9ee740 10 client.18249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0b9ee740 20 client.18249 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29faec0 0x5637f29fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0b9ee740 4 client.18249 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:35:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637f25e1300 con 0x5637f29faec0 2024-09-13T17:35:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0259c640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29faec0 0x5637f29fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7faa0259c640 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29faec0 0x5637f29fd2e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b37cb305b1aa5563 server_cookie=891a030e7117a5de in_seq=0 out_seq=0 2024-09-13T17:35:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.408+0000 7fa9e9ffb640 10 client.18249 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7fa9e9ffb640 1 -- 172.21.15.44:0/960703156 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637f25e1300 con 0x5637f29faec0 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7fa9e9ffb640 10 client.18249 handle_command_reply: tid=0 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 client.18249 shutdown 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 msgr2=0x5637f29f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 0x5637f29f2d20 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00fbd0 tx=0x7fa9fc004a40 comp rx=0 tx=0).stop 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 shutdown_connections 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5637f29faec0 0x5637f29fd2e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637f29944f0 0x5637f29f2d20 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7fa9e9ffb640 0 client.18249 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f2989c50 0x5637f2999ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 --2- 172.21.15.44:0/960703156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f2989330 0x5637f2999570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 >> 172.21.15.44:0/960703156 conn(0x5637f29858c0 msgr2=0x5637f2990950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 shutdown_connections 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.421+0000 7faa0b9ee740 1 -- 172.21.15.44:0/960703156 wait complete. 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa0b9ee740 20 client.18249 trim_cache size 0 max 16384 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:35:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9ec0a1200 msgr2=0x7fa9ec0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9ec0a1200 0x7fa9ec0a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00afd0 tx=0x7fa9fc005f90 comp rx=0 tx=0).stop 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 msgr2=0x7faa041982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa041982c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa9f402aa10 tx=0x7fa9f4032c20 comp rx=0 tx=0).stop 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 shutdown_connections 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa9ec0a1200 0x7fa9ec0a36c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa0419cbf0 0x7faa0419d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa04103fb0 0x7faa04198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 --2- 172.21.15.44:0/3613764501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa040fee60 0x7faa041982c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 >> 172.21.15.44:0/3613764501 conn(0x7faa040fa8d0 msgr2=0x7faa04068d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 shutdown_connections 2024-09-13T17:35:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:25.422+0000 7faa08bf4640 1 -- 172.21.15.44:0/3613764501 wait complete. 2024-09-13T17:35:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:26 smithi044 ceph-mon[31175]: pgmap v270: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-09-13T17:35:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:26 smithi070 ceph-mon[34502]: pgmap v270: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-09-13T17:35:26.745 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:35:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:26 smithi100 ceph-mon[35055]: pgmap v270: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-09-13T17:35:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 -- 172.21.15.44:0/4258222575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 msgr2=0x7ff57010cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 --2- 172.21.15.44:0/4258222575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff57010cc80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff568009cd0 tx=0x7ff56802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 -- 172.21.15.44:0/4258222575 shutdown_connections 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 --2- 172.21.15.44:0/4258222575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff57010cc80 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 --2- 172.21.15.44:0/4258222575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5700fee40 0x7ff5701049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 --2- 172.21.15.44:0/4258222575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5700fe490 0x7ff5700fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 -- 172.21.15.44:0/4258222575 >> 172.21.15.44:0/4258222575 conn(0x7ff5700753c0 msgr2=0x7ff5700fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 -- 172.21.15.44:0/4258222575 shutdown_connections 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.863+0000 7ff5787e2640 1 -- 172.21.15.44:0/4258222575 wait complete. 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 Processor -- start 2024-09-13T17:35:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 -- start start 2024-09-13T17:35:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5700fe490 0x7ff5700ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff5700ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5700ffe80 0x7ff5701a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701a49a0 con 0x7ff570105100 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701a4b10 con 0x7ff5700ffe80 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff5787e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701a4cd0 con 0x7ff5700fe490 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff5700ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff5700ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47314/0 (socket says 172.21.15.44:47314) 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 -- 172.21.15.44:0/916887931 learned_addr learned my addr 172.21.15.44:0/916887931 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff576d7f640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5700ffe80 0x7ff5701a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff57657e640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5700fe490 0x7ff5700ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 -- 172.21.15.44:0/916887931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5700fe490 msgr2=0x7ff5700ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5700fe490 0x7ff5700ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 -- 172.21.15.44:0/916887931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5700ffe80 msgr2=0x7ff5701a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5700ffe80 0x7ff5701a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.864+0000 7ff575d7d640 1 -- 172.21.15.44:0/916887931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568009900 con 0x7ff570105100 2024-09-13T17:35:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.865+0000 7ff575d7d640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff5700ff940 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9c0 tx=0x7ff56000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.865+0000 7ff5677fe640 1 -- 172.21.15.44:0/916887931 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560009700 con 0x7ff570105100 2024-09-13T17:35:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.865+0000 7ff5677fe640 1 -- 172.21.15.44:0/916887931 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff56000cd80 con 0x7ff570105100 2024-09-13T17:35:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.865+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5701a4fb0 con 0x7ff570105100 2024-09-13T17:35:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.865+0000 7ff5677fe640 1 -- 172.21.15.44:0/916887931 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560010430 con 0x7ff570105100 2024-09-13T17:35:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.865+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5701a5480 con 0x7ff570105100 2024-09-13T17:35:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.867+0000 7ff5677fe640 1 -- 172.21.15.44:0/916887931 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff560004eb0 con 0x7ff570105100 2024-09-13T17:35:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 Processor -- start 2024-09-13T17:35:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 -- start start 2024-09-13T17:35:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a483b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483b730 0x5590a4845a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a47a5f20 con 0x5590a4845fd0 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a47a36b0 con 0x5590a483b730 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a479efa0 con 0x5590a483ae10 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff5677fe640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff55c0a12b0 0x7ff55c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47328/0 (socket says 172.21.15.44:47328) 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff575d7d640 1 -- 172.21.15.44:0/2224031963 learned_addr learned my addr 172.21.15.44:0/2224031963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.868+0000 7ff57657e640 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483b730 0x5590a4845a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff576d7f640 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a483b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff576d7f640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff55c0a12b0 0x7ff55c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff5677fe640 1 -- 172.21.15.44:0/916887931 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff560014070 con 0x7ff570105100 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff57657e640 1 -- 172.21.15.44:0/2224031963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 msgr2=0x5590a483b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff57657e640 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a483b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff57657e640 1 -- 172.21.15.44:0/2224031963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 msgr2=0x5590a48483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff57657e640 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff57657e640 1 -- 172.21.15.44:0/2224031963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a4759840 con 0x5590a483b730 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff57657e640 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483b730 0x5590a4845a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff56c004630 tx=0x7ff56c00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86ec3e15e4505f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff576d7f640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff55c0a12b0 0x7ff55c0a3770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff568009ca0 tx=0x7ff568005c50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff564ff9640 1 -- 172.21.15.44:0/2224031963 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003bd0 con 0x5590a483b730 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff564ff9640 1 -- 172.21.15.44:0/2224031963 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff56c003d30 con 0x5590a483b730 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.869+0000 7ff564ff9640 1 -- 172.21.15.44:0/2224031963 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004c20 con 0x5590a483b730 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2224031963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483b730 msgr2=0x5590a4845a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483b730 0x5590a4845a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff56c004630 tx=0x7ff56c00bb20 comp rx=0 tx=0).stop 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2224031963 shutdown_connections 2024-09-13T17:35:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483b730 0x5590a4845a90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2224031963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a483b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2224031963 >> 172.21.15.44:0/2224031963 conn(0x5590a4837380 msgr2=0x5590a4837760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2224031963 shutdown_connections 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2224031963 wait complete. 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.870+0000 7ff57b5dc740 1 -- start start 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 Processor -- start 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a48459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4842d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a48432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a47402a0 con 0x5590a483b730 2024-09-13T17:35:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a4477430 con 0x5590a4845fd0 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a4722680 con 0x5590a483ae10 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57b5dc740 10 client.0 _ll_register_callbacks cb 0x5590a4798610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4842d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4842d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47338/0 (socket says 172.21.15.44:47338) 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 -- 172.21.15.44:0/3619433289 learned_addr learned my addr 172.21.15.44:0/3619433289 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff576d7f640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a48432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57657e640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a48459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 -- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 msgr2=0x5590a48459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a48459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 -- 172.21.15.44:0/3619433289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 msgr2=0x5590a48432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a48432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff575d7d640 1 -- 172.21.15.44:0/3619433289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c004290 con 0x5590a483b730 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.871+0000 7ff57657e640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a48459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff575d7d640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4842d00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c6a0 tx=0x7ff56001dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e298f86fc7bd89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff559ffb640 1 -- 172.21.15.44:0/3619433289 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5600caac0 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff559ffb640 1 -- 172.21.15.44:0/3619433289 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff5600046c0 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590a44924e0 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff559ffb640 1 -- 172.21.15.44:0/3619433289 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff56009a700 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 10 client.18261 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5590a479d620 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff559ffb640 1 -- 172.21.15.44:0/3619433289 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5600b6080 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff559ffb640 1 -- 172.21.15.44:0/3619433289 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff56001f060 con 0x5590a483b730 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 10 client.18261 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 10 client.18261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 20 client.18261 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a4855df0 0x5590a4858210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 4 client.18261 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.872+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a44a37e0 con 0x5590a4855df0 2024-09-13T17:35:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.873+0000 7ff57657e640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a4855df0 0x5590a4858210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.873+0000 7ff559ffb640 10 client.18261 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.873+0000 7ff57657e640 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a4855df0 0x5590a4858210 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abba0b40d78c4121 server_cookie=dd73bf523e5ff8fe in_seq=0 out_seq=0 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff559ffb640 1 -- 172.21.15.44:0/3619433289 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5590a44a37e0 con 0x5590a4855df0 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff559ffb640 10 client.18261 handle_command_reply: tid=0 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 client.18261 shutdown 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 msgr2=0x5590a4842d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4842d00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c6a0 tx=0x7ff56001dfa0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 shutdown_connections 2024-09-13T17:35:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a4855df0 0x5590a4858210 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a48432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4842d00 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff559ffb640 0 client.18261 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/3619433289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a48459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 >> 172.21.15.44:0/3619433289 conn(0x5590a4837380 msgr2=0x5590a483d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 shutdown_connections 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 1 -- 172.21.15.44:0/3619433289 wait complete. 2024-09-13T17:35:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.874+0000 7ff57b5dc740 20 client.18261 trim_cache size 0 max 16384 2024-09-13T17:35:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.888+0000 7ff57b5dc740 1 Processor -- start 2024-09-13T17:35:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.888+0000 7ff57b5dc740 1 -- start start 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a4836d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4837760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a4837ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a4855bc0 con 0x5590a483b730 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a485d280 con 0x5590a4845fd0 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a485d440 con 0x5590a483ae10 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4837760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4837760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47354/0 (socket says 172.21.15.44:47354) 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 -- 172.21.15.44:0/973736170 learned_addr learned my addr 172.21.15.44:0/973736170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57657e640 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a4836d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 -- 172.21.15.44:0/973736170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 msgr2=0x5590a4836d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a4836d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 -- 172.21.15.44:0/973736170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 msgr2=0x5590a4837ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff576d7f640 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a4837ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a4837ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 -- 172.21.15.44:0/973736170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a473f790 con 0x5590a483b730 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff57657e640 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a4836d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff575d7d640 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4837760 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff560096a20 tx=0x7ff5600985b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cc80d79d0dda9b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff5657fa640 1 -- 172.21.15.44:0/973736170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5600a21f0 con 0x5590a483b730 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff5657fa640 1 -- 172.21.15.44:0/973736170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5590a44924e0 con 0x5590a483b730 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.889+0000 7ff5657fa640 1 -- 172.21.15.44:0/973736170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5600cab00 con 0x5590a483b730 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 -- 172.21.15.44:0/973736170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 msgr2=0x5590a4837760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4837760 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff560096a20 tx=0x7ff5600985b0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 -- 172.21.15.44:0/973736170 shutdown_connections 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a4845fd0 0x5590a4837ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a483b730 0x5590a4837760 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/973736170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483ae10 0x5590a4836d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 -- 172.21.15.44:0/973736170 >> 172.21.15.44:0/973736170 conn(0x5590a4837380 msgr2=0x5590a485cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 -- 172.21.15.44:0/973736170 shutdown_connections 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 -- 172.21.15.44:0/973736170 wait complete. 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 -- start start 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.890+0000 7ff57b5dc740 1 Processor -- start 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483ae10 0x5590a4848fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483b730 0x5590a4849500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a48ab820 con 0x5590a4845fd0 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a48ab960 con 0x5590a483ae10 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a48abaf0 con 0x5590a483b730 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 10 client.0 _ll_register_callbacks cb 0x7ff5600048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47360/0 (socket says 172.21.15.44:47360) 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 -- 172.21.15.44:0/2324653853 learned_addr learned my addr 172.21.15.44:0/2324653853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff575d7d640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483b730 0x5590a4849500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57657e640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483ae10 0x5590a4848fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 -- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483b730 msgr2=0x5590a4849500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483b730 0x5590a4849500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 -- 172.21.15.44:0/2324653853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483ae10 msgr2=0x5590a4848fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483ae10 0x5590a4848fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 -- 172.21.15.44:0/2324653853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a479d620 con 0x5590a4845fd0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff575d7d640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483b730 0x5590a4849500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff576d7f640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48ab2e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff568035000 tx=0x7ff5680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87bcc46ad542bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff559ffb640 1 -- 172.21.15.44:0/2324653853 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568031be0 con 0x5590a4845fd0 2024-09-13T17:35:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff559ffb640 1 -- 172.21.15.44:0/2324653853 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff5680044c0 con 0x5590a4845fd0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.891+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff56c003d30 con 0x5590a4845fd0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff559ffb640 1 -- 172.21.15.44:0/2324653853 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff568004680 con 0x5590a4845fd0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 10 client.18267 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5590a4764ea0 con 0x5590a4845fd0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff559ffb640 1 -- 172.21.15.44:0/2324653853 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff56c003d30 con 0x5590a4845fd0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff559ffb640 1 -- 172.21.15.44:0/2324653853 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff568057d20 con 0x5590a4845fd0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 10 client.18267 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 20 client.18267 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a48ad2c0 0x5590a48af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 4 client.18267 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590a44f6e30 con 0x5590a48ad2c0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.892+0000 7ff57657e640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a48ad2c0 0x5590a48af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff559ffb640 10 client.18267 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57657e640 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a48ad2c0 0x5590a48af6e0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805addce99c71865 server_cookie=649b8618aa978c9 in_seq=0 out_seq=0 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff559ffb640 1 -- 172.21.15.44:0/2324653853 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590a44f6e30 con 0x5590a48ad2c0 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff559ffb640 10 client.18267 handle_command_reply: tid=0 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 client.18267 shutdown 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 msgr2=0x5590a48ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48ab2e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff568035000 tx=0x7ff5680095c0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 shutdown_connections 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5590a48ad2c0 0x5590a48af6e0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a483b730 0x5590a4849500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590a4845fd0 0x5590a48ab2e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 --2- 172.21.15.44:0/2324653853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5590a483ae10 0x5590a4848fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff559ffb640 0 client.18267 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 >> 172.21.15.44:0/2324653853 conn(0x5590a4837380 msgr2=0x5590a4837760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 shutdown_connections 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 1 -- 172.21.15.44:0/2324653853 wait complete. 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.893+0000 7ff57b5dc740 20 client.18267 trim_cache size 0 max 16384 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:35:26.901 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:35:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff55c0a12b0 msgr2=0x7ff55c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff55c0a12b0 0x7ff55c0a3770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff568009ca0 tx=0x7ff568005c50 comp rx=0 tx=0).stop 2024-09-13T17:35:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 msgr2=0x7ff5700ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff5700ff940 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9c0 tx=0x7ff56000ee80 comp rx=0 tx=0).stop 2024-09-13T17:35:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 shutdown_connections 2024-09-13T17:35:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff55c0a12b0 0x7ff55c0a3770 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5700ffe80 0x7ff5701a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff570105100 0x7ff5700ff940 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 --2- 172.21.15.44:0/916887931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5700fe490 0x7ff5700ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 >> 172.21.15.44:0/916887931 conn(0x7ff5700753c0 msgr2=0x7ff570103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 shutdown_connections 2024-09-13T17:35:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:26.894+0000 7ff5787e2640 1 -- 172.21.15.44:0/916887931 wait complete. 2024-09-13T17:35:26.905 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:35:26.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.007+0000 7f6e8b525640 1 -- 172.21.15.44:0/1423761681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8410a310 msgr2=0x7f6e8410c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.007+0000 7f6e8b525640 1 --2- 172.21.15.44:0/1423761681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8410a310 0x7f6e8410c7b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009ca0 tx=0x7f6e7c02a950 comp rx=0 tx=0).stop 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.007+0000 7f6e8b525640 1 -- 172.21.15.44:0/1423761681 shutdown_connections 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 --2- 172.21.15.44:0/1423761681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8410a310 0x7f6e8410c7b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 --2- 172.21.15.44:0/1423761681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e84101860 0x7f6e84109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 --2- 172.21.15.44:0/1423761681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e84100f40 0x7f6e84101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- 172.21.15.44:0/1423761681 >> 172.21.15.44:0/1423761681 conn(0x7f6e84075c60 msgr2=0x7f6e840fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- 172.21.15.44:0/1423761681 shutdown_connections 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- 172.21.15.44:0/1423761681 wait complete. 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 Processor -- start 2024-09-13T17:35:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- start start 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e84100f40 0x7f6e84196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 0x7f6e84196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8410a310 0x7f6e8419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419b050 con 0x7f6e84101860 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419b1c0 con 0x7f6e84100f40 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.008+0000 7f6e8b525640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419b380 con 0x7f6e8410a310 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 0x7f6e84196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 0x7f6e84196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47366/0 (socket says 172.21.15.44:47366) 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 -- 172.21.15.44:0/884140203 learned_addr learned my addr 172.21.15.44:0/884140203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e892c1640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e84100f40 0x7f6e84196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e89ac2640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8410a310 0x7f6e8419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 -- 172.21.15.44:0/884140203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8410a310 msgr2=0x7f6e8419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8410a310 0x7f6e8419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 -- 172.21.15.44:0/884140203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e84100f40 msgr2=0x7f6e84196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e84100f40 0x7f6e84196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 -- 172.21.15.44:0/884140203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009900 con 0x7f6e84101860 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e88ac0640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 0x7f6e84196660 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400d950 tx=0x7f6e7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009a30 con 0x7f6e84101860 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e740044b0 con 0x7f6e84101860 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740047a0 con 0x7f6e84101860 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8419b660 con 0x7f6e84101860 2024-09-13T17:35:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.009+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841a2ee0 con 0x7f6e84101860 2024-09-13T17:35:27.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.010+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e84067e20 con 0x7f6e84101860 2024-09-13T17:35:27.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.011+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6e74004020 con 0x7f6e84101860 2024-09-13T17:35:27.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.012+0000 7f6e7a7fc640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6e700a1200 0x7f6e700a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.012+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f6e740cd080 con 0x7f6e84101860 2024-09-13T17:35:27.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.012+0000 7f6e892c1640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6e700a1200 0x7f6e700a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.013+0000 7f6e892c1640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6e700a1200 0x7f6e700a36c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000bbe0 tx=0x7f6e8000b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.015+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6e74096160 con 0x7f6e84101860 2024-09-13T17:35:27.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.223+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e84101320 con 0x7f6e84101860 2024-09-13T17:35:27.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.224+0000 7f6e7a7fc640 1 -- 172.21.15.44:0/884140203 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f6e74017020 con 0x7f6e84101860 2024-09-13T17:35:27.225 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:35:27.225 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:35:27.226 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6e700a1200 msgr2=0x7f6e700a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6e700a1200 0x7f6e700a36c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000bbe0 tx=0x7f6e8000b390 comp rx=0 tx=0).stop 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 msgr2=0x7f6e84196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 0x7f6e84196660 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400d950 tx=0x7f6e7400de10 comp rx=0 tx=0).stop 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 shutdown_connections 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f6e700a1200 0x7f6e700a36c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8410a310 0x7f6e8419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e84101860 0x7f6e84196660 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 --2- 172.21.15.44:0/884140203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e84100f40 0x7f6e84196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 >> 172.21.15.44:0/884140203 conn(0x7f6e84075c60 msgr2=0x7f6e840fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.226+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 shutdown_connections 2024-09-13T17:35:27.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.227+0000 7f6e8b525640 1 -- 172.21.15.44:0/884140203 wait complete. 2024-09-13T17:35:27.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:35:27.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 -- 172.21.15.44:0/3746625242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 msgr2=0x7f494010ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3746625242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494010ed90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4938009ca0 tx=0x7f493802a8b0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 -- 172.21.15.44:0/3746625242 shutdown_connections 2024-09-13T17:35:27.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3746625242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494010ed90 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3746625242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4940101160 0x7f49400fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3746625242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940100cf0 0x7f49400fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 -- 172.21.15.44:0/3746625242 >> 172.21.15.44:0/3746625242 conn(0x7f49400753c0 msgr2=0x7f49400fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 -- 172.21.15.44:0/3746625242 shutdown_connections 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.343+0000 7f4947ce5640 1 -- 172.21.15.44:0/3746625242 wait complete. 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 Processor -- start 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 -- start start 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4940100cf0 0x7f49401982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940101160 0x7f4940198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494019d220 con 0x7f494010c9a0 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494019d390 con 0x7f4940100cf0 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4947ce5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494019d550 con 0x7f4940101160 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4946282640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4946282640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47380/0 (socket says 172.21.15.44:47380) 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4946282640 1 -- 172.21.15.44:0/3846947462 learned_addr learned my addr 172.21.15.44:0/3846947462 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4945a81640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4940100cf0 0x7f49401982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.344+0000 7f4945280640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940101160 0x7f4940198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4946282640 1 -- 172.21.15.44:0/3846947462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940101160 msgr2=0x7f4940198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4946282640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940101160 0x7f4940198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4946282640 1 -- 172.21.15.44:0/3846947462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4940100cf0 msgr2=0x7f49401982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4946282640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4940100cf0 0x7f49401982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4946282640 1 -- 172.21.15.44:0/3846947462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009900 con 0x7f494010c9a0 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4945280640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940101160 0x7f4940198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4946282640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494019cc20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f493802aea0 tx=0x7f4938003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4936ffd640 1 -- 172.21.15.44:0/3846947462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938004240 con 0x7f494010c9a0 2024-09-13T17:35:27.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4936ffd640 1 -- 172.21.15.44:0/3846947462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4938030ef0 con 0x7f494010c9a0 2024-09-13T17:35:27.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4936ffd640 1 -- 172.21.15.44:0/3846947462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938031d40 con 0x7f494010c9a0 2024-09-13T17:35:27.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494019d7d0 con 0x7f494010c9a0 2024-09-13T17:35:27.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.345+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49401a50b0 con 0x7f494010c9a0 2024-09-13T17:35:27.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.347+0000 7f4936ffd640 1 -- 172.21.15.44:0/3846947462 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4938042070 con 0x7f494010c9a0 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.347+0000 7f494aa07740 1 Processor -- start 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 -- start start 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 0x5568fca06b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca070b0 0x5568fca11410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca13d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fc970cc0 con 0x5568fca06790 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fc96e450 con 0x5568fca070b0 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f494aa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fc969d40 con 0x5568fca11950 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 0x5568fca06b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 0x5568fca06b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47386/0 (socket says 172.21.15.44:47386) 2024-09-13T17:35:27.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 -- 172.21.15.44:0/2123802848 learned_addr learned my addr 172.21.15.44:0/2123802848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945a81640 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca070b0 0x5568fca11410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4936ffd640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f492c0a1200 0x7f492c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 -- 172.21.15.44:0/2123802848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 msgr2=0x5568fca13d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca13d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 -- 172.21.15.44:0/2123802848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca070b0 msgr2=0x5568fca11410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4945280640 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca070b0 0x5568fca11410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.348+0000 7f4946282640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f492c0a1200 0x7f492c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f4945280640 1 -- 172.21.15.44:0/2123802848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568fc931620 con 0x5568fca06790 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f4936ffd640 1 -- 172.21.15.44:0/3846947462 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f49380ee510 con 0x7f494010c9a0 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f4945280640 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 0x5568fca06b70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f493000bcf0 tx=0x7f4930011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c133ee32ebea08c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f492bfff640 1 -- 172.21.15.44:0/2123802848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930004030 con 0x5568fca06790 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f4946282640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f492c0a1200 0x7f492c0a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49380b5fd0 tx=0x7f49380b7040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f492bfff640 1 -- 172.21.15.44:0/2123802848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49300035f0 con 0x5568fca06790 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.349+0000 7f492bfff640 1 -- 172.21.15.44:0/2123802848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930012650 con 0x5568fca06790 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 -- 172.21.15.44:0/2123802848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 msgr2=0x5568fca06b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 0x5568fca06b70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f493000bcf0 tx=0x7f4930011750 comp rx=0 tx=0).stop 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 -- 172.21.15.44:0/2123802848 shutdown_connections 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca13d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca070b0 0x5568fca11410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 --2- 172.21.15.44:0/2123802848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca06790 0x5568fca06b70 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 -- 172.21.15.44:0/2123802848 >> 172.21.15.44:0/2123802848 conn(0x5568fca02d00 msgr2=0x5568fca030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 -- 172.21.15.44:0/2123802848 shutdown_connections 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 -- 172.21.15.44:0/2123802848 wait complete. 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 -- start start 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.350+0000 7f494aa07740 1 Processor -- start 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca10ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca11400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fc928180 con 0x5568fca070b0 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fc642430 con 0x5568fca06790 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fc8ed5e0 con 0x5568fca11950 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 10 client.0 _ll_register_callbacks cb 0x5568fc959aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca11400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca11400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47390/0 (socket says 172.21.15.44:47390) 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 -- 172.21.15.44:0/3047064542 learned_addr learned my addr 172.21.15.44:0/3047064542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945280640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca10ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945280640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 -- 172.21.15.44:0/3047064542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 msgr2=0x5568fca0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 -- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 msgr2=0x5568fca10ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca10ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 -- 172.21.15.44:0/3047064542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493000b9d0 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945280640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f4945a81640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca11400 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f493c0114c0 tx=0x7f493c011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c002b871652e791b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f49297fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f49297fa640 1 -- 172.21.15.44:0/3047064542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c016070 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f49297fa640 1 -- 172.21.15.44:0/3047064542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f493c004020 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.351+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568fc993b80 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f49297fa640 1 -- 172.21.15.44:0/3047064542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f493c0041e0 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 10 client.18285 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5568fc9683c0 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f49297fa640 1 -- 172.21.15.44:0/3047064542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5568fc993b80 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f49297fa640 1 -- 172.21.15.44:0/3047064542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f493c0037b0 con 0x5568fca070b0 2024-09-13T17:35:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 10 client.18285 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 10 client.18285 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 20 client.18285 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca212c0 0x5568fca236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 4 client.18285 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568fc95c1f0 con 0x5568fca212c0 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.352+0000 7f4945280640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca212c0 0x5568fca236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.353+0000 7f4945280640 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca212c0 0x5568fca236e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94bae6d5a2285e51 server_cookie=364d637ccc487422 in_seq=0 out_seq=0 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.353+0000 7f49297fa640 10 client.18285 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.353+0000 7f49297fa640 1 -- 172.21.15.44:0/3047064542 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5568fc95c1f0 con 0x5568fca212c0 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.353+0000 7f49297fa640 10 client.18285 handle_command_reply: tid=0 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.353+0000 7f494aa07740 1 client.18285 shutdown 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 msgr2=0x5568fca11400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca11400 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f493c0114c0 tx=0x7f493c011980 comp rx=0 tx=0).stop 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 shutdown_connections 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca212c0 0x5568fca236e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca11400 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 --2- 172.21.15.44:0/3047064542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca10ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f49297fa640 0 client.18285 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 >> 172.21.15.44:0/3047064542 conn(0x5568fca02d00 msgr2=0x5568fca08a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 shutdown_connections 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 1 -- 172.21.15.44:0/3047064542 wait complete. 2024-09-13T17:35:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.354+0000 7f494aa07740 20 client.18285 trim_cache size 0 max 16384 2024-09-13T17:35:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 Processor -- start 2024-09-13T17:35:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 -- start start 2024-09-13T17:35:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca06790 0x5568fca026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca11950 0x5568fca18b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fca03a60 con 0x5568fca070b0 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fca1fba0 con 0x5568fca11950 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f494aa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fca14450 con 0x5568fca06790 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca18650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47404/0 (socket says 172.21.15.44:47404) 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 -- 172.21.15.44:0/3592507190 learned_addr learned my addr 172.21.15.44:0/3592507190 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945280640 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca06790 0x5568fca026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 -- 172.21.15.44:0/3592507190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca06790 msgr2=0x5568fca026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca06790 0x5568fca026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945280640 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca11950 0x5568fca18b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 -- 172.21.15.44:0/3592507190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca11950 msgr2=0x5568fca18b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca11950 0x5568fca18b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945a81640 1 -- 172.21.15.44:0/3592507190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568fc90a770 con 0x5568fca070b0 2024-09-13T17:35:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.365+0000 7f4945280640 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca11950 0x5568fca18b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f4945a81640 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca18650 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f493c003b40 tx=0x7f493c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380378d9f7d939c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f4934ff9640 1 -- 172.21.15.44:0/3592507190 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c016040 con 0x5568fca070b0 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f4934ff9640 1 -- 172.21.15.44:0/3592507190 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f493c005070 con 0x5568fca070b0 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f4934ff9640 1 -- 172.21.15.44:0/3592507190 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c004140 con 0x5568fca070b0 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 -- 172.21.15.44:0/3592507190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 msgr2=0x5568fca18650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca18650 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f493c003b40 tx=0x7f493c005c50 comp rx=0 tx=0).stop 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 -- 172.21.15.44:0/3592507190 shutdown_connections 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca11950 0x5568fca18b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca18650 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 --2- 172.21.15.44:0/3592507190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca06790 0x5568fca026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 -- 172.21.15.44:0/3592507190 >> 172.21.15.44:0/3592507190 conn(0x5568fca02d00 msgr2=0x5568fca28040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 -- 172.21.15.44:0/3592507190 shutdown_connections 2024-09-13T17:35:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.366+0000 7f494aa07740 1 -- 172.21.15.44:0/3592507190 wait complete. 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 -- start start 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 Processor -- start 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca18a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca16e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca17460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fca17b00 con 0x5568fca070b0 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fca17c70 con 0x5568fca06790 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568fca777d0 con 0x5568fca11950 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f494aa07740 10 client.0 _ll_register_callbacks cb 0x7f493000b4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca16e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca16e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47412/0 (socket says 172.21.15.44:47412) 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 -- 172.21.15.44:0/1119069803 learned_addr learned my addr 172.21.15.44:0/1119069803 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945280640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca18a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945280640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca17460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 -- 172.21.15.44:0/1119069803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 msgr2=0x5568fca17460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca17460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 -- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 msgr2=0x5568fca18a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca18a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.367+0000 7f4945a81640 1 -- 172.21.15.44:0/1119069803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568fc9683c0 con 0x5568fca070b0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f4945280640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca18a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f4945a81640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca16e30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x5568fca02c00 tx=0x7f493c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d06ab6cada76878 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f49297fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f49297fa640 1 -- 172.21.15.44:0/1119069803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c016040 con 0x5568fca070b0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f49297fa640 1 -- 172.21.15.44:0/1119069803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f493c000e40 con 0x5568fca070b0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49300035f0 con 0x5568fca070b0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f49297fa640 1 -- 172.21.15.44:0/1119069803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f493c0107c0 con 0x5568fca070b0 2024-09-13T17:35:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 10 client.18291 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5568fc92fe80 con 0x5568fca070b0 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f49297fa640 1 -- 172.21.15.44:0/1119069803 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49300035f0 con 0x5568fca070b0 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f49297fa640 1 -- 172.21.15.44:0/1119069803 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f493c030040 con 0x5568fca070b0 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 10 client.18291 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 10 client.18291 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 20 client.18291 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca78bd0 0x5568fca7aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 4 client.18291 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.368+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568fc65d4e0 con 0x5568fca78bd0 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f4945280640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca78bd0 0x5568fca7aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f49297fa640 10 client.18291 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f4945280640 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca78bd0 0x5568fca7aff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44b4689a2dae5796 server_cookie=577cc50e22e8807 in_seq=0 out_seq=0 2024-09-13T17:35:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f49297fa640 1 -- 172.21.15.44:0/1119069803 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568fc65d4e0 con 0x5568fca78bd0 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f49297fa640 10 client.18291 handle_command_reply: tid=0 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f494aa07740 1 client.18291 shutdown 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 msgr2=0x5568fca16e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.369+0000 7f494aa07740 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca16e30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x5568fca02c00 tx=0x7f493c0039e0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 shutdown_connections 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5568fca78bd0 0x5568fca7aff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568fca11950 0x5568fca17460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568fca070b0 0x5568fca16e30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f49297fa640 0 client.18291 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 --2- 172.21.15.44:0/1119069803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5568fca06790 0x5568fca18a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 >> 172.21.15.44:0/1119069803 conn(0x5568fca02d00 msgr2=0x5568fca01690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 shutdown_connections 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 1 -- 172.21.15.44:0/1119069803 wait complete. 2024-09-13T17:35:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f494aa07740 20 client.18291 trim_cache size 0 max 16384 2024-09-13T17:35:27.379 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:35:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f492c0a1200 msgr2=0x7f492c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f492c0a1200 0x7f492c0a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49380b5fd0 tx=0x7f49380b7040 comp rx=0 tx=0).stop 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 msgr2=0x7f494019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494019cc20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f493802aea0 tx=0x7f4938003ba0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 shutdown_connections 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f492c0a1200 0x7f492c0a36c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4940101160 0x7f4940198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494010c9a0 0x7f494019cc20 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 --2- 172.21.15.44:0/3846947462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4940100cf0 0x7f49401982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 >> 172.21.15.44:0/3846947462 conn(0x7f49400753c0 msgr2=0x7f494010def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 shutdown_connections 2024-09-13T17:35:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.370+0000 7f4947ce5640 1 -- 172.21.15.44:0/3846947462 wait complete. 2024-09-13T17:35:27.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 -- 172.21.15.44:0/3214821132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 msgr2=0x7f95ac101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 --2- 172.21.15.44:0/3214821132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac101230 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f959c009900 tx=0x7f959c02a410 comp rx=0 tx=0).stop 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 -- 172.21.15.44:0/3214821132 shutdown_connections 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 --2- 172.21.15.44:0/3214821132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ac10a350 0x7f95ac10c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 --2- 172.21.15.44:0/3214821132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac101800 0x7f95ac109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 --2- 172.21.15.44:0/3214821132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac101230 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 -- 172.21.15.44:0/3214821132 >> 172.21.15.44:0/3214821132 conn(0x7f95ac0fa930 msgr2=0x7f95ac0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 -- 172.21.15.44:0/3214821132 shutdown_connections 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.485+0000 7f95b219c640 1 -- 172.21.15.44:0/3214821132 wait complete. 2024-09-13T17:35:27.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 Processor -- start 2024-09-13T17:35:27.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 -- start start 2024-09-13T17:35:27.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac0feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac101800 0x7f95ac0ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ac10a350 0x7f95ac0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1a4520 con 0x7f95ac100e50 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1a4690 con 0x7f95ac101800 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95b219c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1a4850 con 0x7f95ac10a350 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac0feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac0feb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47438/0 (socket says 172.21.15.44:47438) 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.486+0000 7f95ab7fe640 1 -- 172.21.15.44:0/4150233131 learned_addr learned my addr 172.21.15.44:0/4150233131 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95abfff640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ac10a350 0x7f95ac0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95aaffd640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac101800 0x7f95ac0ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95ab7fe640 1 -- 172.21.15.44:0/4150233131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ac10a350 msgr2=0x7f95ac0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ac10a350 0x7f95ac0ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95ab7fe640 1 -- 172.21.15.44:0/4150233131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac101800 msgr2=0x7f95ac0ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac101800 0x7f95ac0ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95ab7fe640 1 -- 172.21.15.44:0/4150233131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009560 con 0x7f95ac100e50 2024-09-13T17:35:27.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac0feb30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f959c0057c0 tx=0x7f959c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95a8ff9640 1 -- 172.21.15.44:0/4150233131 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c032e10 con 0x7f95ac100e50 2024-09-13T17:35:27.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95a8ff9640 1 -- 172.21.15.44:0/4150233131 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f959c0046c0 con 0x7f95ac100e50 2024-09-13T17:35:27.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac1a4ad0 con 0x7f95ac100e50 2024-09-13T17:35:27.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95a8ff9640 1 -- 172.21.15.44:0/4150233131 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c003850 con 0x7f95ac100e50 2024-09-13T17:35:27.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.487+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1a4f80 con 0x7f95ac100e50 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 Processor -- start 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95a8ff9640 1 -- 172.21.15.44:0/4150233131 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f959c005210 con 0x7f95ac100e50 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 -- start start 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36727ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367285410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367287d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3671e4cc0 con 0x55e36727a790 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3671e2450 con 0x55e36727b0b0 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3671ddd40 con 0x55e367285950 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36727ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36727ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47454/0 (socket says 172.21.15.44:47454) 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 -- 172.21.15.44:0/1881063348 learned_addr learned my addr 172.21.15.44:0/1881063348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367285410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95aaffd640 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367287d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 -- 172.21.15.44:0/1881063348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 msgr2=0x55e367287d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367287d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 -- 172.21.15.44:0/1881063348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 msgr2=0x55e367285410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367285410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95abfff640 1 -- 172.21.15.44:0/1881063348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3671a5620 con 0x55e36727a790 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95aaffd640 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367287d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.490+0000 7f95a8ff9640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95940a1200 0x7f95940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95940a1200 0x7f95940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f95abfff640 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36727ab70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009f70 tx=0x7f95a4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518daa93113f6bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f95a8ff9640 1 -- 172.21.15.44:0/4150233131 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f959c0edfb0 con 0x7f95ac100e50 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f9591ffb640 1 -- 172.21.15.44:0/1881063348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4004d20 con 0x55e36727a790 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f9591ffb640 1 -- 172.21.15.44:0/1881063348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95a4004e80 con 0x55e36727a790 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f9591ffb640 1 -- 172.21.15.44:0/1881063348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4012850 con 0x55e36727a790 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1881063348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 msgr2=0x55e36727ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95940a1200 0x7f95940a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f959c0eb440 tx=0x7f959c072040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.491+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36727ab70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009f70 tx=0x7f95a4010800 comp rx=0 tx=0).stop 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1881063348 shutdown_connections 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367287d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367285410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1881063348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36727ab70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1881063348 >> 172.21.15.44:0/1881063348 conn(0x55e367276d00 msgr2=0x55e3672770e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1881063348 shutdown_connections 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1881063348 wait complete. 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- start start 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 Processor -- start 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727a790 0x55e367284f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 0x55e367282350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367282890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36719c180 con 0x55e36727b0b0 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e366eb6430 con 0x55e36727a790 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3671615e0 con 0x55e367285950 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95b4eb6740 10 client.0 _ll_register_callbacks cb 0x55e3671cdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.492+0000 7f95aaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 0x55e367282350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 0x55e367282350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47460/0 (socket says 172.21.15.44:47460) 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 -- 172.21.15.44:0/3438499232 learned_addr learned my addr 172.21.15.44:0/3438499232 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95abfff640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727a790 0x55e367284f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95abfff640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367282890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 -- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 msgr2=0x55e367282890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367282890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 -- 172.21.15.44:0/3438499232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727a790 msgr2=0x55e367284f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727a790 0x55e367284f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 -- 172.21.15.44:0/3438499232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a4009bd0 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95abfff640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727a790 0x55e367284f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95aaffd640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 0x55e367282350 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f95a000de70 tx=0x7f95a00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617786aa5677975c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f958affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f958affd640 1 -- 172.21.15.44:0/3438499232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0004230 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f958affd640 1 -- 172.21.15.44:0/3438499232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f95a0018070 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3671d01f0 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f958affd640 1 -- 172.21.15.44:0/3438499232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a0004390 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95b4eb6740 10 client.18303 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e3671dc3c0 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.493+0000 7f958affd640 1 -- 172.21.15.44:0/3438499232 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e3671d01f0 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f958affd640 1 -- 172.21.15.44:0/3438499232 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95a0004b20 con 0x55e36727b0b0 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95b4eb6740 10 client.18303 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95b4eb6740 10 client.18303 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95b4eb6740 20 client.18303 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e367295280 0x55e3672976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95b4eb6740 4 client.18303 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e366ed14e0 con 0x55e367295280 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95abfff640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e367295280 0x55e3672976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f958affd640 10 client.18303 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.494+0000 7f95abfff640 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e367295280 0x55e3672976a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67ffb013e704d4b4 server_cookie=ffb97aa055616c57 in_seq=0 out_seq=0 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f958affd640 1 -- 172.21.15.44:0/3438499232 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e366ed14e0 con 0x55e367295280 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f958affd640 10 client.18303 handle_command_reply: tid=0 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 client.18303 shutdown 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 msgr2=0x55e367282350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 0x55e367282350 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f95a000de70 tx=0x7f95a00117b0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 shutdown_connections 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e367295280 0x55e3672976a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367282890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727b0b0 0x55e367282350 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/3438499232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727a790 0x55e367284f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f958affd640 0 client.18303 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.495+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 >> 172.21.15.44:0/3438499232 conn(0x55e367276d00 msgr2=0x55e36727ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.496+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 shutdown_connections 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.496+0000 7f95b4eb6740 1 -- 172.21.15.44:0/3438499232 wait complete. 2024-09-13T17:35:27.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.496+0000 7f95b4eb6740 20 client.18303 trim_cache size 0 max 16384 2024-09-13T17:35:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 Processor -- start 2024-09-13T17:35:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 -- start start 2024-09-13T17:35:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e367282350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367282890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367280be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3672724b0 con 0x55e36727a790 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e367272920 con 0x55e36727b0b0 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e367271390 con 0x55e367285950 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e367282350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e367282350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47462/0 (socket says 172.21.15.44:47462) 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 -- 172.21.15.44:0/1709221633 learned_addr learned my addr 172.21.15.44:0/1709221633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 -- 172.21.15.44:0/1709221633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 msgr2=0x55e367280be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367280be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 -- 172.21.15.44:0/1709221633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 msgr2=0x55e367282890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367282890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 -- 172.21.15.44:0/1709221633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e36717e770 con 0x55e36727a790 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95abfff640 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e367282350 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009f70 tx=0x7f95a40124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469be2afae10a441 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.507+0000 7f95927fc640 1 -- 172.21.15.44:0/1709221633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40294f0 con 0x55e36727a790 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95927fc640 1 -- 172.21.15.44:0/1709221633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95a4005590 con 0x55e36727a790 2024-09-13T17:35:27.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95927fc640 1 -- 172.21.15.44:0/1709221633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4012830 con 0x55e36727a790 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1709221633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 msgr2=0x55e367282350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e367282350 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009f70 tx=0x7f95a40124f0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1709221633 shutdown_connections 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e367285950 0x55e367280be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36727b0b0 0x55e367282890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/1709221633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e367282350 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1709221633 >> 172.21.15.44:0/1709221633 conn(0x55e367276d00 msgr2=0x55e3672910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1709221633 shutdown_connections 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 -- 172.21.15.44:0/1709221633 wait complete. 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.508+0000 7f95b4eb6740 1 -- start start 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 Processor -- start 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36728c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e36727b0b0 0x55e367288e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e367285950 0x55e3672893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e367289ac0 con 0x55e36727a790 2024-09-13T17:35:27.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e367289c30 con 0x55e367285950 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3672eb780 con 0x55e36727b0b0 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95b4eb6740 10 client.0 _ll_register_callbacks cb 0x7f95a4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36728c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36728c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47478/0 (socket says 172.21.15.44:47478) 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 -- 172.21.15.44:0/2945501471 learned_addr learned my addr 172.21.15.44:0/2945501471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95aaffd640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e36727b0b0 0x55e367288e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e367285950 0x55e3672893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 -- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e36727b0b0 msgr2=0x55e367288e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e36727b0b0 0x55e367288e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 -- 172.21.15.44:0/2945501471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e367285950 msgr2=0x55e3672893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e367285950 0x55e3672893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.509+0000 7f95abfff640 1 -- 172.21.15.44:0/2945501471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a4023040 con 0x55e36727a790 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95abfff640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36728c9e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f95a402b040 tx=0x7f95a4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a359be4e2455a14b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f958affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f958affd640 1 -- 172.21.15.44:0/2945501471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4005100 con 0x55e36727a790 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f958affd640 1 -- 172.21.15.44:0/2945501471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f95a4005260 con 0x55e36727a790 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e367207b80 con 0x55e36727a790 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f958affd640 1 -- 172.21.15.44:0/2945501471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a4005420 con 0x55e36727a790 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 10 client.18309 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e3671a3e80 con 0x55e36727a790 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f958affd640 1 -- 172.21.15.44:0/2945501471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e367207b80 con 0x55e36727a790 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f958affd640 1 -- 172.21.15.44:0/2945501471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95a4012c10 con 0x55e36727a790 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 10 client.18309 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 10 client.18309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 20 client.18309 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e3672ec9d0 0x55e3672eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 4 client.18309 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.510+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e366ee27e0 con 0x55e3672ec9d0 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95aaffd640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e3672ec9d0 0x55e3672eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f958affd640 10 client.18309 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95aaffd640 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e3672ec9d0 0x55e3672eedc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ce98e1e9f3eba56 server_cookie=10848f339b9911fb in_seq=0 out_seq=0 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f958affd640 1 -- 172.21.15.44:0/2945501471 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e366ee27e0 con 0x55e3672ec9d0 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f958affd640 10 client.18309 handle_command_reply: tid=0 2024-09-13T17:35:27.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 client.18309 shutdown 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 msgr2=0x55e36728c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36728c9e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f95a402b040 tx=0x7f95a4009820 comp rx=0 tx=0).stop 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 shutdown_connections 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55e3672ec9d0 0x55e3672eedc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e36727b0b0 0x55e367288e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e367285950 0x55e3672893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f958affd640 0 client.18309 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 --2- 172.21.15.44:0/2945501471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e36727a790 0x55e36728c9e0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.511+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 >> 172.21.15.44:0/2945501471 conn(0x55e367276d00 msgr2=0x55e3672910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 shutdown_connections 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b4eb6740 1 -- 172.21.15.44:0/2945501471 wait complete. 2024-09-13T17:35:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b4eb6740 20 client.18309 trim_cache size 0 max 16384 2024-09-13T17:35:27.521 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:35:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95940a1200 msgr2=0x7f95940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95940a1200 0x7f95940a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f959c0eb440 tx=0x7f959c072040 comp rx=0 tx=0).stop 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 msgr2=0x7f95ac0feb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac0feb30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f959c0057c0 tx=0x7f959c032c20 comp rx=0 tx=0).stop 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 shutdown_connections 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95940a1200 0x7f95940a36c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ac10a350 0x7f95ac0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac101800 0x7f95ac0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 --2- 172.21.15.44:0/4150233131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac100e50 0x7f95ac0feb30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 >> 172.21.15.44:0/4150233131 conn(0x7f95ac0fa930 msgr2=0x7f95ac0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 shutdown_connections 2024-09-13T17:35:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.512+0000 7f95b219c640 1 -- 172.21.15.44:0/4150233131 wait complete. 2024-09-13T17:35:27.523 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:27 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/884140203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:35:27.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:35:27.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 -- 172.21.15.44:0/3995922558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f8101860 msgr2=0x7f28f8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 --2- 172.21.15.44:0/3995922558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f8101860 0x7f28f8109d40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009900 tx=0x7f28ec02a410 comp rx=0 tx=0).stop 2024-09-13T17:35:27.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 -- 172.21.15.44:0/3995922558 shutdown_connections 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 --2- 172.21.15.44:0/3995922558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f810a310 0x7f28f810c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 --2- 172.21.15.44:0/3995922558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f8101860 0x7f28f8109d40 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 --2- 172.21.15.44:0/3995922558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 0x7f28f8101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 -- 172.21.15.44:0/3995922558 >> 172.21.15.44:0/3995922558 conn(0x7f28f80fa930 msgr2=0x7f28f80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 -- 172.21.15.44:0/3995922558 shutdown_connections 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.628+0000 7f28fe279640 1 -- 172.21.15.44:0/3995922558 wait complete. 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 Processor -- start 2024-09-13T17:35:27.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 -- start start 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 0x7f28f81961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f8101860 0x7f28f8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 0x7f28f819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f819b110 con 0x7f28f810a310 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f819b280 con 0x7f28f8101860 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fe279640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f819b440 con 0x7f28f8100f40 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fc816640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 0x7f28f819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fc816640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 0x7f28f819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47496/0 (socket says 172.21.15.44:47496) 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28fc816640 1 -- 172.21.15.44:0/1792301381 learned_addr learned my addr 172.21.15.44:0/1792301381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.629+0000 7f28f77fe640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f8101860 0x7f28f8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28f7fff640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 0x7f28f81961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fc816640 1 -- 172.21.15.44:0/1792301381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 msgr2=0x7f28f81961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fc816640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 0x7f28f81961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fc816640 1 -- 172.21.15.44:0/1792301381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f8101860 msgr2=0x7f28f8196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fc816640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f8101860 0x7f28f8196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fc816640 1 -- 172.21.15.44:0/1792301381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec009560 con 0x7f28f810a310 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28f7fff640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 0x7f28f81961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fc816640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 0x7f28f819ab10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f28f000eb10 tx=0x7f28f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28f57fa640 1 -- 172.21.15.44:0/1792301381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0004b20 con 0x7f28f810a310 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28f57fa640 1 -- 172.21.15.44:0/1792301381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28f0004c80 con 0x7f28f810a310 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f819b720 con 0x7f28f810a310 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28f57fa640 1 -- 172.21.15.44:0/1792301381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0005a10 con 0x7f28f810a310 2024-09-13T17:35:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.630+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f81a2fa0 con 0x7f28f810a310 2024-09-13T17:35:27.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.632+0000 7f28f57fa640 1 -- 172.21.15.44:0/1792301381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f28f0005020 con 0x7f28f810a310 2024-09-13T17:35:27.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 Processor -- start 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 -- start start 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f62790 0x561029f62b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f6fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029ecccc0 con 0x561029f630b0 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029eca450 con 0x561029f62790 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f2901077740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029ec5d40 con 0x561029f6d950 2024-09-13T17:35:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f57fa640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28e00a1200 0x7f28e00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f6d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47504/0 (socket says 172.21.15.44:47504) 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 -- 172.21.15.44:0/4216707094 learned_addr learned my addr 172.21.15.44:0/4216707094 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f62790 0x561029f62b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28fc816640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28e00a1200 0x7f28e00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f6fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f57fa640 1 -- 172.21.15.44:0/1792301381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f28f0014070 con 0x7f28f810a310 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 -- 172.21.15.44:0/4216707094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 msgr2=0x561029f6fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f6fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 -- 172.21.15.44:0/4216707094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f62790 msgr2=0x561029f62b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f62790 0x561029f62b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f7fff640 1 -- 172.21.15.44:0/4216707094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561029e8d620 con 0x561029f630b0 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.633+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f28f7fff640 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f6d410 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f28e800bd40 tx=0x7f28e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a9af4b3fc1b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f28fc816640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28e00a1200 0x7f28e00a36c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f28f009a3a0 tx=0x7f28f001b6d0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f28de7fc640 1 -- 172.21.15.44:0/4216707094 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e801a660 con 0x561029f630b0 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f28de7fc640 1 -- 172.21.15.44:0/4216707094 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28e800be30 con 0x561029f630b0 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f28de7fc640 1 -- 172.21.15.44:0/4216707094 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80119d0 con 0x561029f630b0 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 -- 172.21.15.44:0/4216707094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 msgr2=0x561029f6d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f6d410 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f28e800bd40 tx=0x7f28e800f730 comp rx=0 tx=0).stop 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 -- 172.21.15.44:0/4216707094 shutdown_connections 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f6d410 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 --2- 172.21.15.44:0/4216707094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f62790 0x561029f62b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.634+0000 7f2901077740 1 -- 172.21.15.44:0/4216707094 >> 172.21.15.44:0/4216707094 conn(0x561029f5ed00 msgr2=0x561029f5f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 -- 172.21.15.44:0/4216707094 shutdown_connections 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 -- 172.21.15.44:0/4216707094 wait complete. 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 -- start start 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 Processor -- start 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f6cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f630b0 0x561029f6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029f6a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029e84180 con 0x561029f62790 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029b9e430 con 0x561029f6d950 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029e495e0 con 0x561029f630b0 2024-09-13T17:35:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f2901077740 10 client.0 _ll_register_callbacks cb 0x561029eb5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.635+0000 7f28f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f6cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f6cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47518/0 (socket says 172.21.15.44:47518) 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 -- 172.21.15.44:0/2259783563 learned_addr learned my addr 172.21.15.44:0/2259783563 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f7fff640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f630b0 0x561029f6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029f6a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 -- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f630b0 msgr2=0x561029f6a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f630b0 0x561029f6a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 -- 172.21.15.44:0/2259783563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 msgr2=0x561029f6a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029f6a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 -- 172.21.15.44:0/2259783563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e800b9a0 con 0x561029f62790 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28f77fe640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f6cf20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f28ec02aa00 tx=0x7f28ec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7114101bf44a025 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28d77fe640 1 -- 172.21.15.44:0/2259783563 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec032dc0 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28d77fe640 1 -- 172.21.15.44:0/2259783563 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f28ec0035f0 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561029eefb80 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f28d77fe640 1 -- 172.21.15.44:0/2259783563 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28ec0037b0 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f2901077740 10 client.18321 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.636+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561029ec43c0 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f28d77fe640 1 -- 172.21.15.44:0/2259783563 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561029eefb80 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f28d77fe640 1 -- 172.21.15.44:0/2259783563 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28ec03c7e0 con 0x561029f62790 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f2901077740 10 client.18321 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f2901077740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f2901077740 20 client.18321 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f2901077740 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029f7d120 0x561029f7f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f2901077740 4 client.18321 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561029eb81f0 con 0x561029f7d120 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.637+0000 7f28f7fff640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029f7d120 0x561029f7f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f28d77fe640 10 client.18321 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f28f7fff640 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029f7d120 0x561029f7f540 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54bdf12bb19023e server_cookie=af2d7339651c7c00 in_seq=0 out_seq=0 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f28d77fe640 1 -- 172.21.15.44:0/2259783563 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561029eb81f0 con 0x561029f7d120 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f28d77fe640 10 client.18321 handle_command_reply: tid=0 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 client.18321 shutdown 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 msgr2=0x561029f6cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f6cf20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f28ec02aa00 tx=0x7f28ec02a410 comp rx=0 tx=0).stop 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 shutdown_connections 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029f7d120 0x561029f7f540 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f630b0 0x561029f6a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029f6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f28d77fe640 0 client.18321 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 --2- 172.21.15.44:0/2259783563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f6cf20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.638+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 >> 172.21.15.44:0/2259783563 conn(0x561029f5ed00 msgr2=0x561029f64a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.639+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 shutdown_connections 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.639+0000 7f2901077740 1 -- 172.21.15.44:0/2259783563 wait complete. 2024-09-13T17:35:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.639+0000 7f2901077740 20 client.18321 trim_cache size 0 max 16384 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 Processor -- start 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 -- start start 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f630b0 0x561029f74620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029f69ff0 con 0x561029f62790 2024-09-13T17:35:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029f63750 con 0x561029f630b0 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f2901077740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029f6b680 con 0x561029f6d950 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f83ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47528/0 (socket says 172.21.15.44:47528) 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 -- 172.21.15.44:0/1884578682 learned_addr learned my addr 172.21.15.44:0/1884578682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 -- 172.21.15.44:0/1884578682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 msgr2=0x561029f74b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 -- 172.21.15.44:0/1884578682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f630b0 msgr2=0x561029f74620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f7fff640 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f630b0 0x561029f74620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f630b0 0x561029f74620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 -- 172.21.15.44:0/1884578682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561029e66770 con 0x561029f62790 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.650+0000 7f28f77fe640 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f83ea0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ef0c2f2c56ddd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f28deffd640 1 -- 172.21.15.44:0/1884578682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec032dc0 con 0x561029f62790 2024-09-13T17:35:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f28deffd640 1 -- 172.21.15.44:0/1884578682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28ec03c900 con 0x561029f62790 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f28deffd640 1 -- 172.21.15.44:0/1884578682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec031c30 con 0x561029f62790 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 -- 172.21.15.44:0/1884578682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 msgr2=0x561029f83ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f83ea0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec009780 comp rx=0 tx=0).stop 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 -- 172.21.15.44:0/1884578682 shutdown_connections 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f6d950 0x561029f74b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f630b0 0x561029f74620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 --2- 172.21.15.44:0/1884578682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f62790 0x561029f83ea0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 -- 172.21.15.44:0/1884578682 >> 172.21.15.44:0/1884578682 conn(0x561029f5ed00 msgr2=0x561029f790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 -- 172.21.15.44:0/1884578682 shutdown_connections 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 -- 172.21.15.44:0/1884578682 wait complete. 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.651+0000 7f2901077740 1 -- start start 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 Processor -- start 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f62790 0x561029f729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029fcc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029fccb30 con 0x561029f630b0 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029fccca0 con 0x561029f6d950 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561029fcce60 con 0x561029f62790 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f2901077740 10 client.0 _ll_register_callbacks cb 0x7f28e800b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f72f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f72f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47536/0 (socket says 172.21.15.44:47536) 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 -- 172.21.15.44:0/4182621912 learned_addr learned my addr 172.21.15.44:0/4182621912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f62790 0x561029f729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029fcc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 -- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f62790 msgr2=0x561029f729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f62790 0x561029f729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 -- 172.21.15.44:0/4182621912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 msgr2=0x561029fcc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029fcc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.652+0000 7f28f7fff640 1 -- 172.21.15.44:0/4182621912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561029ec43c0 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28f7fff640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f72f20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x561029f5fb60 tx=0x7f28e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d89e3c39969829c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28d77fe640 1 -- 172.21.15.44:0/4182621912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e801e040 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e800be30 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28d77fe640 1 -- 172.21.15.44:0/4182621912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f28e8005570 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28d77fe640 1 -- 172.21.15.44:0/4182621912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e8028170 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 10 client.18327 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561029e8be80 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28d77fe640 1 -- 172.21.15.44:0/4182621912 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28e800be30 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f28d77fe640 1 -- 172.21.15.44:0/4182621912 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28e80119c0 con 0x561029f630b0 2024-09-13T17:35:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 10 client.18327 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 20 client.18327 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029fd46f0 0x561029fd6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 4 client.18327 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.653+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561029bb94e0 con 0x561029fd46f0 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.654+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029fd46f0 0x561029fd6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.654+0000 7f28f77fe640 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029fd46f0 0x561029fd6b10 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30b70951bed8eea server_cookie=748f2b7bf4a4c5a6 in_seq=0 out_seq=0 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.654+0000 7f28d77fe640 10 client.18327 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.654+0000 7f28d77fe640 1 -- 172.21.15.44:0/4182621912 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561029bb94e0 con 0x561029fd46f0 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.654+0000 7f28d77fe640 10 client.18327 handle_command_reply: tid=0 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.654+0000 7f2901077740 1 client.18327 shutdown 2024-09-13T17:35:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 msgr2=0x561029f72f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f72f20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x561029f5fb60 tx=0x7f28e800b8b0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 shutdown_connections 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561029fd46f0 0x561029fd6b10 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561029f6d950 0x561029fcc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561029f630b0 0x561029f72f20 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28d77fe640 0 client.18327 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 --2- 172.21.15.44:0/4182621912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561029f62790 0x561029f729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 >> 172.21.15.44:0/4182621912 conn(0x561029f5ed00 msgr2=0x561029f6a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 shutdown_connections 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 1 -- 172.21.15.44:0/4182621912 wait complete. 2024-09-13T17:35:27.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f2901077740 20 client.18327 trim_cache size 0 max 16384 2024-09-13T17:35:27.663 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:35:27.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28e00a1200 msgr2=0x7f28e00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28e00a1200 0x7f28e00a36c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f28f009a3a0 tx=0x7f28f001b6d0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 msgr2=0x7f28f819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 0x7f28f819ab10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f28f000eb10 tx=0x7f28f000c2d0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 shutdown_connections 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28e00a1200 0x7f28e00a36c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f810a310 0x7f28f819ab10 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f8101860 0x7f28f8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 --2- 172.21.15.44:0/1792301381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f8100f40 0x7f28f81961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 >> 172.21.15.44:0/1792301381 conn(0x7f28f80fa930 msgr2=0x7f28f80fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 shutdown_connections 2024-09-13T17:35:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:27.655+0000 7f28fe279640 1 -- 172.21.15.44:0/1792301381 wait complete. 2024-09-13T17:35:27.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:27 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/884140203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:35:27.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:27 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/884140203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:35:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:28 smithi044 ceph-mon[31175]: pgmap v271: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:35:28.668 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:35:28.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:35:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:28 smithi070 ceph-mon[34502]: pgmap v271: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:35:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:28 smithi100 ceph-mon[35055]: pgmap v271: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-13T17:35:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 -- 172.21.15.44:0/28161216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb14104c10 msgr2=0x7fdb14104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/28161216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb14104c10 0x7fdb14104ff0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009cd0 tx=0x7fdb1002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 -- 172.21.15.44:0/28161216 shutdown_connections 2024-09-13T17:35:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/28161216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb140ff020 0x7fdb140ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/28161216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb140ff9c0 0x7fdb1410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/28161216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb14104c10 0x7fdb14104ff0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 -- 172.21.15.44:0/28161216 >> 172.21.15.44:0/28161216 conn(0x7fdb14074980 msgr2=0x7fdb14074d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 -- 172.21.15.44:0/28161216 shutdown_connections 2024-09-13T17:35:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.787+0000 7fdb1a99d640 1 -- 172.21.15.44:0/28161216 wait complete. 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 Processor -- start 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 -- start start 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 0x7fdb14196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb140ff9c0 0x7fdb141966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb14104c10 0x7fdb1419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1419b060 con 0x7fdb140ff020 2024-09-13T17:35:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1419b1d0 con 0x7fdb14104c10 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a99d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1419b390 con 0x7fdb140ff9c0 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 0x7fdb14196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 0x7fdb14196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47554/0 (socket says 172.21.15.44:47554) 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 -- 172.21.15.44:0/2481314264 learned_addr learned my addr 172.21.15.44:0/2481314264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1919a640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb140ff9c0 0x7fdb141966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb14104c10 0x7fdb1419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 -- 172.21.15.44:0/2481314264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb140ff9c0 msgr2=0x7fdb141966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb140ff9c0 0x7fdb141966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 -- 172.21.15.44:0/2481314264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb14104c10 msgr2=0x7fdb1419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb14104c10 0x7fdb1419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1999b640 1 -- 172.21.15.44:0/2481314264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb10009900 con 0x7fdb140ff020 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.788+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb14104c10 0x7fdb1419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.789+0000 7fdb1999b640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 0x7fdb14196180 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fdb10004af0 tx=0x7fdb10030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.789+0000 7fdb0affd640 1 -- 172.21.15.44:0/2481314264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10031b80 con 0x7fdb140ff020 2024-09-13T17:35:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.789+0000 7fdb0affd640 1 -- 172.21.15.44:0/2481314264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb10003e00 con 0x7fdb140ff020 2024-09-13T17:35:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.789+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1419b610 con 0x7fdb140ff020 2024-09-13T17:35:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.789+0000 7fdb0affd640 1 -- 172.21.15.44:0/2481314264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10045410 con 0x7fdb140ff020 2024-09-13T17:35:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.789+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb141a2ef0 con 0x7fdb140ff020 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.791+0000 7fdb0affd640 1 -- 172.21.15.44:0/2481314264 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdb1000a040 con 0x7fdb140ff020 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.791+0000 7fdb1e9f5740 1 Processor -- start 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 -- start start 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fb4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fc1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171f1fc60 con 0x55b171fb52c0 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171f1d3f0 con 0x55b171fb49a0 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171f18ce0 con 0x55b171fbfb60 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47568/0 (socket says 172.21.15.44:47568) 2024-09-13T17:35:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 -- 172.21.15.44:0/1206856864 learned_addr learned my addr 172.21.15.44:0/1206856864 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fb4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1919a640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fc1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb0affd640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdb000a1280 0x7fdb000a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdb000a1280 0x7fdb000a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb0affd640 1 -- 172.21.15.44:0/2481314264 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fdb100ee6a0 con 0x7fdb140ff020 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 -- 172.21.15.44:0/1206856864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 msgr2=0x55b171fc1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fc1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 -- 172.21.15.44:0/1206856864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 msgr2=0x55b171fb4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fb4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1999b640 1 -- 172.21.15.44:0/1206856864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b171ed4970 con 0x55b171fb52c0 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1919a640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fc1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.792+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.793+0000 7fdb1999b640 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbf620 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdb100eb9f0 tx=0x7fdb10032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3328a43095ef4f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.793+0000 7fdafffff640 1 -- 172.21.15.44:0/1206856864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb100f41a0 con 0x55b171fb52c0 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.793+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdb000a1280 0x7fdb000a3740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00b5c0 tx=0x7fdb0c004730 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.793+0000 7fdafffff640 1 -- 172.21.15.44:0/1206856864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb100bb210 con 0x55b171fb52c0 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.793+0000 7fdafffff640 1 -- 172.21.15.44:0/1206856864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb100c38c0 con 0x55b171fb52c0 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1206856864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 msgr2=0x55b171fbf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbf620 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdb100eb9f0 tx=0x7fdb10032fd0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1206856864 shutdown_connections 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fc1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbf620 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1206856864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1206856864 >> 172.21.15.44:0/1206856864 conn(0x55b171fb0f30 msgr2=0x55b171fb1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1206856864 shutdown_connections 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1206856864 wait complete. 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 -- start start 2024-09-13T17:35:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.794+0000 7fdb1e9f5740 1 Processor -- start 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fbc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fbd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171ed6f80 con 0x55b171fb52c0 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171ed6c20 con 0x55b171fb49a0 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171ea3ab0 con 0x55b171fbfb60 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 10 client.0 _ll_register_callbacks cb 0x55b171f12350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47572/0 (socket says 172.21.15.44:47572) 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 -- 172.21.15.44:0/3133650949 learned_addr learned my addr 172.21.15.44:0/3133650949 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fbd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1999b640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fbc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 -- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 msgr2=0x55b171fbd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fbd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 -- 172.21.15.44:0/3133650949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 msgr2=0x55b171fbc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fbc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 -- 172.21.15.44:0/3133650949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb141a2ef0 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fbd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1919a640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbcee0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bb00 tx=0x7fdb04004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634b8070568e1e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdafd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdafd7fa640 1 -- 172.21.15.44:0/3133650949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016070 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdafd7fa640 1 -- 172.21.15.44:0/3133650949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fdb04003e80 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.795+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b171c0c590 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdafd7fa640 1 -- 172.21.15.44:0/3133650949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb04011440 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 10 client.18339 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b171f17360 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdafd7fa640 1 -- 172.21.15.44:0/3133650949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b171c0c590 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdafd7fa640 1 -- 172.21.15.44:0/3133650949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb0401b650 con 0x55b171fb52c0 2024-09-13T17:35:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 10 client.18339 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 20 client.18339 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fcf8b0 0x55b171fd1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 4 client.18339 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b171c1d860 con 0x55b171fcf8b0 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.796+0000 7fdb1999b640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fcf8b0 0x55b171fd1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.797+0000 7fdb1999b640 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fcf8b0 0x55b171fd1cd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed0007f98657c83 server_cookie=d8e1236a66372b3d in_seq=0 out_seq=0 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.797+0000 7fdafd7fa640 10 client.18339 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.797+0000 7fdafd7fa640 1 -- 172.21.15.44:0/3133650949 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b171c1d860 con 0x55b171fcf8b0 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.797+0000 7fdafd7fa640 10 client.18339 handle_command_reply: tid=0 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.797+0000 7fdb1e9f5740 1 client.18339 shutdown 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 msgr2=0x55b171fbcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbcee0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bb00 tx=0x7fdb04004540 comp rx=0 tx=0).stop 2024-09-13T17:35:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 shutdown_connections 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fcf8b0 0x55b171fd1cd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fbfb60 0x55b171fbd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbcee0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdafd7fa640 0 client.18339 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3133650949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb49a0 0x55b171fbc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 >> 172.21.15.44:0/3133650949 conn(0x55b171fb0f30 msgr2=0x55b171fb6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 shutdown_connections 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3133650949 wait complete. 2024-09-13T17:35:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.798+0000 7fdb1e9f5740 20 client.18339 trim_cache size 0 max 16384 2024-09-13T17:35:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.813+0000 7fdb1e9f5740 1 Processor -- start 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.813+0000 7fdb1e9f5740 1 -- start start 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb0f30 0x55b171fb0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fbb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171fd6ea0 con 0x55b171fb52c0 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171fcbe00 con 0x55b171fb0f30 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171fb1b30 con 0x55b171fb49a0 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb0f30 0x55b171fb0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47576/0 (socket says 172.21.15.44:47576) 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 -- 172.21.15.44:0/3123957337 learned_addr learned my addr 172.21.15.44:0/3123957337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1919a640 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fbb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 -- 172.21.15.44:0/3123957337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 msgr2=0x55b171fbb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fbb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 -- 172.21.15.44:0/3123957337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb0f30 msgr2=0x55b171fb0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb0f30 0x55b171fb0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 -- 172.21.15.44:0/3123957337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b171eb91c0 con 0x55b171fb52c0 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1999b640 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb0f30 0x55b171fb0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.814+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbd250 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00f850 tx=0x7fdb0c009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804bae63d02025c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb08e05640 1 -- 172.21.15.44:0/3123957337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c012a00 con 0x55b171fb52c0 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb08e05640 1 -- 172.21.15.44:0/3123957337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb0c023460 con 0x55b171fb52c0 2024-09-13T17:35:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb08e05640 1 -- 172.21.15.44:0/3123957337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c023ea0 con 0x55b171fb52c0 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3123957337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 msgr2=0x55b171fbd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbd250 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00f850 tx=0x7fdb0c009690 comp rx=0 tx=0).stop 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3123957337 shutdown_connections 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb52c0 0x55b171fbd250 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/3123957337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb0f30 0x55b171fb0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3123957337 >> 172.21.15.44:0/3123957337 conn(0x55b171fb8300 msgr2=0x55b171fbb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3123957337 shutdown_connections 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.815+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/3123957337 wait complete. 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 -- start start 2024-09-13T17:35:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 Processor -- start 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 0x55b171fc5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb52c0 0x55b172025d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171fc61a0 con 0x55b171fb0f30 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1720262c0 con 0x55b171fb52c0 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b172026400 con 0x55b171fb49a0 2024-09-13T17:35:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1e9f5740 10 client.0 _ll_register_callbacks cb 0x7fdb0c009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 0x55b171fc5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1999b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 0x55b171fc5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47578/0 (socket says 172.21.15.44:47578) 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1999b640 1 -- 172.21.15.44:0/1269956923 learned_addr learned my addr 172.21.15.44:0/1269956923 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.816+0000 7fdb1919a640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb52c0 0x55b172025d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1999b640 1 -- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 msgr2=0x55b171fc5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1999b640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fc5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1999b640 1 -- 172.21.15.44:0/1269956923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb52c0 msgr2=0x55b172025d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1999b640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb52c0 0x55b172025d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1999b640 1 -- 172.21.15.44:0/1269956923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b171ed4970 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1a19c640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb52c0 0x55b172025d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1999b640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 0x55b171fc5340 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdb100bb380 tx=0x7fdb100bf270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55902bd5824386d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdafd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdafd7fa640 1 -- 172.21.15.44:0/1269956923 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb100bbb40 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdafd7fa640 1 -- 172.21.15.44:0/1269956923 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fdb100c38c0 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b171fc2320 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdafd7fa640 1 -- 172.21.15.44:0/1269956923 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb100bb440 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1e9f5740 10 client.18345 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fdb0c00f510 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.817+0000 7fdafd7fa640 1 -- 172.21.15.44:0/1269956923 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b171fc2320 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdafd7fa640 1 -- 172.21.15.44:0/1269956923 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb100f6070 con 0x55b171fb0f30 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1e9f5740 10 client.18345 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1e9f5740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1e9f5740 20 client.18345 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fbfb60 0x55b172029720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1e9f5740 4 client.18345 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7fdb100bb210 con 0x55b171fbfb60 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1919a640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fbfb60 0x55b172029720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdafd7fa640 10 client.18345 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.818+0000 7fdb1919a640 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fbfb60 0x55b172029720 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=407a83c254d7a4d0 server_cookie=a33a84d220e48f53 in_seq=0 out_seq=0 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdafd7fa640 1 -- 172.21.15.44:0/1269956923 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fdb100bb210 con 0x55b171fbfb60 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdafd7fa640 10 client.18345 handle_command_reply: tid=0 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 client.18345 shutdown 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 msgr2=0x55b171fc5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 0x55b171fc5340 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdb100bb380 tx=0x7fdb100bf270 comp rx=0 tx=0).stop 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 shutdown_connections 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55b171fbfb60 0x55b172029720 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b171fb49a0 0x55b171fc5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b171fb52c0 0x55b172025d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdafd7fa640 0 client.18345 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 --2- 172.21.15.44:0/1269956923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b171fb0f30 0x55b171fc5340 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 >> 172.21.15.44:0/1269956923 conn(0x55b171fb8300 msgr2=0x55b171fbe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 shutdown_connections 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.819+0000 7fdb1e9f5740 1 -- 172.21.15.44:0/1269956923 wait complete. 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1e9f5740 20 client.18345 trim_cache size 0 max 16384 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:35:28.827 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "a99d0287-d019-422d-b5dd-b723d0fe7560", 2024-09-13T17:35:28.828 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:35:28.828 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:35:28.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdb000a1280 msgr2=0x7fdb000a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdb000a1280 0x7fdb000a3740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00b5c0 tx=0x7fdb0c004730 comp rx=0 tx=0).stop 2024-09-13T17:35:28.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 msgr2=0x7fdb14196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 0x7fdb14196180 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fdb10004af0 tx=0x7fdb10030e50 comp rx=0 tx=0).stop 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 shutdown_connections 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fdb000a1280 0x7fdb000a3740 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb140ff9c0 0x7fdb141966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb14104c10 0x7fdb1419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 --2- 172.21.15.44:0/2481314264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb140ff020 0x7fdb14196180 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 >> 172.21.15.44:0/2481314264 conn(0x7fdb14074980 msgr2=0x7fdb140fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 shutdown_connections 2024-09-13T17:35:28.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.820+0000 7fdb1a99d640 1 -- 172.21.15.44:0/2481314264 wait complete. 2024-09-13T17:35:28.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:35:28.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 -- 172.21.15.44:0/3588737541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21240ff830 msgr2=0x7f2124103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 --2- 172.21.15.44:0/3588737541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21240ff830 0x7f2124103870 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f210c009930 tx=0x7f210c02a420 comp rx=0 tx=0).stop 2024-09-13T17:35:28.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 -- 172.21.15.44:0/3588737541 shutdown_connections 2024-09-13T17:35:28.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 --2- 172.21.15.44:0/3588737541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2124103fd0 0x7f212410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 --2- 172.21.15.44:0/3588737541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21240ff830 0x7f2124103870 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 --2- 172.21.15.44:0/3588737541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21240fee80 0x7f21240ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 -- 172.21.15.44:0/3588737541 >> 172.21.15.44:0/3588737541 conn(0x7f21240fa930 msgr2=0x7f21240fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 -- 172.21.15.44:0/3588737541 shutdown_connections 2024-09-13T17:35:28.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.935+0000 7f212897d640 1 -- 172.21.15.44:0/3588737541 wait complete. 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 Processor -- start 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 -- start start 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21240fee80 0x7f21241961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2124103fd0 0x7f2124196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 0x7f212419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212419b500 con 0x7f212419ab10 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212419b670 con 0x7f21240fee80 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f212897d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212419b7e0 con 0x7f2124103fd0 2024-09-13T17:35:28.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 0x7f212419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 0x7f212419afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47596/0 (socket says 172.21.15.44:47596) 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.936+0000 7f2122d9d640 1 -- 172.21.15.44:0/2973541256 learned_addr learned my addr 172.21.15.44:0/2973541256 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2121d9b640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2124103fd0 0x7f2124196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f212259c640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21240fee80 0x7f21241961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2122d9d640 1 -- 172.21.15.44:0/2973541256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2124103fd0 msgr2=0x7f2124196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2122d9d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2124103fd0 0x7f2124196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2122d9d640 1 -- 172.21.15.44:0/2973541256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21240fee80 msgr2=0x7f21241961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2122d9d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21240fee80 0x7f21241961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2122d9d640 1 -- 172.21.15.44:0/2973541256 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c009560 con 0x7f212419ab10 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f212259c640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21240fee80 0x7f21241961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f2122d9d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 0x7f212419afc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f211c00eaa0 tx=0x7f211c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f211b7fe640 1 -- 172.21.15.44:0/2973541256 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00cdb0 con 0x7f212419ab10 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f211b7fe640 1 -- 172.21.15.44:0/2973541256 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f211c005020 con 0x7f212419ab10 2024-09-13T17:35:28.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21241a2dc0 con 0x7f212419ab10 2024-09-13T17:35:28.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f211b7fe640 1 -- 172.21.15.44:0/2973541256 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0116d0 con 0x7f212419ab10 2024-09-13T17:35:28.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.937+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21241a3240 con 0x7f212419ab10 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.939+0000 7f212b6a0740 1 Processor -- start 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f211b7fe640 1 -- 172.21.15.44:0/2973541256 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f211c005190 con 0x7f212419ab10 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 -- start start 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e4cf0 0x562deb7e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 0x562deb7f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7f43e0 0x562deb7f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb74ff20 con 0x562deb7e5610 2024-09-13T17:35:28.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb74d6b0 con 0x562deb7e4cf0 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212b6a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb748fa0 con 0x562deb7f43e0 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 0x562deb7f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 0x562deb7f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47604/0 (socket says 172.21.15.44:47604) 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f212259c640 1 -- 172.21.15.44:0/1982970306 learned_addr learned my addr 172.21.15.44:0/1982970306 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f2121d9b640 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7f43e0 0x562deb7f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f2121d9b640 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e4cf0 0x562deb7e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.940+0000 7f211b7fe640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21100a1280 0x7f21100a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f212259c640 1 -- 172.21.15.44:0/1982970306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7f43e0 msgr2=0x562deb7f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f212259c640 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7f43e0 0x562deb7f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f212259c640 1 -- 172.21.15.44:0/1982970306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e4cf0 msgr2=0x562deb7e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f212259c640 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e4cf0 0x562deb7e50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f2122d9d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21100a1280 0x7f21100a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f212259c640 1 -- 172.21.15.44:0/1982970306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562deb703840 con 0x562deb7e5610 2024-09-13T17:35:28.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f2121d9b640 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7f43e0 0x562deb7f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f211b7fe640 1 -- 172.21.15.44:0/2973541256 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f211c0cd170 con 0x7f212419ab10 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f212259c640 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 0x562deb7f3ea0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f2114009d70 tx=0x7f21140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83796a54a2844c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f2122d9d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21100a1280 0x7f21100a3740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f211c0ca600 tx=0x7f211c01d000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f2118ff9640 1 -- 172.21.15.44:0/1982970306 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211401b670 con 0x562deb7e5610 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f2118ff9640 1 -- 172.21.15.44:0/1982970306 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2114009e60 con 0x562deb7e5610 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.941+0000 7f2118ff9640 1 -- 172.21.15.44:0/1982970306 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114012a90 con 0x562deb7e5610 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 -- 172.21.15.44:0/1982970306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 msgr2=0x562deb7f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 0x562deb7f3ea0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f2114009d70 tx=0x7f21140117b0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 -- 172.21.15.44:0/1982970306 shutdown_connections 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7f43e0 0x562deb7f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e5610 0x562deb7f3ea0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 --2- 172.21.15.44:0/1982970306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e4cf0 0x562deb7e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 -- 172.21.15.44:0/1982970306 >> 172.21.15.44:0/1982970306 conn(0x562deb7e1380 msgr2=0x562deb7e1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 -- 172.21.15.44:0/1982970306 shutdown_connections 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 -- 172.21.15.44:0/1982970306 wait complete. 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 -- start start 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.942+0000 7f212b6a0740 1 Processor -- start 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e4cf0 0x562deb7f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 0x562deb7f1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb6ea2a0 con 0x562deb7f43e0 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb421430 con 0x562deb7e5610 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb6cc680 con 0x562deb7e4cf0 2024-09-13T17:35:28.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 10 client.0 _ll_register_callbacks cb 0x562deb742610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 0x562deb7f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 0x562deb7f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47612/0 (socket says 172.21.15.44:47612) 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 -- 172.21.15.44:0/3509490358 learned_addr learned my addr 172.21.15.44:0/3509490358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e4cf0 0x562deb7f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212259c640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 -- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e4cf0 msgr2=0x562deb7f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e4cf0 0x562deb7f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 -- 172.21.15.44:0/3509490358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 msgr2=0x562deb7f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 -- 172.21.15.44:0/3509490358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2114009970 con 0x562deb7f43e0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 0x562deb7f1690 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f2124069730 tx=0x7f210c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b220c50a440bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:28.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2109ffb640 1 -- 172.21.15.44:0/3509490358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c031d70 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2109ffb640 1 -- 172.21.15.44:0/3509490358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f210c0035f0 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562deb43c4e0 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.943+0000 7f2109ffb640 1 -- 172.21.15.44:0/3509490358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210c0037b0 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 10 client.18357 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562deb747620 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f2109ffb640 1 -- 172.21.15.44:0/3509490358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562deb43c4e0 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f2109ffb640 1 -- 172.21.15.44:0/3509490358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f210c03d870 con 0x562deb7f43e0 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 10 client.18357 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 20 client.18357 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb804140 0x562deb806560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 4 client.18357 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562deb44d7e0 con 0x562deb804140 2024-09-13T17:35:28.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.944+0000 7f212259c640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb804140 0x562deb806560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212259c640 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb804140 0x562deb806560 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f907a226e455447 server_cookie=de2a7f15f75472bc in_seq=0 out_seq=0 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f2109ffb640 10 client.18357 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f2109ffb640 1 -- 172.21.15.44:0/3509490358 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562deb44d7e0 con 0x562deb804140 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f2109ffb640 10 client.18357 handle_command_reply: tid=0 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 client.18357 shutdown 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 msgr2=0x562deb7f1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 0x562deb7f1690 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f2124069730 tx=0x7f210c02a420 comp rx=0 tx=0).stop 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 shutdown_connections 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb804140 0x562deb806560 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7f43e0 0x562deb7f1690 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f2109ffb640 0 client.18357 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3509490358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e4cf0 0x562deb7f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 >> 172.21.15.44:0/3509490358 conn(0x562deb7e1380 msgr2=0x562deb7e7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 shutdown_connections 2024-09-13T17:35:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.945+0000 7f212b6a0740 1 -- 172.21.15.44:0/3509490358 wait complete. 2024-09-13T17:35:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.946+0000 7f212b6a0740 20 client.18357 trim_cache size 0 max 16384 2024-09-13T17:35:28.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.959+0000 7f212b6a0740 1 Processor -- start 2024-09-13T17:35:28.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 -- start start 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb7f1c90 con 0x562deb7e4cf0 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb7f39a0 con 0x562deb7e5610 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212b6a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb800080 con 0x562deb7e1380 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7ec300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47624/0 (socket says 172.21.15.44:47624) 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 -- 172.21.15.44:0/3071194834 learned_addr learned my addr 172.21.15.44:0/3071194834 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 -- 172.21.15.44:0/3071194834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 msgr2=0x562deb7e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 -- 172.21.15.44:0/3071194834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 msgr2=0x562deb7ec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 -- 172.21.15.44:0/3071194834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562deb6e9790 con 0x562deb7e4cf0 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f212259c640 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7ec300 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2114011cb0 tx=0x7f21140058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f187d72a54794cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.960+0000 7f21197fa640 1 -- 172.21.15.44:0/3071194834 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211402c030 con 0x562deb7e4cf0 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f21197fa640 1 -- 172.21.15.44:0/3071194834 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2114021040 con 0x562deb7e4cf0 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f21197fa640 1 -- 172.21.15.44:0/3071194834 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140280d0 con 0x562deb7e4cf0 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 -- 172.21.15.44:0/3071194834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 msgr2=0x562deb7ec300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7ec300 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2114011cb0 tx=0x7f21140058b0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 -- 172.21.15.44:0/3071194834 shutdown_connections 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7ec300 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3071194834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 -- 172.21.15.44:0/3071194834 >> 172.21.15.44:0/3071194834 conn(0x562deb7f3f10 msgr2=0x562deb7e1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 -- 172.21.15.44:0/3071194834 shutdown_connections 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 -- 172.21.15.44:0/3071194834 wait complete. 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 -- start start 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.961+0000 7f212b6a0740 1 Processor -- start 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb7f8420 con 0x562deb7e4cf0 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb7f8590 con 0x562deb7e5610 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deb85a660 con 0x562deb7e1380 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 10 client.0 _ll_register_callbacks cb 0x7f210c004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47638/0 (socket says 172.21.15.44:47638) 2024-09-13T17:35:28.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 -- 172.21.15.44:0/3343805873 learned_addr learned my addr 172.21.15.44:0/3343805873 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 -- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 msgr2=0x562deb7fb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7fb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 -- 172.21.15.44:0/3343805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 msgr2=0x562deb7f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 -- 172.21.15.44:0/3343805873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562deb703840 con 0x562deb7e4cf0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212259c640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7f77e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2114011cb0 tx=0x7f2114009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c9d5dc5cc85567 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f2109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f2109ffb640 1 -- 172.21.15.44:0/3343805873 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211402c040 con 0x562deb7e4cf0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f2109ffb640 1 -- 172.21.15.44:0/3343805873 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f211402d070 con 0x562deb7e4cf0 2024-09-13T17:35:28.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.962+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2114009e60 con 0x562deb7e4cf0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f2109ffb640 1 -- 172.21.15.44:0/3343805873 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2114004e80 con 0x562deb7e4cf0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 10 client.18363 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562deb70eea0 con 0x562deb7e4cf0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f2109ffb640 1 -- 172.21.15.44:0/3343805873 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2114009e60 con 0x562deb7e4cf0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f2109ffb640 1 -- 172.21.15.44:0/3343805873 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21140287a0 con 0x562deb7e4cf0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 10 client.18363 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 20 client.18363 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb7f43e0 0x562deb85d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 4 client.18363 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562deb4a0e30 con 0x562deb7f43e0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.963+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb7f43e0 0x562deb85d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f2121d9b640 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb7f43e0 0x562deb85d8f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d31d52210adc8c server_cookie=43978df99777028d in_seq=0 out_seq=0 2024-09-13T17:35:28.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f2109ffb640 10 client.18363 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f2109ffb640 1 -- 172.21.15.44:0/3343805873 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562deb4a0e30 con 0x562deb7f43e0 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f2109ffb640 10 client.18363 handle_command_reply: tid=0 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 client.18363 shutdown 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 msgr2=0x562deb7f77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7f77e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2114011cb0 tx=0x7f2114009850 comp rx=0 tx=0).stop 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 shutdown_connections 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562deb7f43e0 0x562deb85d8f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562deb7e5610 0x562deb7f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562deb7e4cf0 0x562deb7f77e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f2109ffb640 0 client.18363 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 --2- 172.21.15.44:0/3343805873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562deb7e1380 0x562deb7fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 >> 172.21.15.44:0/3343805873 conn(0x562deb7f3f10 msgr2=0x562deb7e1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 shutdown_connections 2024-09-13T17:35:28.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 1 -- 172.21.15.44:0/3343805873 wait complete. 2024-09-13T17:35:28.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.964+0000 7f212b6a0740 20 client.18363 trim_cache size 0 max 16384 2024-09-13T17:35:28.972 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:35:28.972 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:35:28.972 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:35:28.972 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21100a1280 msgr2=0x7f21100a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21100a1280 0x7f21100a3740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f211c0ca600 tx=0x7f211c01d000 comp rx=0 tx=0).stop 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 msgr2=0x7f212419afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 0x7f212419afc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f211c00eaa0 tx=0x7f211c00ef60 comp rx=0 tx=0).stop 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 shutdown_connections 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f21100a1280 0x7f21100a3740 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2124103fd0 0x7f2124196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f212419ab10 0x7f212419afc0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 --2- 172.21.15.44:0/2973541256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21240fee80 0x7f21241961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:28.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 >> 172.21.15.44:0/2973541256 conn(0x7f21240fa930 msgr2=0x7f2124068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:28.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 shutdown_connections 2024-09-13T17:35:28.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:28.965+0000 7f212897d640 1 -- 172.21.15.44:0/2973541256 wait complete. 2024-09-13T17:35:28.976 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:35:28.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:35:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 -- 172.21.15.44:0/2375904055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af0104000 msgr2=0x7f4af010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/2375904055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af0104000 0x7f4af010bb50 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009cd0 tx=0x7f4ae802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 -- 172.21.15.44:0/2375904055 shutdown_connections 2024-09-13T17:35:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/2375904055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af0104000 0x7f4af010bb50 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/2375904055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00ff860 0x7f4af01038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/2375904055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af00feeb0 0x7f4af00ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 -- 172.21.15.44:0/2375904055 >> 172.21.15.44:0/2375904055 conn(0x7f4af00fa900 msgr2=0x7f4af00fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 -- 172.21.15.44:0/2375904055 shutdown_connections 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.078+0000 7f4af4bb5640 1 -- 172.21.15.44:0/2375904055 wait complete. 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 Processor -- start 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 -- start start 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00feeb0 0x7f4af0100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af0104000 0x7f4af0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 0x7f4af01a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a49d0 con 0x7f4af01013c0 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a4b40 con 0x7f4af00feeb0 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4af4bb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a4d00 con 0x7f4af0104000 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 0x7f4af01a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 0x7f4af01a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47650/0 (socket says 172.21.15.44:47650) 2024-09-13T17:35:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 -- 172.21.15.44:0/115017951 learned_addr learned my addr 172.21.15.44:0/115017951 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aedd9b640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af0104000 0x7f4af0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aee59c640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00feeb0 0x7f4af0100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 -- 172.21.15.44:0/115017951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af0104000 msgr2=0x7f4af0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af0104000 0x7f4af0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 -- 172.21.15.44:0/115017951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00feeb0 msgr2=0x7f4af0100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00feeb0 0x7f4af0100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 -- 172.21.15.44:0/115017951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009900 con 0x7f4af01013c0 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aee59c640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00feeb0 0x7f4af0100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.079+0000 7f4aeed9d640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 0x7f4af01a4380 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8004af0 tx=0x7f4ae8030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.080+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8031b80 con 0x7f4af01013c0 2024-09-13T17:35:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.080+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ae8003e00 con 0x7f4af01013c0 2024-09-13T17:35:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.080+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af01a4f80 con 0x7f4af01013c0 2024-09-13T17:35:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.080+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8045410 con 0x7f4af01013c0 2024-09-13T17:35:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.080+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01a5450 con 0x7f4af01013c0 2024-09-13T17:35:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.081+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af0100380 con 0x7f4af01013c0 2024-09-13T17:35:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.083+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4ae800a040 con 0x7f4af01013c0 2024-09-13T17:35:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.083+0000 7f4adf7fe640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ad80a1200 0x7f4ad80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.083+0000 7f4aee59c640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ad80a1200 0x7f4ad80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.083+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4ae80ee370 con 0x7f4af01013c0 2024-09-13T17:35:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.084+0000 7f4aee59c640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ad80a1200 0x7f4ad80a36c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009c20 tx=0x7f4ae0005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.086+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4ae80b7450 con 0x7f4af01013c0 2024-09-13T17:35:29.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.294+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4af00ff290 con 0x7f4af01013c0 2024-09-13T17:35:29.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.295+0000 7f4adf7fe640 1 -- 172.21.15.44:0/115017951 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f4ae80baa70 con 0x7f4af01013c0 2024-09-13T17:35:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:35:29.296 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:35:29.296 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ad80a1200 msgr2=0x7f4ad80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ad80a1200 0x7f4ad80a36c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009c20 tx=0x7f4ae0005f90 comp rx=0 tx=0).stop 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 msgr2=0x7f4af01a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 0x7f4af01a4380 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8004af0 tx=0x7f4ae8030e50 comp rx=0 tx=0).stop 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 shutdown_connections 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ad80a1200 0x7f4ad80a36c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af0104000 0x7f4af0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af01013c0 0x7f4af01a4380 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 --2- 172.21.15.44:0/115017951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4af00feeb0 0x7f4af0100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 >> 172.21.15.44:0/115017951 conn(0x7f4af00fa900 msgr2=0x7f4af00689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 shutdown_connections 2024-09-13T17:35:29.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.297+0000 7f4af4bb5640 1 -- 172.21.15.44:0/115017951 wait complete. 2024-09-13T17:35:29.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:35:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 -- 172.21.15.44:0/2071032755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c104c10 msgr2=0x7f460c104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 --2- 172.21.15.44:0/2071032755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c104c10 0x7f460c104ff0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009cd0 tx=0x7f45fc02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 -- 172.21.15.44:0/2071032755 shutdown_connections 2024-09-13T17:35:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 --2- 172.21.15.44:0/2071032755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c0ff9c0 0x7f460c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 --2- 172.21.15.44:0/2071032755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f460c0ff020 0x7f460c0ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 --2- 172.21.15.44:0/2071032755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c104c10 0x7f460c104ff0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 -- 172.21.15.44:0/2071032755 >> 172.21.15.44:0/2071032755 conn(0x7f460c074980 msgr2=0x7f460c074d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 -- 172.21.15.44:0/2071032755 shutdown_connections 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 -- 172.21.15.44:0/2071032755 wait complete. 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 Processor -- start 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.413+0000 7f461279d640 1 -- start start 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f460c0ff020 0x7f460c100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 0x7f460c100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c104c10 0x7f460c1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1a4510 con 0x7f460c0ff9c0 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1a4680 con 0x7f460c0ff020 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1a4840 con 0x7f460c104c10 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 0x7f460c100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 0x7f460c100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40182/0 (socket says 172.21.15.44:40182) 2024-09-13T17:35:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 -- 172.21.15.44:0/27321519 learned_addr learned my addr 172.21.15.44:0/27321519 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461179b640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f460c0ff020 0x7f460c100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4611f9c640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c104c10 0x7f460c1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 -- 172.21.15.44:0/27321519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c104c10 msgr2=0x7f460c1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c104c10 0x7f460c1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 -- 172.21.15.44:0/27321519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f460c0ff020 msgr2=0x7f460c100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f460c0ff020 0x7f460c100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 -- 172.21.15.44:0/27321519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc009900 con 0x7f460c0ff9c0 2024-09-13T17:35:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4611f9c640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c104c10 0x7f460c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f4610f9a640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 0x7f460c100d90 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f460000d950 tx=0x7f460000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f460a7fc640 1 -- 172.21.15.44:0/27321519 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600009c30 con 0x7f460c0ff9c0 2024-09-13T17:35:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f460a7fc640 1 -- 172.21.15.44:0/27321519 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4600009d90 con 0x7f460c0ff9c0 2024-09-13T17:35:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c1a4b20 con 0x7f460c0ff9c0 2024-09-13T17:35:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f460a7fc640 1 -- 172.21.15.44:0/27321519 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460000f620 con 0x7f460c0ff9c0 2024-09-13T17:35:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.414+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c1a5080 con 0x7f460c0ff9c0 2024-09-13T17:35:29.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f460a7fc640 1 -- 172.21.15.44:0/27321519 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f46000109f0 con 0x7f460c0ff9c0 2024-09-13T17:35:29.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 Processor -- start 2024-09-13T17:35:29.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 -- start start 2024-09-13T17:35:29.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a593d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a5a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a4fdcc0 con 0x55880a594290 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a4fb450 con 0x55880a59eb30 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f46167fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a4f6d40 con 0x55880a593970 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f461179b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f461179b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40198/0 (socket says 172.21.15.44:40198) 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f461179b640 1 -- 172.21.15.44:0/2197127018 learned_addr learned my addr 172.21.15.44:0/2197127018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.417+0000 7f4610f9a640 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a5a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f460a7fc640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45f40a1280 0x7f45f40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f4611f9c640 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a593d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f4611f9c640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45f40a1280 0x7f45f40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f461179b640 1 -- 172.21.15.44:0/2197127018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 msgr2=0x55880a593d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f461179b640 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a593d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f461179b640 1 -- 172.21.15.44:0/2197127018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 msgr2=0x55880a5a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f461179b640 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a5a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f461179b640 1 -- 172.21.15.44:0/2197127018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55880a4be620 con 0x55880a594290 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f4611f9c640 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a593d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f460a7fc640 1 -- 172.21.15.44:0/27321519 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f46000ccc00 con 0x7f460c0ff9c0 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f461179b640 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59e5f0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f45fc02ae00 tx=0x7f45fc005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6af3919b903929 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f4611f9c640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45f40a1280 0x7f45f40a3740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4604009f10 tx=0x7f4604005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f45f37fe640 1 -- 172.21.15.44:0/2197127018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0307a0 con 0x55880a594290 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f45f37fe640 1 -- 172.21.15.44:0/2197127018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45fc040070 con 0x55880a594290 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.418+0000 7f45f37fe640 1 -- 172.21.15.44:0/2197127018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc003b90 con 0x55880a594290 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 -- 172.21.15.44:0/2197127018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 msgr2=0x55880a59e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59e5f0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f45fc02ae00 tx=0x7f45fc005fa0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 -- 172.21.15.44:0/2197127018 shutdown_connections 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a5a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59e5f0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 --2- 172.21.15.44:0/2197127018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a593d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 -- 172.21.15.44:0/2197127018 >> 172.21.15.44:0/2197127018 conn(0x55880a58fee0 msgr2=0x55880a5902c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 -- 172.21.15.44:0/2197127018 shutdown_connections 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 -- 172.21.15.44:0/2197127018 wait complete. 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 -- start start 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.419+0000 7f46167fc740 1 Processor -- start 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a593970 0x55880a59e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a59eb30 0x55880a59b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a4b5180 con 0x55880a594290 2024-09-13T17:35:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a1cf430 con 0x55880a593970 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a47a5e0 con 0x55880a59eb30 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f46167fc740 10 client.0 _ll_register_callbacks cb 0x55880a4e6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40210/0 (socket says 172.21.15.44:40210) 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 -- 172.21.15.44:0/2501294634 learned_addr learned my addr 172.21.15.44:0/2501294634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f461179b640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a593970 0x55880a59e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4611f9c640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a59eb30 0x55880a59b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 -- 172.21.15.44:0/2501294634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a59eb30 msgr2=0x55880a59b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a59eb30 0x55880a59b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 -- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a593970 msgr2=0x55880a59e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a593970 0x55880a59e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4610f9a640 1 -- 172.21.15.44:0/2501294634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c10c320 con 0x55880a594290 2024-09-13T17:35:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.420+0000 7f4611f9c640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a59eb30 0x55880a59b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f4610f9a640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59b460 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4600099930 tx=0x7f460000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319f9abf04532e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f45f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f45f0ff9640 1 -- 172.21.15.44:0/2501294634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46000999a0 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f45f0ff9640 1 -- 172.21.15.44:0/2501294634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f46000a1e20 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55880a520b80 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f45f0ff9640 1 -- 172.21.15.44:0/2501294634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f460009d7d0 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 10 client.18381 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55880a4f53c0 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f45f0ff9640 1 -- 172.21.15.44:0/2501294634 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55880a520b80 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f45f0ff9640 1 -- 172.21.15.44:0/2501294634 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46000a1a20 con 0x55880a594290 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 10 client.18381 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 10 client.18381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:35:29.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 20 client.18381 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a5ae450 0x55880a5b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 4 client.18381 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55880a4e91f0 con 0x55880a5ae450 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.421+0000 7f461179b640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a5ae450 0x55880a5b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.422+0000 7f461179b640 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a5ae450 0x55880a5b0840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f70266a1a140c99 server_cookie=d1db4e876297716b in_seq=0 out_seq=0 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.422+0000 7f45f0ff9640 10 client.18381 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.422+0000 7f45f0ff9640 1 -- 172.21.15.44:0/2501294634 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55880a4e91f0 con 0x55880a5ae450 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.422+0000 7f45f0ff9640 10 client.18381 handle_command_reply: tid=0 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 client.18381 shutdown 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 msgr2=0x55880a59b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59b460 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4600099930 tx=0x7f460000bf60 comp rx=0 tx=0).stop 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 shutdown_connections 2024-09-13T17:35:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a5ae450 0x55880a5b0840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a59eb30 0x55880a59b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a594290 0x55880a59b460 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 --2- 172.21.15.44:0/2501294634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a593970 0x55880a59e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f45f0ff9640 0 client.18381 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 >> 172.21.15.44:0/2501294634 conn(0x55880a58fee0 msgr2=0x55880a58f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 shutdown_connections 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 1 -- 172.21.15.44:0/2501294634 wait complete. 2024-09-13T17:35:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.423+0000 7f46167fc740 20 client.18381 trim_cache size 0 max 16384 2024-09-13T17:35:29.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 Processor -- start 2024-09-13T17:35:29.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 -- start start 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 0x55880a599d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a594290 0x55880a59a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a59ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a59a790 con 0x55880a593970 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a58bcd0 con 0x55880a59eb30 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f46167fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a594b10 con 0x55880a594290 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 0x55880a599d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 0x55880a599d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40214/0 (socket says 172.21.15.44:40214) 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 -- 172.21.15.44:0/3208765516 learned_addr learned my addr 172.21.15.44:0/3208765516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f4611f9c640 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a59ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 -- 172.21.15.44:0/3208765516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a594290 msgr2=0x55880a59a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a594290 0x55880a59a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 -- 172.21.15.44:0/3208765516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 msgr2=0x55880a59ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.434+0000 7f461179b640 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a59ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f461179b640 1 -- 172.21.15.44:0/3208765516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55880a497770 con 0x55880a593970 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f461179b640 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 0x55880a599d10 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f45fc006010 tx=0x7f45fc049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53a3450ab9b5980 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f45f3fff640 1 -- 172.21.15.44:0/3208765516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc04c040 con 0x55880a593970 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f45f3fff640 1 -- 172.21.15.44:0/3208765516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45fc031770 con 0x55880a593970 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f45f3fff640 1 -- 172.21.15.44:0/3208765516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc051900 con 0x55880a593970 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 -- 172.21.15.44:0/3208765516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 msgr2=0x55880a599d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 0x55880a599d10 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f45fc006010 tx=0x7f45fc049040 comp rx=0 tx=0).stop 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 -- 172.21.15.44:0/3208765516 shutdown_connections 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a594290 0x55880a59a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a59eb30 0x55880a59ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 --2- 172.21.15.44:0/3208765516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a593970 0x55880a599d10 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 -- 172.21.15.44:0/3208765516 >> 172.21.15.44:0/3208765516 conn(0x55880a58fee0 msgr2=0x55880a5902c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.435+0000 7f46167fc740 1 -- 172.21.15.44:0/3208765516 shutdown_connections 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 -- 172.21.15.44:0/3208765516 wait complete. 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 -- start start 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 Processor -- start 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a5a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a594290 0x55880a5a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 0x55880a5a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a5a6140 con 0x55880a59eb30 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a604920 con 0x55880a594290 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880a604a60 con 0x55880a593970 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f46167fc740 10 client.0 _ll_register_callbacks cb 0x7f45fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:29.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f4611f9c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 0x55880a5a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f4611f9c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 0x55880a5a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40224/0 (socket says 172.21.15.44:40224) 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.436+0000 7f4611f9c640 1 -- 172.21.15.44:0/3313719225 learned_addr learned my addr 172.21.15.44:0/3313719225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4610f9a640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a594290 0x55880a5a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f461179b640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a5a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4611f9c640 1 -- 172.21.15.44:0/3313719225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 msgr2=0x55880a5a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4611f9c640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a5a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4611f9c640 1 -- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a594290 msgr2=0x55880a5a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4611f9c640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a594290 0x55880a5a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4611f9c640 1 -- 172.21.15.44:0/3313719225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55880a4be620 con 0x55880a59eb30 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f4611f9c640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 0x55880a5a2c70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f460401fb60 tx=0x7f46040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6d1b7aac9a51c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f45f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f45f0ff9640 1 -- 172.21.15.44:0/3313719225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604024070 con 0x55880a59eb30 2024-09-13T17:35:29.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f45f0ff9640 1 -- 172.21.15.44:0/3313719225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f460401eb70 con 0x55880a59eb30 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45fc040070 con 0x55880a59eb30 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f45f0ff9640 1 -- 172.21.15.44:0/3313719225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f460401ed30 con 0x55880a59eb30 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f46167fc740 10 client.18387 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55880a4bce80 con 0x55880a59eb30 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.437+0000 7f45f0ff9640 1 -- 172.21.15.44:0/3313719225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45fc040070 con 0x55880a59eb30 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f45f0ff9640 1 -- 172.21.15.44:0/3313719225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f460401a040 con 0x55880a59eb30 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 10 client.18387 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 10 client.18387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 20 client.18387 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a606290 0x55880a6086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 4 client.18387 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55880a1ea4e0 con 0x55880a606290 2024-09-13T17:35:29.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f461179b640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a606290 0x55880a6086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f45f0ff9640 10 client.18387 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f461179b640 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a606290 0x55880a6086b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb322d3769a7d976 server_cookie=30ad18eea5cbf5ab in_seq=0 out_seq=0 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f45f0ff9640 1 -- 172.21.15.44:0/3313719225 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55880a1ea4e0 con 0x55880a606290 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f45f0ff9640 10 client.18387 handle_command_reply: tid=0 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.438+0000 7f46167fc740 1 client.18387 shutdown 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 msgr2=0x55880a5a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 0x55880a5a2c70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f460401fb60 tx=0x7f46040046c0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 shutdown_connections 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55880a606290 0x55880a6086b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55880a59eb30 0x55880a5a2c70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f45f0ff9640 0 client.18387 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55880a594290 0x55880a5a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 --2- 172.21.15.44:0/3313719225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55880a593970 0x55880a5a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 >> 172.21.15.44:0/3313719225 conn(0x55880a58fee0 msgr2=0x55880a599ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 shutdown_connections 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 1 -- 172.21.15.44:0/3313719225 wait complete. 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f46167fc740 20 client.18387 trim_cache size 0 max 16384 2024-09-13T17:35:29.447 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:35:29.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45f40a1280 msgr2=0x7f45f40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45f40a1280 0x7f45f40a3740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4604009f10 tx=0x7f4604005f90 comp rx=0 tx=0).stop 2024-09-13T17:35:29.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 msgr2=0x7f460c100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 0x7f460c100d90 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f460000d950 tx=0x7f460000de10 comp rx=0 tx=0).stop 2024-09-13T17:35:29.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 shutdown_connections 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f45f40a1280 0x7f45f40a3740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c104c10 0x7f460c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f460c0ff9c0 0x7f460c100d90 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 --2- 172.21.15.44:0/27321519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f460c0ff020 0x7f460c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 >> 172.21.15.44:0/27321519 conn(0x7f460c074980 msgr2=0x7f460c067810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 shutdown_connections 2024-09-13T17:35:29.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.439+0000 7f461279d640 1 -- 172.21.15.44:0/27321519 wait complete. 2024-09-13T17:35:29.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:35:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 -- 172.21.15.44:0/1717514205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43040728e0 msgr2=0x7f4304072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 --2- 172.21.15.44:0/1717514205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43040728e0 0x7f4304072cc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4300009900 tx=0x7f430002a480 comp rx=0 tx=0).stop 2024-09-13T17:35:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 -- 172.21.15.44:0/1717514205 shutdown_connections 2024-09-13T17:35:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 --2- 172.21.15.44:0/1717514205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4304073200 0x7f430410c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 --2- 172.21.15.44:0/1717514205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f430410c560 0x7f430410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 --2- 172.21.15.44:0/1717514205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43040728e0 0x7f4304072cc0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 -- 172.21.15.44:0/1717514205 >> 172.21.15.44:0/1717514205 conn(0x7f43040fba20 msgr2=0x7f43040fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 -- 172.21.15.44:0/1717514205 shutdown_connections 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.554+0000 7f430bc20640 1 -- 172.21.15.44:0/1717514205 wait complete. 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 Processor -- start 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 -- start start 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43040728e0 0x7f4304198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4304073200 0x7f4304198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 0x7f430419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430419d330 con 0x7f430410c560 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430419d4a0 con 0x7f4304073200 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430bc20640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430419d660 con 0x7f43040728e0 2024-09-13T17:35:29.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 0x7f430419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 0x7f430419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40238/0 (socket says 172.21.15.44:40238) 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 -- 172.21.15.44:0/1621923015 learned_addr learned my addr 172.21.15.44:0/1621923015 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f43091bb640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4304073200 0x7f4304198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f43099bc640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43040728e0 0x7f4304198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 -- 172.21.15.44:0/1621923015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43040728e0 msgr2=0x7f4304198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43040728e0 0x7f4304198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 -- 172.21.15.44:0/1621923015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4304073200 msgr2=0x7f4304198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4304073200 0x7f4304198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.555+0000 7f430a1bd640 1 -- 172.21.15.44:0/1621923015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4300009560 con 0x7f430410c560 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.556+0000 7f430a1bd640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 0x7f430419cd30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00eb10 tx=0x7f42fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.556+0000 7f42faffd640 1 -- 172.21.15.44:0/1621923015 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc004b20 con 0x7f430410c560 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.556+0000 7f42faffd640 1 -- 172.21.15.44:0/1621923015 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42fc004c80 con 0x7f430410c560 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.556+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430419d940 con 0x7f430410c560 2024-09-13T17:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.556+0000 7f42faffd640 1 -- 172.21.15.44:0/1621923015 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc005a10 con 0x7f430410c560 2024-09-13T17:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.556+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43041a51c0 con 0x7f430410c560 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.558+0000 7f42faffd640 1 -- 172.21.15.44:0/1621923015 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f42fc005020 con 0x7f430410c560 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.558+0000 7f430ea21740 1 Processor -- start 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 -- start start 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e5bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e66410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e66950 0x560ff0e68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0dc5cc0 con 0x560ff0e5c0b0 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0dc3450 con 0x560ff0e5b790 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430ea21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0dbed40 con 0x560ff0e66950 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f42faffd640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f42f00a1280 0x7f42f00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e66410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e66410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40250/0 (socket says 172.21.15.44:40250) 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 -- 172.21.15.44:0/3720615045 learned_addr learned my addr 172.21.15.44:0/3720615045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43091bb640 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e5bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f430a1bd640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f42f00a1280 0x7f42f00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43091bb640 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e66950 0x560ff0e68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 -- 172.21.15.44:0/3720615045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e66950 msgr2=0x560ff0e68d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e66950 0x560ff0e68d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 -- 172.21.15.44:0/3720615045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 msgr2=0x560ff0e5bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e5bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f42faffd640 1 -- 172.21.15.44:0/1621923015 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f42fc014070 con 0x7f430410c560 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43099bc640 1 -- 172.21.15.44:0/3720615045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff0d86620 con 0x560ff0e5c0b0 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.559+0000 7f43091bb640 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f43099bc640 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e66410 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4300005940 tx=0x7f4300005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00aa7fc370e3aca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430a1bd640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f42f00a1280 0x7f42f00a3740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42fc09a060 tx=0x7f42fc096040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f42effff640 1 -- 172.21.15.44:0/3720615045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300004030 con 0x560ff0e5c0b0 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f42effff640 1 -- 172.21.15.44:0/3720615045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4300031960 con 0x560ff0e5c0b0 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f42effff640 1 -- 172.21.15.44:0/3720615045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300003a80 con 0x560ff0e5c0b0 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 -- 172.21.15.44:0/3720615045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 msgr2=0x560ff0e66410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e66410 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4300005940 tx=0x7f4300005e80 comp rx=0 tx=0).stop 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 -- 172.21.15.44:0/3720615045 shutdown_connections 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e66950 0x560ff0e68d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e66410 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 --2- 172.21.15.44:0/3720615045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.560+0000 7f430ea21740 1 -- 172.21.15.44:0/3720615045 >> 172.21.15.44:0/3720615045 conn(0x560ff0e57d00 msgr2=0x560ff0e580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 -- 172.21.15.44:0/3720615045 shutdown_connections 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 -- 172.21.15.44:0/3720615045 wait complete. 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 -- start start 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 Processor -- start 2024-09-13T17:35:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e63730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0d7d180 con 0x560ff0e5c0b0 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0a97430 con 0x560ff0e66950 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.561+0000 7f430ea21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0d425e0 con 0x560ff0e5b790 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f430ea21740 10 client.0 _ll_register_callbacks cb 0x560ff0daeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40252/0 (socket says 172.21.15.44:40252) 2024-09-13T17:35:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 -- 172.21.15.44:0/3284049050 learned_addr learned my addr 172.21.15.44:0/3284049050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43091bb640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43091bb640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e63730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 -- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 msgr2=0x560ff0e65f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e65f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 -- 172.21.15.44:0/3284049050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 msgr2=0x560ff0e63730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e63730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 -- 172.21.15.44:0/3284049050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430410f630 con 0x560ff0e5c0b0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43091bb640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e63730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f43099bc640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e631f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4300000c00 tx=0x7f4300032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cea4c6aa0d5ebc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f42ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f42ed7fa640 1 -- 172.21.15.44:0/3284049050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300051670 con 0x560ff0e5c0b0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f42ed7fa640 1 -- 172.21.15.44:0/3284049050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f43000517d0 con 0x560ff0e5c0b0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ff0de8b80 con 0x560ff0e5c0b0 2024-09-13T17:35:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f42ed7fa640 1 -- 172.21.15.44:0/3284049050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4300051990 con 0x560ff0e5c0b0 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f430ea21740 10 client.18399 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.562+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560ff0dbd3c0 con 0x560ff0e5c0b0 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f42ed7fa640 1 -- 172.21.15.44:0/3284049050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ff0de8b80 con 0x560ff0e5c0b0 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f42ed7fa640 1 -- 172.21.15.44:0/3284049050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4300042020 con 0x560ff0e5c0b0 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f430ea21740 10 client.18399 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f430ea21740 10 client.18399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f430ea21740 20 client.18399 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f430ea21740 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0e75fc0 0x560ff0e783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f430ea21740 4 client.18399 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ff0db11f0 con 0x560ff0e75fc0 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.563+0000 7f43091bb640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0e75fc0 0x560ff0e783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f42ed7fa640 10 client.18399 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f43091bb640 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0e75fc0 0x560ff0e783e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9834d8210a15d2e server_cookie=7b3b724a8aacefaf in_seq=0 out_seq=0 2024-09-13T17:35:29.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f42ed7fa640 1 -- 172.21.15.44:0/3284049050 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560ff0db11f0 con 0x560ff0e75fc0 2024-09-13T17:35:29.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f42ed7fa640 10 client.18399 handle_command_reply: tid=0 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f430ea21740 1 client.18399 shutdown 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 msgr2=0x560ff0e631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.564+0000 7f430ea21740 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e631f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4300000c00 tx=0x7f4300032b60 comp rx=0 tx=0).stop 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 shutdown_connections 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0e75fc0 0x560ff0e783e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e63730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e631f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 --2- 172.21.15.44:0/3284049050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f42ed7fa640 0 client.18399 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 >> 172.21.15.44:0/3284049050 conn(0x560ff0e57d00 msgr2=0x560ff0e5da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 shutdown_connections 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 1 -- 172.21.15.44:0/3284049050 wait complete. 2024-09-13T17:35:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.565+0000 7f430ea21740 20 client.18399 trim_cache size 0 max 16384 2024-09-13T17:35:29.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 Processor -- start 2024-09-13T17:35:29.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 -- start start 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5c0b0 0x560ff0e61aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 0x560ff0e61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0e69400 con 0x560ff0e66950 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0e64360 con 0x560ff0e5b790 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f430ea21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0e62520 con 0x560ff0e5c0b0 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 0x560ff0e61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 0x560ff0e61fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40266/0 (socket says 172.21.15.44:40266) 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 -- 172.21.15.44:0/1040885782 learned_addr learned my addr 172.21.15.44:0/1040885782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43099bc640 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5c0b0 0x560ff0e61aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 -- 172.21.15.44:0/1040885782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5c0b0 msgr2=0x560ff0e61aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5c0b0 0x560ff0e61aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 -- 172.21.15.44:0/1040885782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 msgr2=0x560ff0e576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 -- 172.21.15.44:0/1040885782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff0d5f770 con 0x560ff0e66950 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.576+0000 7f43091bb640 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 0x560ff0e61fe0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005d00 tx=0x7f42f4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756d18e9c54ab256 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f42f8e05640 1 -- 172.21.15.44:0/1040885782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4028070 con 0x560ff0e66950 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f42f8e05640 1 -- 172.21.15.44:0/1040885782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42f4012450 con 0x560ff0e66950 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f42f8e05640 1 -- 172.21.15.44:0/1040885782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4010430 con 0x560ff0e66950 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 -- 172.21.15.44:0/1040885782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 msgr2=0x560ff0e61fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 0x560ff0e61fe0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005d00 tx=0x7f42f4014040 comp rx=0 tx=0).stop 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 -- 172.21.15.44:0/1040885782 shutdown_connections 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5c0b0 0x560ff0e61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e66950 0x560ff0e61fe0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 --2- 172.21.15.44:0/1040885782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e5b790 0x560ff0e576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 -- 172.21.15.44:0/1040885782 >> 172.21.15.44:0/1040885782 conn(0x560ff0e57d00 msgr2=0x560ff0e7cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 -- 172.21.15.44:0/1040885782 shutdown_connections 2024-09-13T17:35:29.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 -- 172.21.15.44:0/1040885782 wait complete. 2024-09-13T17:35:29.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.577+0000 7f430ea21740 1 -- start start 2024-09-13T17:35:29.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 Processor -- start 2024-09-13T17:35:29.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e6d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e6d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e6b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0e6c080 con 0x560ff0e5c0b0 2024-09-13T17:35:29.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0e6c1f0 con 0x560ff0e66950 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff0e6c3b0 con 0x560ff0e5b790 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f430ea21740 10 client.0 _ll_register_callbacks cb 0x7f42fc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e6d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e6d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40274/0 (socket says 172.21.15.44:40274) 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 -- 172.21.15.44:0/2357646295 learned_addr learned my addr 172.21.15.44:0/2357646295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43091bb640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e6b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43091bb640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e6d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 -- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 msgr2=0x560ff0e6d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e6d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 -- 172.21.15.44:0/2357646295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 msgr2=0x560ff0e6b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e6b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43099bc640 1 -- 172.21.15.44:0/2357646295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f401a040 con 0x560ff0e5c0b0 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.578+0000 7f43091bb640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e6b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:29.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f43099bc640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e6d980 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f430003ccb0 tx=0x7f430003c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5796f86f92c63f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f42ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f42ed7fa640 1 -- 172.21.15.44:0/2357646295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300004230 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4300031960 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f42ed7fa640 1 -- 172.21.15.44:0/2357646295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4300000e40 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f42ed7fa640 1 -- 172.21.15.44:0/2357646295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4300031480 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 10 client.18405 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560ff0d84e80 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f42ed7fa640 1 -- 172.21.15.44:0/2357646295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4300031960 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f42ed7fa640 1 -- 172.21.15.44:0/2357646295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4300042040 con 0x560ff0e5c0b0 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 10 client.18405 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 10 client.18405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 20 client.18405 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0ecd200 0x560ff0ecf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 4 client.18405 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.579+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560ff0ab24e0 con 0x560ff0ecd200 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f43091bb640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0ecd200 0x560ff0ecf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f42ed7fa640 10 client.18405 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f43091bb640 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0ecd200 0x560ff0ecf620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f002d9ae563b0045 server_cookie=d8580a5f16c50add in_seq=0 out_seq=0 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f42ed7fa640 1 -- 172.21.15.44:0/2357646295 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560ff0ab24e0 con 0x560ff0ecd200 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f42ed7fa640 10 client.18405 handle_command_reply: tid=0 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f430ea21740 1 client.18405 shutdown 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 msgr2=0x560ff0e6d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f430ea21740 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e6d980 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f430003ccb0 tx=0x7f430003c670 comp rx=0 tx=0).stop 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 shutdown_connections 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.580+0000 7f430ea21740 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x560ff0ecd200 0x560ff0ecf620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560ff0e66950 0x560ff0e6b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff0e5c0b0 0x560ff0e6d980 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 1 --2- 172.21.15.44:0/2357646295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ff0e5b790 0x560ff0e6d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f42ed7fa640 0 client.18405 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:35:29.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 >> 172.21.15.44:0/2357646295 conn(0x560ff0e57d00 msgr2=0x560ff0e72100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 shutdown_connections 2024-09-13T17:35:29.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 1 -- 172.21.15.44:0/2357646295 wait complete. 2024-09-13T17:35:29.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430ea21740 20 client.18405 trim_cache size 0 max 16384 2024-09-13T17:35:29.588 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f42f00a1280 msgr2=0x7f42f00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f42f00a1280 0x7f42f00a3740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42fc09a060 tx=0x7f42fc096040 comp rx=0 tx=0).stop 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 msgr2=0x7f430419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 0x7f430419cd30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00eb10 tx=0x7f42fc00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 shutdown_connections 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f42f00a1280 0x7f42f00a3740 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f430410c560 0x7f430419cd30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4304073200 0x7f4304198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 --2- 172.21.15.44:0/1621923015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43040728e0 0x7f4304198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 >> 172.21.15.44:0/1621923015 conn(0x7f43040fba20 msgr2=0x7f43040fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 shutdown_connections 2024-09-13T17:35:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.581+0000 7f430bc20640 1 -- 172.21.15.44:0/1621923015 wait complete. 2024-09-13T17:35:29.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:29 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:35:29.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:29 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:29.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:29 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:35:29.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:29 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:29.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:29 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:35:29.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:29 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/115017951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:35:29.594 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:35:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:29 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:35:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:29 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:29 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:35:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:29 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:29 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:35:29.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:29 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/115017951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:35:29.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 -- 172.21.15.44:0/40394521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554104210 msgr2=0x7fc55410bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 --2- 172.21.15.44:0/40394521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554104210 0x7fc55410bd60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009cd0 tx=0x7fc54c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 -- 172.21.15.44:0/40394521 shutdown_connections 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 --2- 172.21.15.44:0/40394521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540fff90 0x7fc554103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 --2- 172.21.15.44:0/40394521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554104210 0x7fc55410bd60 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 --2- 172.21.15.44:0/40394521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5540ff5e0 0x7fc5540ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 -- 172.21.15.44:0/40394521 >> 172.21.15.44:0/40394521 conn(0x7fc5540753c0 msgr2=0x7fc5540fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 -- 172.21.15.44:0/40394521 shutdown_connections 2024-09-13T17:35:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.697+0000 7fc55ade2640 1 -- 172.21.15.44:0/40394521 wait complete. 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 Processor -- start 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 -- start start 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540ff5e0 0x7fc554195220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc554104210 0x7fc554195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 0x7fc55419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55419a540 con 0x7fc554199b50 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55419a6b0 con 0x7fc554104210 2024-09-13T17:35:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55ade2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55419a820 con 0x7fc5540ff5e0 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55937f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 0x7fc55419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55937f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 0x7fc55419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40282/0 (socket says 172.21.15.44:40282) 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc55937f640 1 -- 172.21.15.44:0/2779323548 learned_addr learned my addr 172.21.15.44:0/2779323548 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc553fff640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc554104210 0x7fc554195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.698+0000 7fc558b7e640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540ff5e0 0x7fc554195220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55937f640 1 -- 172.21.15.44:0/2779323548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540ff5e0 msgr2=0x7fc554195220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55937f640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540ff5e0 0x7fc554195220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55937f640 1 -- 172.21.15.44:0/2779323548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc554104210 msgr2=0x7fc554195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55937f640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc554104210 0x7fc554195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55937f640 1 -- 172.21.15.44:0/2779323548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009900 con 0x7fc554199b50 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc558b7e640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540ff5e0 0x7fc554195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55937f640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 0x7fc55419a000 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc54c004af0 tx=0x7fc54c030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc551ffb640 1 -- 172.21.15.44:0/2779323548 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c031b80 con 0x7fc554199b50 2024-09-13T17:35:29.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5541a1e00 con 0x7fc554199b50 2024-09-13T17:35:29.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc551ffb640 1 -- 172.21.15.44:0/2779323548 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc54c003e00 con 0x7fc554199b50 2024-09-13T17:35:29.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc551ffb640 1 -- 172.21.15.44:0/2779323548 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c004840 con 0x7fc554199b50 2024-09-13T17:35:29.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.699+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5541a2260 con 0x7fc554199b50 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.701+0000 7fc551ffb640 1 -- 172.21.15.44:0/2779323548 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc54c00a040 con 0x7fc554199b50 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.701+0000 7fc55dbda740 1 Processor -- start 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 -- start start 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 0x5559b4cc0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4ccb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4ccbb60 0x5559b4ccdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4c28cc0 con 0x5559b4cc09a0 2024-09-13T17:35:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4c26450 con 0x5559b4ccbb60 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55dbda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4c21d40 con 0x5559b4cc12c0 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc551ffb640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5440a1200 0x7fc5440a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 0x5559b4cc0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 0x5559b4cc0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40286/0 (socket says 172.21.15.44:40286) 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 -- 172.21.15.44:0/2231219416 learned_addr learned my addr 172.21.15.44:0/2231219416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc55937f640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5440a1200 0x7fc5440a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc558b7e640 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4ccb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4ccbb60 0x5559b4ccdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc551ffb640 1 -- 172.21.15.44:0/2779323548 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc54c0ee4b0 con 0x7fc554199b50 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 -- 172.21.15.44:0/2231219416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 msgr2=0x5559b4ccb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4ccb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 -- 172.21.15.44:0/2231219416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4ccbb60 msgr2=0x5559b4ccdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4ccbb60 0x5559b4ccdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 -- 172.21.15.44:0/2231219416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559b4be9620 con 0x5559b4cc09a0 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.702+0000 7fc553fff640 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 0x5559b4cc0d80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc540013530 tx=0x7fc54000a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a9f8b69d699ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55937f640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5440a1200 0x7fc5440a36c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0baef0 tx=0x7fc54c02a8e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc53effd640 1 -- 172.21.15.44:0/2231219416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400118b0 con 0x5559b4cc09a0 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc53effd640 1 -- 172.21.15.44:0/2231219416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc540011ea0 con 0x5559b4cc09a0 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc53effd640 1 -- 172.21.15.44:0/2231219416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54001ed50 con 0x5559b4cc09a0 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 -- 172.21.15.44:0/2231219416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 msgr2=0x5559b4cc0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 0x5559b4cc0d80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc540013530 tx=0x7fc54000a760 comp rx=0 tx=0).stop 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 -- 172.21.15.44:0/2231219416 shutdown_connections 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4ccb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4ccbb60 0x5559b4ccdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 --2- 172.21.15.44:0/2231219416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc09a0 0x5559b4cc0d80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 -- 172.21.15.44:0/2231219416 >> 172.21.15.44:0/2231219416 conn(0x5559b4cbcf10 msgr2=0x5559b4cbd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 -- 172.21.15.44:0/2231219416 shutdown_connections 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.703+0000 7fc55dbda740 1 -- 172.21.15.44:0/2231219416 wait complete. 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 -- start start 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 Processor -- start 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4ccb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc12c0 0x5559b4cc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 0x5559b4cc8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4be0180 con 0x5559b4ccbb60 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b48fa430 con 0x5559b4cc12c0 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4ba55e0 con 0x5559b4cc09a0 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc55dbda740 10 client.0 _ll_register_callbacks cb 0x5559b4c11aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 0x5559b4cc8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 0x5559b4cc8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40290/0 (socket says 172.21.15.44:40290) 2024-09-13T17:35:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc553fff640 1 -- 172.21.15.44:0/388398069 learned_addr learned my addr 172.21.15.44:0/388398069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.704+0000 7fc558b7e640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc12c0 0x5559b4cc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4ccb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 -- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 msgr2=0x5559b4ccb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4ccb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 -- 172.21.15.44:0/388398069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc12c0 msgr2=0x5559b4cc8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc12c0 0x5559b4cc8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 -- 172.21.15.44:0/388398069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54000ace0 con 0x5559b4ccbb60 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc553fff640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 0x5559b4cc8970 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc540005790 tx=0x7fc54000a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d82b22c88f5fef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc533fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc533fff640 1 -- 172.21.15.44:0/388398069 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540022070 con 0x5559b4ccbb60 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc533fff640 1 -- 172.21.15.44:0/388398069 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc540011b40 con 0x5559b4ccbb60 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559b4c4bb80 con 0x5559b4ccbb60 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc533fff640 1 -- 172.21.15.44:0/388398069 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc540011d00 con 0x5559b4ccbb60 2024-09-13T17:35:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc55dbda740 10 client.18417 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5559b4c203c0 con 0x5559b4ccbb60 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.705+0000 7fc533fff640 1 -- 172.21.15.44:0/388398069 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5559b4c4bb80 con 0x5559b4ccbb60 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc533fff640 1 -- 172.21.15.44:0/388398069 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc54000f4d0 con 0x5559b4ccbb60 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc55dbda740 10 client.18417 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc55dbda740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc55dbda740 20 client.18417 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc55dbda740 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4cdb380 0x5559b4cdd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc55dbda740 4 client.18417 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559b4c141f0 con 0x5559b4cdb380 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc558b7e640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4cdb380 0x5559b4cdd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc533fff640 10 client.18417 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.706+0000 7fc558b7e640 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4cdb380 0x5559b4cdd7a0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=493f269cfd04d21d server_cookie=e0b77c98d9cfc638 in_seq=0 out_seq=0 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc533fff640 1 -- 172.21.15.44:0/388398069 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5559b4c141f0 con 0x5559b4cdb380 2024-09-13T17:35:29.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc533fff640 10 client.18417 handle_command_reply: tid=0 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 client.18417 shutdown 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 msgr2=0x5559b4cc8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 0x5559b4cc8970 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc540005790 tx=0x7fc54000a510 comp rx=0 tx=0).stop 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 shutdown_connections 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4cdb380 0x5559b4cdd7a0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4ccbb60 0x5559b4cc8970 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc12c0 0x5559b4cc8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc533fff640 0 client.18417 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 --2- 172.21.15.44:0/388398069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4ccb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 >> 172.21.15.44:0/388398069 conn(0x5559b4cbcf10 msgr2=0x5559b4cb9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 shutdown_connections 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 1 -- 172.21.15.44:0/388398069 wait complete. 2024-09-13T17:35:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.707+0000 7fc55dbda740 20 client.18417 trim_cache size 0 max 16384 2024-09-13T17:35:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 Processor -- start 2024-09-13T17:35:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 -- start start 2024-09-13T17:35:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc09a0 0x5559b4cd2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4cc6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4c4bb80 con 0x5559b4cbcf10 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4cdb1b0 con 0x5559b4cc09a0 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.718+0000 7fc55dbda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4cb68a0 con 0x5559b4cc12c0 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40296/0 (socket says 172.21.15.44:40296) 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 -- 172.21.15.44:0/1144698585 learned_addr learned my addr 172.21.15.44:0/1144698585 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc558b7e640 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc09a0 0x5559b4cd2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 -- 172.21.15.44:0/1144698585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 msgr2=0x5559b4cc6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4cc6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 -- 172.21.15.44:0/1144698585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc09a0 msgr2=0x5559b4cd2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc09a0 0x5559b4cd2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 -- 172.21.15.44:0/1144698585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559b4bc2770 con 0x5559b4cbcf10 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc553fff640 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2880 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc540013480 tx=0x7fc540005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ebaefc72addaad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc53f7fe640 1 -- 172.21.15.44:0/1144698585 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540022040 con 0x5559b4cbcf10 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc53f7fe640 1 -- 172.21.15.44:0/1144698585 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc540023040 con 0x5559b4cbcf10 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.719+0000 7fc53f7fe640 1 -- 172.21.15.44:0/1144698585 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54000f6a0 con 0x5559b4cbcf10 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- 172.21.15.44:0/1144698585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 msgr2=0x5559b4cd2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2880 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc540013480 tx=0x7fc540005e50 comp rx=0 tx=0).stop 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- 172.21.15.44:0/1144698585 shutdown_connections 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc12c0 0x5559b4cc6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cc09a0 0x5559b4cd2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- 172.21.15.44:0/1144698585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2880 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- 172.21.15.44:0/1144698585 >> 172.21.15.44:0/1144698585 conn(0x5559b4cbd890 msgr2=0x5559b4ce2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- 172.21.15.44:0/1144698585 shutdown_connections 2024-09-13T17:35:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- 172.21.15.44:0/1144698585 wait complete. 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- start start 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 Processor -- start 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4cc4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 0x5559b4cc5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4cd3180 con 0x5559b4cc12c0 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4cc5540 con 0x5559b4cbcf10 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.720+0000 7fc55dbda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559b4cc56d0 con 0x5559b4cc09a0 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc55dbda740 10 client.0 _ll_register_callbacks cb 0x7fc54000a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 0x5559b4cc5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 0x5559b4cc5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40298/0 (socket says 172.21.15.44:40298) 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 -- 172.21.15.44:0/151823897 learned_addr learned my addr 172.21.15.44:0/151823897 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc558b7e640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4cc4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 -- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 msgr2=0x5559b4cc4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4cc4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 -- 172.21.15.44:0/151823897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cbcf10 msgr2=0x5559b4cd2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 -- 172.21.15.44:0/151823897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559b4c203c0 con 0x5559b4cc12c0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc553fff640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 0x5559b4cc5000 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc54000a800 tx=0x7fc54000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf81f140fa076521 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc533fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc533fff640 1 -- 172.21.15.44:0/151823897 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54000f650 con 0x5559b4cc12c0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc533fff640 1 -- 172.21.15.44:0/151823897 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc540011940 con 0x5559b4cc12c0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.721+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc540011ea0 con 0x5559b4cc12c0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc533fff640 1 -- 172.21.15.44:0/151823897 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc540011b00 con 0x5559b4cc12c0 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 10 client.18423 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5559b4be7e80 con 0x5559b4cc12c0 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc533fff640 1 -- 172.21.15.44:0/151823897 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc540011ea0 con 0x5559b4cc12c0 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc533fff640 1 -- 172.21.15.44:0/151823897 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc540004660 con 0x5559b4cc12c0 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 10 client.18423 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 20 client.18423 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4ccbb60 0x5559b4d34740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 4 client.18423 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559b49154e0 con 0x5559b4ccbb60 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.722+0000 7fc558b7e640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4ccbb60 0x5559b4d34740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc533fff640 10 client.18423 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc558b7e640 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4ccbb60 0x5559b4d34740 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfd4ecdb68e9d19 server_cookie=200dd2d056e39b7 in_seq=0 out_seq=0 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc533fff640 1 -- 172.21.15.44:0/151823897 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559b49154e0 con 0x5559b4ccbb60 2024-09-13T17:35:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc533fff640 10 client.18423 handle_command_reply: tid=0 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 client.18423 shutdown 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 msgr2=0x5559b4cc5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 0x5559b4cc5000 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc54000a800 tx=0x7fc54000af70 comp rx=0 tx=0).stop 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 shutdown_connections 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5559b4ccbb60 0x5559b4d34740 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559b4cc09a0 0x5559b4cc4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559b4cc12c0 0x5559b4cc5000 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 --2- 172.21.15.44:0/151823897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559b4cbcf10 0x5559b4cd2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc533fff640 0 client.18423 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 >> 172.21.15.44:0/151823897 conn(0x5559b4cbd890 msgr2=0x5559b4cc6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 shutdown_connections 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 1 -- 172.21.15.44:0/151823897 wait complete. 2024-09-13T17:35:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.723+0000 7fc55dbda740 20 client.18423 trim_cache size 0 max 16384 2024-09-13T17:35:29.731 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:35:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5440a1200 msgr2=0x7fc5440a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5440a1200 0x7fc5440a36c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0baef0 tx=0x7fc54c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 msgr2=0x7fc55419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 0x7fc55419a000 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc54c004af0 tx=0x7fc54c030e50 comp rx=0 tx=0).stop 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 shutdown_connections 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc5440a1200 0x7fc5440a36c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc554199b50 0x7fc55419a000 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc554104210 0x7fc554195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 --2- 172.21.15.44:0/2779323548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540ff5e0 0x7fc554195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 >> 172.21.15.44:0/2779323548 conn(0x7fc5540753c0 msgr2=0x7fc5541022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 shutdown_connections 2024-09-13T17:35:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:29.724+0000 7fc55ade2640 1 -- 172.21.15.44:0/2779323548 wait complete. 2024-09-13T17:35:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:29 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:35:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:29 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:29 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:35:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:29 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:29.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:29 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:35:29.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:29 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/115017951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:35:30.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:30 smithi044 ceph-mon[31175]: pgmap v272: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:35:30.675 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:30 smithi100 ceph-mon[35055]: pgmap v272: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:35:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:30 smithi070 ceph-mon[34502]: pgmap v272: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:35:30.736 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:35:30.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.857+0000 7fb47ae44640 1 -- 172.21.15.44:0/1359313955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474100f30 msgr2=0x7fb474101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.857+0000 7fb47ae44640 1 --2- 172.21.15.44:0/1359313955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474100f30 0x7fb474101310 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb468009900 tx=0x7fb46802a480 comp rx=0 tx=0).stop 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- 172.21.15.44:0/1359313955 shutdown_connections 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 --2- 172.21.15.44:0/1359313955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb47410a2c0 0x7fb47410c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 --2- 172.21.15.44:0/1359313955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb474101850 0x7fb474109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 --2- 172.21.15.44:0/1359313955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474100f30 0x7fb474101310 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- 172.21.15.44:0/1359313955 >> 172.21.15.44:0/1359313955 conn(0x7fb4740fa8c0 msgr2=0x7fb4740fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- 172.21.15.44:0/1359313955 shutdown_connections 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- 172.21.15.44:0/1359313955 wait complete. 2024-09-13T17:35:30.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 Processor -- start 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- start start 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb474100f30 0x7fb474196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 0x7fb4741966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb47410a2c0 0x7fb47419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47419b0c0 con 0x7fb474101850 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47419b230 con 0x7fb474100f30 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.858+0000 7fb47ae44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47419b3f0 con 0x7fb47410a2c0 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 0x7fb4741966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 0x7fb4741966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40302/0 (socket says 172.21.15.44:40302) 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 -- 172.21.15.44:0/4117420919 learned_addr learned my addr 172.21.15.44:0/4117420919 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb4793e1640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb47410a2c0 0x7fb47419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb478be0640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb474100f30 0x7fb474196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 -- 172.21.15.44:0/4117420919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb47410a2c0 msgr2=0x7fb47419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb47410a2c0 0x7fb47419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 -- 172.21.15.44:0/4117420919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb474100f30 msgr2=0x7fb474196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb474100f30 0x7fb474196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 -- 172.21.15.44:0/4117420919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468009560 con 0x7fb474101850 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb473fff640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 0x7fb4741966d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb46000d950 tx=0x7fb46000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb471ffb640 1 -- 172.21.15.44:0/4117420919 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460009c30 con 0x7fb474101850 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb471ffb640 1 -- 172.21.15.44:0/4117420919 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb460009d90 con 0x7fb474101850 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb471ffb640 1 -- 172.21.15.44:0/4117420919 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46000f620 con 0x7fb474101850 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.859+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47419b6d0 con 0x7fb474101850 2024-09-13T17:35:30.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.860+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4741a2f50 con 0x7fb474101850 2024-09-13T17:35:30.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.862+0000 7fb471ffb640 1 -- 172.21.15.44:0/4117420919 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4600109f0 con 0x7fb474101850 2024-09-13T17:35:30.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.862+0000 7fb47dc36740 1 Processor -- start 2024-09-13T17:35:30.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.862+0000 7fb47dc36740 1 -- start start 2024-09-13T17:35:30.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db0097c0 0x5606db009ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db014440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db014980 0x5606db016d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb47dc36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606daf74c60 con 0x5606db014980 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb47dc36740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606daf723f0 con 0x5606db00a0e0 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb47dc36740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606daf6dce0 con 0x5606db0097c0 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb471ffb640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4640a1200 0x7fb4640a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb473fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db014980 0x5606db016d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb478be0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db014440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4640a1200 0x7fb4640a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb478be0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db014440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:57110/0 (socket says 172.21.15.44:57110) 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db0097c0 0x5606db009ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb478be0640 1 -- 172.21.15.44:0/1556768370 learned_addr learned my addr 172.21.15.44:0/1556768370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:30.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb471ffb640 1 -- 172.21.15.44:0/4117420919 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb4600cccb0 con 0x7fb474101850 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 -- 172.21.15.44:0/1556768370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 msgr2=0x5606db014440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db014440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 -- 172.21.15.44:0/1556768370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db014980 msgr2=0x5606db016d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db014980 0x5606db016d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.863+0000 7fb4793e1640 1 -- 172.21.15.44:0/1556768370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606daf29970 con 0x5606db0097c0 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.864+0000 7fb4793e1640 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db0097c0 0x5606db009ba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005fd0 tx=0x7fb46c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a878c6108059a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.864+0000 7fb4793e1640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4640a1200 0x7fb4640a36c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00fc30 tx=0x7fb46c017000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.864+0000 7fb45effd640 1 -- 172.21.15.44:0/1556768370 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c012df0 con 0x5606db0097c0 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.864+0000 7fb45effd640 1 -- 172.21.15.44:0/1556768370 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb46c009c40 con 0x5606db0097c0 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.864+0000 7fb45effd640 1 -- 172.21.15.44:0/1556768370 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c004030 con 0x5606db0097c0 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 -- 172.21.15.44:0/1556768370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db0097c0 msgr2=0x5606db009ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db0097c0 0x5606db009ba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005fd0 tx=0x7fb46c00f730 comp rx=0 tx=0).stop 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 -- 172.21.15.44:0/1556768370 shutdown_connections 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db014980 0x5606db016d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db014440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 --2- 172.21.15.44:0/1556768370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db0097c0 0x5606db009ba0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 -- 172.21.15.44:0/1556768370 >> 172.21.15.44:0/1556768370 conn(0x5606db005d50 msgr2=0x5606db006130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 -- 172.21.15.44:0/1556768370 shutdown_connections 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 -- 172.21.15.44:0/1556768370 wait complete. 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 -- start start 2024-09-13T17:35:30.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.865+0000 7fb47dc36740 1 Processor -- start 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db014360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db00a0e0 0x5606db011680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db014980 0x5606db011ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606daf2bf80 con 0x5606db0097c0 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606daf2bc20 con 0x5606db014980 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606daef8ab0 con 0x5606db00a0e0 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 10 client.0 _ll_register_callbacks cb 0x5606daf67350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db014360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db014360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40320/0 (socket says 172.21.15.44:40320) 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 -- 172.21.15.44:0/4287319682 learned_addr learned my addr 172.21.15.44:0/4287319682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb473fff640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db00a0e0 0x5606db011680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb4793e1640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db014980 0x5606db011ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 -- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db00a0e0 msgr2=0x5606db011680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db00a0e0 0x5606db011680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 -- 172.21.15.44:0/4287319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db014980 msgr2=0x5606db011ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db014980 0x5606db011ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 -- 172.21.15.44:0/4287319682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c01a040 con 0x5606db0097c0 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb473fff640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db00a0e0 0x5606db011680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:35:30.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb4793e1640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db014980 0x5606db011ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb478be0640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db014360 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb46802aa70 tx=0x7fb468004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a0a933f50a45c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb453fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb453fff640 1 -- 172.21.15.44:0/4287319682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468031920 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb453fff640 1 -- 172.21.15.44:0/4287319682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb468031a80 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.866+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606dac61590 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb453fff640 1 -- 172.21.15.44:0/4287319682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb468031c40 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 10 client.18435 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5606daf6c360 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb453fff640 1 -- 172.21.15.44:0/4287319682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5606dac61590 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb453fff640 1 -- 172.21.15.44:0/4287319682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb468046470 con 0x5606db0097c0 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 10 client.18435 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:30.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 20 client.18435 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db026ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 4 client.18435 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606dac72860 con 0x5606db0246e0 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.867+0000 7fb473fff640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db026ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.868+0000 7fb453fff640 10 client.18435 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.868+0000 7fb473fff640 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db026ad0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0d5241c2f50eac server_cookie=74601360cdb1fca8 in_seq=0 out_seq=0 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.868+0000 7fb453fff640 1 -- 172.21.15.44:0/4287319682 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5606dac72860 con 0x5606db0246e0 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.868+0000 7fb453fff640 10 client.18435 handle_command_reply: tid=0 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.868+0000 7fb47dc36740 1 client.18435 shutdown 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 msgr2=0x5606db014360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db014360 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb46802aa70 tx=0x7fb468004270 comp rx=0 tx=0).stop 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 shutdown_connections 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db026ad0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb453fff640 0 client.18435 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db00a0e0 0x5606db011680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db014980 0x5606db011ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 --2- 172.21.15.44:0/4287319682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db014360 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 >> 172.21.15.44:0/4287319682 conn(0x5606db005d50 msgr2=0x5606db00b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 shutdown_connections 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 1 -- 172.21.15.44:0/4287319682 wait complete. 2024-09-13T17:35:30.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.869+0000 7fb47dc36740 20 client.18435 trim_cache size 0 max 16384 2024-09-13T17:35:30.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.883+0000 7fb47dc36740 1 Processor -- start 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.883+0000 7fb47dc36740 1 -- start start 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db011630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db001390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb47dc36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606db00aaa0 con 0x5606db0097c0 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb47dc36740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606db020590 con 0x5606db00a0e0 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb47dc36740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606db009660 con 0x5606db014980 2024-09-13T17:35:30.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db011630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db011630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40328/0 (socket says 172.21.15.44:40328) 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 -- 172.21.15.44:0/3491539591 learned_addr learned my addr 172.21.15.44:0/3491539591 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb4793e1640 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb473fff640 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db001390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 -- 172.21.15.44:0/3491539591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 msgr2=0x5606db01b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 -- 172.21.15.44:0/3491539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 msgr2=0x5606db001390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db001390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 -- 172.21.15.44:0/3491539591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606daf0e1c0 con 0x5606db0097c0 2024-09-13T17:35:30.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb473fff640 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db001390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb478be0640 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db011630 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb468006010 tx=0x7fb468046a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e7c611a75c865a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb45f7fe640 1 -- 172.21.15.44:0/3491539591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468046af0 con 0x5606db0097c0 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb45f7fe640 1 -- 172.21.15.44:0/3491539591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb468041070 con 0x5606db0097c0 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.884+0000 7fb45f7fe640 1 -- 172.21.15.44:0/3491539591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468003950 con 0x5606db0097c0 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 -- 172.21.15.44:0/3491539591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 msgr2=0x5606db011630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db011630 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb468006010 tx=0x7fb468046a20 comp rx=0 tx=0).stop 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 -- 172.21.15.44:0/3491539591 shutdown_connections 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db001390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 --2- 172.21.15.44:0/3491539591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db011630 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 -- 172.21.15.44:0/3491539591 >> 172.21.15.44:0/3491539591 conn(0x5606db005d50 msgr2=0x5606db010f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 -- 172.21.15.44:0/3491539591 shutdown_connections 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 -- 172.21.15.44:0/3491539591 wait complete. 2024-09-13T17:35:30.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.885+0000 7fb47dc36740 1 -- start start 2024-09-13T17:35:30.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 Processor -- start 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db01b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db019b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606db01a600 con 0x5606db0097c0 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606db01a770 con 0x5606db00a0e0 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606db01a930 con 0x5606db014980 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb47dc36740 10 client.0 _ll_register_callbacks cb 0x7fb4680315f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db01b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db01b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40334/0 (socket says 172.21.15.44:40334) 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 -- 172.21.15.44:0/2893634517 learned_addr learned my addr 172.21.15.44:0/2893634517 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb473fff640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db019b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb4793e1640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 -- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 msgr2=0x5606db01a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 -- 172.21.15.44:0/2893634517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 msgr2=0x5606db019b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db019b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.886+0000 7fb478be0640 1 -- 172.21.15.44:0/2893634517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606daf6c360 con 0x5606db0097c0 2024-09-13T17:35:30.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb478be0640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db01b6c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb4680057c0 tx=0x7fb468031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b04a9c7927678e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:30.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb453fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb453fff640 1 -- 172.21.15.44:0/2893634517 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468031d30 con 0x5606db0097c0 2024-09-13T17:35:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb453fff640 1 -- 172.21.15.44:0/2893634517 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb468000e40 con 0x5606db0097c0 2024-09-13T17:35:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb46c009c40 con 0x5606db0097c0 2024-09-13T17:35:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb453fff640 1 -- 172.21.15.44:0/2893634517 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb468031e90 con 0x5606db0097c0 2024-09-13T17:35:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 10 client.18441 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5606daf33990 con 0x5606db0097c0 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb453fff640 1 -- 172.21.15.44:0/2893634517 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb46c009c40 con 0x5606db0097c0 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb453fff640 1 -- 172.21.15.44:0/2893634517 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb46803d710 con 0x5606db0097c0 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 10 client.18441 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 20 client.18441 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db07da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 4 client.18441 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.887+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5606dacc5eb0 con 0x5606db0246e0 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.888+0000 7fb473fff640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db07da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.888+0000 7fb473fff640 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db07da70 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230ace90a8d7a40 server_cookie=7686d66142af9e82 in_seq=0 out_seq=0 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.888+0000 7fb453fff640 10 client.18441 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb453fff640 1 -- 172.21.15.44:0/2893634517 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5606dacc5eb0 con 0x5606db0246e0 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb453fff640 10 client.18441 handle_command_reply: tid=0 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 client.18441 shutdown 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 msgr2=0x5606db01b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db01b6c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb4680057c0 tx=0x7fb468031560 comp rx=0 tx=0).stop 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 shutdown_connections 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5606db0246e0 0x5606db07da70 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606db014980 0x5606db01a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5606db00a0e0 0x5606db019b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 --2- 172.21.15.44:0/2893634517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606db0097c0 0x5606db01b6c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb453fff640 0 client.18441 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.889+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 >> 172.21.15.44:0/2893634517 conn(0x5606db005d50 msgr2=0x5606db004940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 shutdown_connections 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47dc36740 1 -- 172.21.15.44:0/2893634517 wait complete. 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47dc36740 20 client.18441 trim_cache size 0 max 16384 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:35:30.899 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "be8bee0c-b1e1-4319-8fcd-6f7d7fc58a64", 2024-09-13T17:35:30.900 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:35:30.900 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4640a1200 msgr2=0x7fb4640a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4640a1200 0x7fb4640a36c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00fc30 tx=0x7fb46c017000 comp rx=0 tx=0).stop 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 msgr2=0x7fb4741966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 0x7fb4741966d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb46000d950 tx=0x7fb46000de10 comp rx=0 tx=0).stop 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 shutdown_connections 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb4640a1200 0x7fb4640a36c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb47410a2c0 0x7fb47419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb474101850 0x7fb4741966d0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 --2- 172.21.15.44:0/4117420919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb474100f30 0x7fb474196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 >> 172.21.15.44:0/4117420919 conn(0x7fb4740fa8c0 msgr2=0x7fb4740fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:30.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 shutdown_connections 2024-09-13T17:35:30.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:30.890+0000 7fb47ae44640 1 -- 172.21.15.44:0/4117420919 wait complete. 2024-09-13T17:35:30.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:35:31.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 -- 172.21.15.44:0/2936518374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3084100900 msgr2=0x7f3084104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 --2- 172.21.15.44:0/2936518374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3084100900 0x7f3084104940 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3078009930 tx=0x7f307802a420 comp rx=0 tx=0).stop 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 -- 172.21.15.44:0/2936518374 shutdown_connections 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 --2- 172.21.15.44:0/2936518374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30841050a0 0x7f308410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 --2- 172.21.15.44:0/2936518374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3084100900 0x7f3084104940 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 --2- 172.21.15.44:0/2936518374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30840fff50 0x7f3084100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 -- 172.21.15.44:0/2936518374 >> 172.21.15.44:0/2936518374 conn(0x7f30840fb9e0 msgr2=0x7f30840fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 -- 172.21.15.44:0/2936518374 shutdown_connections 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.007+0000 7f3089e5a640 1 -- 172.21.15.44:0/2936518374 wait complete. 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 Processor -- start 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 -- start start 2024-09-13T17:35:31.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f3084196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 0x7f308419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419b460 con 0x7f308419aa70 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419b5d0 con 0x7f30840fff50 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3089e5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419b740 con 0x7f30841050a0 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3083fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 0x7f308419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3083fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 0x7f308419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40346/0 (socket says 172.21.15.44:40346) 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.008+0000 7f3083fff640 1 -- 172.21.15.44:0/3402200685 learned_addr learned my addr 172.21.15.44:0/3402200685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f30837fe640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3082ffd640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f3084196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3083fff640 1 -- 172.21.15.44:0/3402200685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 msgr2=0x7f3084196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3083fff640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f3084196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3083fff640 1 -- 172.21.15.44:0/3402200685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 msgr2=0x7f3084196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3083fff640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3083fff640 1 -- 172.21.15.44:0/3402200685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078009560 con 0x7f308419aa70 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f30837fe640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3083fff640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 0x7f308419af20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f307c00eb10 tx=0x7f307c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3080ff9640 1 -- 172.21.15.44:0/3402200685 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c004b20 con 0x7f308419aa70 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3080ff9640 1 -- 172.21.15.44:0/3402200685 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f307c004c80 con 0x7f308419aa70 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841a2d20 con 0x7f308419aa70 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3080ff9640 1 -- 172.21.15.44:0/3402200685 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c005a10 con 0x7f308419aa70 2024-09-13T17:35:31.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.009+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841a31a0 con 0x7f308419aa70 2024-09-13T17:35:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f3080ff9640 1 -- 172.21.15.44:0/3402200685 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f307c005020 con 0x7f308419aa70 2024-09-13T17:35:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 Processor -- start 2024-09-13T17:35:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 -- start start 2024-09-13T17:35:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b3f3e0 0x5633e8b417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8a9af20 con 0x5633e8b2fcf0 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8a986b0 con 0x5633e8b3f3e0 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.012+0000 7f308cc58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8a93fa0 con 0x5633e8b30610 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b300d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40348/0 (socket says 172.21.15.44:40348) 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 -- 172.21.15.44:0/3852036195 learned_addr learned my addr 172.21.15.44:0/3852036195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f30837fe640 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 -- 172.21.15.44:0/3852036195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 msgr2=0x5633e8b3eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 -- 172.21.15.44:0/3852036195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b3f3e0 msgr2=0x5633e8b417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b3f3e0 0x5633e8b417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 -- 172.21.15.44:0/3852036195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e8a4e840 con 0x5633e8b2fcf0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f30837fe640 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3080ff9640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3082ffd640 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b300d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f3084069630 tx=0x7f30780314f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467a691f032e8eda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f30837fe640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3080ff9640 1 -- 172.21.15.44:0/3402200685 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f307c014070 con 0x7f308419aa70 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3069ffb640 1 -- 172.21.15.44:0/3852036195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078004030 con 0x5633e8b2fcf0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3069ffb640 1 -- 172.21.15.44:0/3852036195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3078031960 con 0x5633e8b2fcf0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.013+0000 7f3069ffb640 1 -- 172.21.15.44:0/3852036195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078003b90 con 0x5633e8b2fcf0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 -- 172.21.15.44:0/3852036195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 msgr2=0x5633e8b300d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f30837fe640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5633e8b329e0 tx=0x7f307400a310 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b300d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f3084069630 tx=0x7f30780314f0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 -- 172.21.15.44:0/3852036195 shutdown_connections 2024-09-13T17:35:31.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b3f3e0 0x5633e8b417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 --2- 172.21.15.44:0/3852036195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b300d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 -- 172.21.15.44:0/3852036195 >> 172.21.15.44:0/3852036195 conn(0x5633e8b2c380 msgr2=0x5633e8b2c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 -- 172.21.15.44:0/3852036195 shutdown_connections 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 -- 172.21.15.44:0/3852036195 wait complete. 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.014+0000 7f308cc58740 1 -- start start 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 Processor -- start 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b2fcf0 0x5633e8b3edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b30610 0x5633e8b3c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 0x5633e8b3c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8a352a0 con 0x5633e8b3f3e0 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e876c430 con 0x5633e8b30610 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8a17680 con 0x5633e8b2fcf0 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f308cc58740 10 client.0 _ll_register_callbacks cb 0x5633e8a8d610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 0x5633e8b3c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 0x5633e8b3c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40352/0 (socket says 172.21.15.44:40352) 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 -- 172.21.15.44:0/349268834 learned_addr learned my addr 172.21.15.44:0/349268834 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f30837fe640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b2fcf0 0x5633e8b3edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3082ffd640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b30610 0x5633e8b3c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 -- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b2fcf0 msgr2=0x5633e8b3edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b2fcf0 0x5633e8b3edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 -- 172.21.15.44:0/349268834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b30610 msgr2=0x5633e8b3c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b30610 0x5633e8b3c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 -- 172.21.15.44:0/349268834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308410d430 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3083fff640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 0x5633e8b3c6d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f307c096e80 tx=0x7f307c004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea2d93cf03eccc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.015+0000 7f3062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f3062ffd640 1 -- 172.21.15.44:0/349268834 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c09a410 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f3062ffd640 1 -- 172.21.15.44:0/349268834 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f307c09e240 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633e87874e0 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f3062ffd640 1 -- 172.21.15.44:0/349268834 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307c0a2890 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 10 client.18453 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5633e8a92620 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f3062ffd640 1 -- 172.21.15.44:0/349268834 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5633e87874e0 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f3062ffd640 1 -- 172.21.15.44:0/349268834 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f307c01f060 con 0x5633e8b3f3e0 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 10 client.18453 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:31.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 20 client.18453 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b4f120 0x5633e8b51510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 4 client.18453 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.016+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633e87987e0 con 0x5633e8b4f120 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.017+0000 7f30837fe640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b4f120 0x5633e8b51510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.017+0000 7f30837fe640 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b4f120 0x5633e8b51510 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f4fcc22e28fc35 server_cookie=eb62205d40f0b576 in_seq=0 out_seq=0 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.017+0000 7f3062ffd640 10 client.18453 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.017+0000 7f3062ffd640 1 -- 172.21.15.44:0/349268834 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5633e87987e0 con 0x5633e8b4f120 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.017+0000 7f3062ffd640 10 client.18453 handle_command_reply: tid=0 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 client.18453 shutdown 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 msgr2=0x5633e8b3c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 0x5633e8b3c6d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f307c096e80 tx=0x7f307c004df0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 shutdown_connections 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b4f120 0x5633e8b51510 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b3f3e0 0x5633e8b3c6d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f3062ffd640 0 client.18453 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b30610 0x5633e8b3c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 --2- 172.21.15.44:0/349268834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b2fcf0 0x5633e8b3edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 >> 172.21.15.44:0/349268834 conn(0x5633e8b2c380 msgr2=0x5633e8b320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 shutdown_connections 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 1 -- 172.21.15.44:0/349268834 wait complete. 2024-09-13T17:35:31.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.018+0000 7f308cc58740 20 client.18453 trim_cache size 0 max 16384 2024-09-13T17:35:31.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 Processor -- start 2024-09-13T17:35:31.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 -- start start 2024-09-13T17:35:31.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b46640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e87874e0 con 0x5633e8b2fcf0 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8b4bf80 con 0x5633e8b2c380 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f308cc58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8b36e40 con 0x5633e8b30610 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b46640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b46640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40354/0 (socket says 172.21.15.44:40354) 2024-09-13T17:35:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.032+0000 7f3082ffd640 1 -- 172.21.15.44:0/721646710 learned_addr learned my addr 172.21.15.44:0/721646710 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f3082ffd640 1 -- 172.21.15.44:0/721646710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 msgr2=0x5633e8b3a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f3082ffd640 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f3082ffd640 1 -- 172.21.15.44:0/721646710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 msgr2=0x5633e8b46100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f3082ffd640 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f3082ffd640 1 -- 172.21.15.44:0/721646710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e8a34790 con 0x5633e8b2fcf0 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f3082ffd640 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b46640 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3078009930 tx=0x7f3078031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5cc8f7abaa7884 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f306a7fc640 1 -- 172.21.15.44:0/721646710 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078031cd0 con 0x5633e8b2fcf0 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f306a7fc640 1 -- 172.21.15.44:0/721646710 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3078003730 con 0x5633e8b2fcf0 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.033+0000 7f306a7fc640 1 -- 172.21.15.44:0/721646710 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307804e440 con 0x5633e8b2fcf0 2024-09-13T17:35:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 -- 172.21.15.44:0/721646710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 msgr2=0x5633e8b46640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b46640 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3078009930 tx=0x7f3078031840 comp rx=0 tx=0).stop 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 -- 172.21.15.44:0/721646710 shutdown_connections 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b3a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b46640 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 --2- 172.21.15.44:0/721646710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 -- 172.21.15.44:0/721646710 >> 172.21.15.44:0/721646710 conn(0x5633e8b3ef50 msgr2=0x5633e8b37300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 -- 172.21.15.44:0/721646710 shutdown_connections 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 -- 172.21.15.44:0/721646710 wait complete. 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 -- start start 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 Processor -- start 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.034+0000 7f308cc58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b42ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8b435d0 con 0x5633e8b2fcf0 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8b43740 con 0x5633e8b2c380 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e8ba55c0 con 0x5633e8b30610 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 10 client.0 _ll_register_callbacks cb 0x7f307c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b42970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40358/0 (socket says 172.21.15.44:40358) 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 -- 172.21.15.44:0/2715455472 learned_addr learned my addr 172.21.15.44:0/2715455472 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f30837fe640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 -- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 msgr2=0x5633e8b42ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b42ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 -- 172.21.15.44:0/2715455472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 msgr2=0x5633e8b464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 -- 172.21.15.44:0/2715455472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633e8a4e840 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f30837fe640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3082ffd640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b42970 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f307802aa10 tx=0x7f30780038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd94101a111955f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3062ffd640 1 -- 172.21.15.44:0/2715455472 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078052070 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3062ffd640 1 -- 172.21.15.44:0/2715455472 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f3078003a90 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3078031960 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f3062ffd640 1 -- 172.21.15.44:0/2715455472 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3078031ce0 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 10 client.18459 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.035+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5633e8a59ea0 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f3062ffd640 1 -- 172.21.15.44:0/2715455472 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3078031960 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f3062ffd640 1 -- 172.21.15.44:0/2715455472 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f307804e9a0 con 0x5633e8b2fcf0 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f308cc58740 10 client.18459 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f308cc58740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f308cc58740 20 client.18459 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f308cc58740 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b3f3e0 0x5633e8ba8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f308cc58740 4 client.18459 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633e87ebe30 con 0x5633e8b3f3e0 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f30837fe640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b3f3e0 0x5633e8ba8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f30837fe640 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b3f3e0 0x5633e8ba8850 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8216c46341833f server_cookie=fc347de7439db041 in_seq=0 out_seq=0 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.036+0000 7f3062ffd640 10 client.18459 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3062ffd640 1 -- 172.21.15.44:0/2715455472 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5633e87ebe30 con 0x5633e8b3f3e0 2024-09-13T17:35:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3062ffd640 10 client.18459 handle_command_reply: tid=0 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 client.18459 shutdown 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 msgr2=0x5633e8b42970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b42970 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f307802aa10 tx=0x7f30780038a0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 shutdown_connections 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5633e8b3f3e0 0x5633e8ba8850 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633e8b30610 0x5633e8b42ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633e8b2fcf0 0x5633e8b42970 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3062ffd640 0 client.18459 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 --2- 172.21.15.44:0/2715455472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5633e8b2c380 0x5633e8b464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 >> 172.21.15.44:0/2715455472 conn(0x5633e8b3ef50 msgr2=0x5633e8b3a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 shutdown_connections 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 1 -- 172.21.15.44:0/2715455472 wait complete. 2024-09-13T17:35:31.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f308cc58740 20 client.18459 trim_cache size 0 max 16384 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "be8bee0c-b1e1-4319-8fcd-6f7d7fc58a64": { 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "be8bee0c-b1e1-4319-8fcd-6f7d7fc58a64", 2024-09-13T17:35:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:35:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:35:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:35:31.048 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:35:31.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 msgr2=0x7f306c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5633e8b329e0 tx=0x7f307400a310 comp rx=0 tx=0).stop 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 msgr2=0x7f308419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 0x7f308419af20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f307c00eb10 tx=0x7f307c00c2d0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 shutdown_connections 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f306c0a1280 0x7f306c0a3740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.037+0000 7f3089e5a640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841050a0 0x7f3084196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.038+0000 7f3089e5a640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308419aa70 0x7f308419af20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.038+0000 7f3089e5a640 1 --2- 172.21.15.44:0/3402200685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30840fff50 0x7f3084196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:31.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.038+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 >> 172.21.15.44:0/3402200685 conn(0x7f30840fb9e0 msgr2=0x7f30840fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:31.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.038+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 shutdown_connections 2024-09-13T17:35:31.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:35:31.038+0000 7f3089e5a640 1 -- 172.21.15.44:0/3402200685 wait complete. 2024-09-13T17:35:31.051 INFO:tasks.cephfs.filesystem:scrub status for tag:be8bee0c-b1e1-4319-8fcd-6f7d7fc58a64 - {'path': '/', 'tag': 'be8bee0c-b1e1-4319-8fcd-6f7d7fc58a64', 'options': 'recursive,force'} 2024-09-13T17:35:31.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:31 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:35:31.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:31 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:31.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:31 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:35:31.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:31 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:31.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:31 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:35:31.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:31 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:31.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:31 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:35:31.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:31 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:31.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:31 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:35:31.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:31 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:31.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:31 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:35:31.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:31 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:35:32.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:32 smithi044 ceph-mon[31175]: pgmap v273: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:32.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:32 smithi070 ceph-mon[34502]: pgmap v273: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:32.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:32 smithi100 ceph-mon[35055]: pgmap v273: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:33.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:33 smithi044 ceph-mon[31175]: pgmap v274: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:33 smithi070 ceph-mon[34502]: pgmap v274: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:33.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:33 smithi100 ceph-mon[35055]: pgmap v274: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:34.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:34 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:35:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:34 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:35:34.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:34 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:35:35.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:35 smithi044 ceph-mon[31175]: pgmap v275: 129 pgs: 129 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-13T17:35:35.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:35 smithi070 ceph-mon[34502]: pgmap v275: 129 pgs: 129 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-13T17:35:35.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:35 smithi100 ceph-mon[35055]: pgmap v275: 129 pgs: 129 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-13T17:35:37.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:37 smithi044 ceph-mon[31175]: pgmap v276: 129 pgs: 129 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-13T17:35:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:37 smithi070 ceph-mon[34502]: pgmap v276: 129 pgs: 129 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-13T17:35:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:37 smithi100 ceph-mon[35055]: pgmap v276: 129 pgs: 129 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-13T17:35:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 -- 172.21.15.44:0/672166151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c100860 msgr2=0x7fd46c1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 --2- 172.21.15.44:0/672166151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c100860 0x7fd46c1048a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd458009930 tx=0x7fd45802a420 comp rx=0 tx=0).stop 2024-09-13T17:35:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 -- 172.21.15.44:0/672166151 shutdown_connections 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 --2- 172.21.15.44:0/672166151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c105000 0x7fd46c10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 --2- 172.21.15.44:0/672166151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c100860 0x7fd46c1048a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 --2- 172.21.15.44:0/672166151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c0ffeb0 0x7fd46c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 -- 172.21.15.44:0/672166151 >> 172.21.15.44:0/672166151 conn(0x7fd46c0fb960 msgr2=0x7fd46c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 -- 172.21.15.44:0/672166151 shutdown_connections 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.520+0000 7fd47320d640 1 -- 172.21.15.44:0/672166151 wait complete. 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 Processor -- start 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 -- start start 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c0ffeb0 0x7fd46c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 0x7fd46c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c073230 0x7fd46c1a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1a48a0 con 0x7fd46c105000 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1a4a10 con 0x7fd46c073230 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd47320d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1a4bd0 con 0x7fd46c0ffeb0 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 0x7fd46c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 0x7fd46c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40384/0 (socket says 172.21.15.44:40384) 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd46bfff640 1 -- 172.21.15.44:0/2828072221 learned_addr learned my addr 172.21.15.44:0/2828072221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd470fa9640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c0ffeb0 0x7fd46c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.521+0000 7fd4717aa640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c073230 0x7fd46c1a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd46bfff640 1 -- 172.21.15.44:0/2828072221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c0ffeb0 msgr2=0x7fd46c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd46bfff640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c0ffeb0 0x7fd46c0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd46bfff640 1 -- 172.21.15.44:0/2828072221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c073230 msgr2=0x7fd46c1a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd46bfff640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c073230 0x7fd46c1a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd46bfff640 1 -- 172.21.15.44:0/2828072221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458009560 con 0x7fd46c105000 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd4717aa640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c073230 0x7fd46c1a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd46bfff640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 0x7fd46c072cf0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd458005ab0 tx=0x7fd458032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd469ffb640 1 -- 172.21.15.44:0/2828072221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458032e10 con 0x7fd46c105000 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd469ffb640 1 -- 172.21.15.44:0/2828072221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4580046c0 con 0x7fd46c105000 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c1a4e50 con 0x7fd46c105000 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd469ffb640 1 -- 172.21.15.44:0/2828072221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458003830 con 0x7fd46c105000 2024-09-13T17:35:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.522+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c1a5280 con 0x7fd46c105000 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd469ffb640 1 -- 172.21.15.44:0/2828072221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd458004830 con 0x7fd46c105000 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 Processor -- start 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 -- start start 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0330 0x556ac1ce0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1ceafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ced8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1c4bc30 con 0x556ac1ceb4f0 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1c493c0 con 0x556ac1ce0c50 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd476009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1c44cb0 con 0x556ac1ce0330 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.525+0000 7fd469ffb640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd45c0a12b0 0x7fd45c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ced8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd4717aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0330 0x556ac1ce0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ced8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40386/0 (socket says 172.21.15.44:40386) 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 -- 172.21.15.44:0/691835870 learned_addr learned my addr 172.21.15.44:0/691835870 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd470fa9640 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1ceafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd4717aa640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd45c0a12b0 0x7fd45c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd469ffb640 1 -- 172.21.15.44:0/2828072221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd4580edbe0 con 0x7fd46c105000 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 -- 172.21.15.44:0/691835870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0330 msgr2=0x556ac1ce0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0330 0x556ac1ce0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 -- 172.21.15.44:0/691835870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 msgr2=0x556ac1ceafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1ceafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 -- 172.21.15.44:0/691835870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ac1beb910 con 0x556ac1ceb4f0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd4717aa640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd45c0a12b0 0x7fd45c0a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd464009fd0 tx=0x7fd464005bd0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd46bfff640 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ced8e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd4580eaea0 tx=0x7fd45804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fbd9e6f3514798 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd456ffd640 1 -- 172.21.15.44:0/691835870 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580ba920 con 0x556ac1ceb4f0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.526+0000 7fd456ffd640 1 -- 172.21.15.44:0/691835870 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4580be750 con 0x556ac1ceb4f0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd456ffd640 1 -- 172.21.15.44:0/691835870 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580eb1e0 con 0x556ac1ceb4f0 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 -- 172.21.15.44:0/691835870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 msgr2=0x556ac1ced8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ced8e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd4580eaea0 tx=0x7fd45804c000 comp rx=0 tx=0).stop 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 -- 172.21.15.44:0/691835870 shutdown_connections 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ced8e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1ceafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 --2- 172.21.15.44:0/691835870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0330 0x556ac1ce0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 -- 172.21.15.44:0/691835870 >> 172.21.15.44:0/691835870 conn(0x556ac1cdc8c0 msgr2=0x556ac1cdcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 -- 172.21.15.44:0/691835870 shutdown_connections 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.527+0000 7fd476009740 1 -- 172.21.15.44:0/691835870 wait complete. 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 -- start start 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 Processor -- start 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1ceadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1ce8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1c02e80 con 0x556ac1ce0330 2024-09-13T17:35:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1c02b20 con 0x556ac1ceb4f0 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1bc8260 con 0x556ac1ce0c50 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd476009740 10 client.0 _ll_register_callbacks cb 0x556ac1bfddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd470fa9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1ceadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd470fa9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1ceadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40394/0 (socket says 172.21.15.44:40394) 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd470fa9640 1 -- 172.21.15.44:0/1983843658 learned_addr learned my addr 172.21.15.44:0/1983843658 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd46bfff640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1ce8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.528+0000 7fd4717aa640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd470fa9640 1 -- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 msgr2=0x556ac1ce8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd470fa9640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1ce8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd470fa9640 1 -- 172.21.15.44:0/1983843658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ceb4f0 msgr2=0x556ac1ce8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd470fa9640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd470fa9640 1 -- 172.21.15.44:0/1983843658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c1a5280 con 0x556ac1ce0330 2024-09-13T17:35:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd4717aa640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd470fa9640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1ceadd0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fd460010a50 tx=0x7fd460010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7fe3a75877db03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd44bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd44bfff640 1 -- 172.21.15.44:0/1983843658 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460015070 con 0x556ac1ce0330 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd44bfff640 1 -- 172.21.15.44:0/1983843658 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd46000fb50 con 0x556ac1ce0330 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ac1c6d680 con 0x556ac1ce0330 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd44bfff640 1 -- 172.21.15.44:0/1983843658 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd46000fd10 con 0x556ac1ce0330 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd476009740 10 client.18471 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ac1c43330 con 0x556ac1ce0330 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.529+0000 7fd44bfff640 1 -- 172.21.15.44:0/1983843658 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ac1c6d680 con 0x556ac1ce0330 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd44bfff640 1 -- 172.21.15.44:0/1983843658 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd460018370 con 0x556ac1ce0330 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd476009740 10 client.18471 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd476009740 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd476009740 20 client.18471 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd476009740 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1cfabe0 0x556ac1cfd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd476009740 4 client.18471 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ac1c385d0 con 0x556ac1cfabe0 2024-09-13T17:35:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd46bfff640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1cfabe0 0x556ac1cfd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd46bfff640 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1cfabe0 0x556ac1cfd000 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e55145eee3c976 server_cookie=30163ff4c9299072 in_seq=0 out_seq=0 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.530+0000 7fd44bfff640 10 client.18471 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd44bfff640 1 -- 172.21.15.44:0/1983843658 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556ac1c385d0 con 0x556ac1cfabe0 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd44bfff640 10 client.18471 handle_command_reply: tid=0 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 client.18471 shutdown 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 msgr2=0x556ac1ceadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1ceadd0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fd460010a50 tx=0x7fd460010f10 comp rx=0 tx=0).stop 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 shutdown_connections 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1cfabe0 0x556ac1cfd000 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1ce8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd44bfff640 0 client.18471 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 --2- 172.21.15.44:0/1983843658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1ceadd0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 >> 172.21.15.44:0/1983843658 conn(0x556ac1cdc8c0 msgr2=0x556ac1cdbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 shutdown_connections 2024-09-13T17:35:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.531+0000 7fd476009740 1 -- 172.21.15.44:0/1983843658 wait complete. 2024-09-13T17:35:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.532+0000 7fd476009740 20 client.18471 trim_cache size 0 max 16384 2024-09-13T17:35:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 Processor -- start 2024-09-13T17:35:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 -- start start 2024-09-13T17:35:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0330 0x556ac1cd8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1cea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1c6d6b0 con 0x556ac1ceb4f0 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1cf94d0 con 0x556ac1ce0330 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd476009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1ce9380 con 0x556ac1ce0c50 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd4717aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd4717aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40404/0 (socket says 172.21.15.44:40404) 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.544+0000 7fd4717aa640 1 -- 172.21.15.44:0/3981502064 learned_addr learned my addr 172.21.15.44:0/3981502064 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd470fa9640 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0330 0x556ac1cd8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd46bfff640 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1cea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4717aa640 1 -- 172.21.15.44:0/3981502064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 msgr2=0x556ac1cea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4717aa640 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1cea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4717aa640 1 -- 172.21.15.44:0/3981502064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0330 msgr2=0x556ac1cd8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4717aa640 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0330 0x556ac1cd8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4717aa640 1 -- 172.21.15.44:0/3981502064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ac1be5300 con 0x556ac1ceb4f0 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd470fa9640 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0330 0x556ac1cd8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4717aa640 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce5240 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd464012a70 tx=0x7fd4640096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4360301bb46634 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4577fe640 1 -- 172.21.15.44:0/3981502064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464004b20 con 0x556ac1ceb4f0 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4577fe640 1 -- 172.21.15.44:0/3981502064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd464021e00 con 0x556ac1ceb4f0 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.545+0000 7fd4577fe640 1 -- 172.21.15.44:0/3981502064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640054f0 con 0x556ac1ceb4f0 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 -- 172.21.15.44:0/3981502064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 msgr2=0x556ac1ce5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce5240 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd464012a70 tx=0x7fd4640096f0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 -- 172.21.15.44:0/3981502064 shutdown_connections 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ce0c50 0x556ac1cea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ceb4f0 0x556ac1ce5240 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 --2- 172.21.15.44:0/3981502064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0330 0x556ac1cd8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 -- 172.21.15.44:0/3981502064 >> 172.21.15.44:0/3981502064 conn(0x556ac1cdc8c0 msgr2=0x556ac1cd7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 -- 172.21.15.44:0/3981502064 shutdown_connections 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 -- 172.21.15.44:0/3981502064 wait complete. 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 -- start start 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.546+0000 7fd476009740 1 Processor -- start 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1cf0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1cf1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ceb4f0 0x556ac1cf1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1d512a0 con 0x556ac1ce0330 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1d51410 con 0x556ac1ce0c50 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac1d515d0 con 0x556ac1ceb4f0 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd476009740 10 client.0 _ll_register_callbacks cb 0x7fd46000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1cf0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1cf0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40416/0 (socket says 172.21.15.44:40416) 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 -- 172.21.15.44:0/270565484 learned_addr learned my addr 172.21.15.44:0/270565484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd4717aa640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ceb4f0 0x556ac1cf1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd46bfff640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1cf1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 -- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ceb4f0 msgr2=0x556ac1cf1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ceb4f0 0x556ac1cf1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 -- 172.21.15.44:0/270565484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 msgr2=0x556ac1cf1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1cf1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 -- 172.21.15.44:0/270565484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46401a040 con 0x556ac1ce0330 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd46bfff640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1cf1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd470fa9640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1cf0d10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd460018780 tx=0x7fd4600185a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b953ad7364a19f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.547+0000 7fd44bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd44bfff640 1 -- 172.21.15.44:0/270565484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460015040 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd44bfff640 1 -- 172.21.15.44:0/270565484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd46000a340 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ac1ce1890 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd44bfff640 1 -- 172.21.15.44:0/270565484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd460010640 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 10 client.18477 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ac1c0a8e0 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd44bfff640 1 -- 172.21.15.44:0/270565484 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ac1ce1890 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd44bfff640 1 -- 172.21.15.44:0/270565484 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd460021040 con 0x556ac1ce0330 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 10 client.18477 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 20 client.18477 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1d52d40 0x556ac1d55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 4 client.18477 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:35:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fd4580be750 con 0x556ac1d52d40 2024-09-13T17:35:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.548+0000 7fd46bfff640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1d52d40 0x556ac1d55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.549+0000 7fd46bfff640 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1d52d40 0x556ac1d55130 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d1b0ecb028d2c8 server_cookie=d3053037272f7dd7 in_seq=0 out_seq=0 2024-09-13T17:35:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.549+0000 7fd44bfff640 10 client.18477 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.561+0000 7fd44bfff640 1 -- 172.21.15.44:0/270565484 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fd4580be750 con 0x556ac1d52d40 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.561+0000 7fd44bfff640 10 client.18477 handle_command_reply: tid=0 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.561+0000 7fd476009740 1 client.18477 shutdown 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.561+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 msgr2=0x556ac1cf0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1cf0d10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd460018780 tx=0x7fd4600185a0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 shutdown_connections 2024-09-13T17:35:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556ac1d52d40 0x556ac1d55130 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ac1ceb4f0 0x556ac1cf1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556ac1ce0c50 0x556ac1cf1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 --2- 172.21.15.44:0/270565484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ac1ce0330 0x556ac1cf0d10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd44bfff640 0 client.18477 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 >> 172.21.15.44:0/270565484 conn(0x556ac1cdc8c0 msgr2=0x556ac1ce78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 shutdown_connections 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 1 -- 172.21.15.44:0/270565484 wait complete. 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd476009740 20 client.18477 trim_cache size 0 max 16384 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:35:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:35:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:35:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd45c0a12b0 msgr2=0x7fd45c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd45c0a12b0 0x7fd45c0a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd464009fd0 tx=0x7fd464005bd0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 msgr2=0x7fd46c072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 0x7fd46c072cf0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd458005ab0 tx=0x7fd458032c20 comp rx=0 tx=0).stop 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 shutdown_connections 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd45c0a12b0 0x7fd45c0a3770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd46c073230 0x7fd46c1a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.562+0000 7fd47320d640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd46c105000 0x7fd46c072cf0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.563+0000 7fd47320d640 1 --2- 172.21.15.44:0/2828072221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c0ffeb0 0x7fd46c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.563+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 >> 172.21.15.44:0/2828072221 conn(0x7fd46c0fb960 msgr2=0x7fd46c0fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.563+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 shutdown_connections 2024-09-13T17:35:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:38.563+0000 7fd47320d640 1 -- 172.21.15.44:0/2828072221 wait complete. 2024-09-13T17:35:39.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:39 smithi044 ceph-mon[31175]: pgmap v277: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 20 op/s 2024-09-13T17:35:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:39 smithi070 ceph-mon[34502]: pgmap v277: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 20 op/s 2024-09-13T17:35:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:39 smithi100 ceph-mon[35055]: pgmap v277: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 20 op/s 2024-09-13T17:35:41.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:41 smithi044 ceph-mon[31175]: pgmap v278: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-13T17:35:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:41 smithi070 ceph-mon[34502]: pgmap v278: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-13T17:35:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:41 smithi100 ceph-mon[35055]: pgmap v278: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-13T17:35:43.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:43 smithi044 ceph-mon[31175]: pgmap v279: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-13T17:35:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:43 smithi070 ceph-mon[34502]: pgmap v279: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-13T17:35:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:43 smithi100 ceph-mon[35055]: pgmap v279: 129 pgs: 129 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-13T17:35:45.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:45 smithi044 ceph-mon[31175]: pgmap v280: 129 pgs: 129 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-13T17:35:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:45 smithi070 ceph-mon[34502]: pgmap v280: 129 pgs: 129 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-13T17:35:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:45 smithi100 ceph-mon[35055]: pgmap v280: 129 pgs: 129 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-13T17:35:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:47 smithi044 ceph-mon[31175]: pgmap v281: 129 pgs: 129 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:47 smithi070 ceph-mon[34502]: pgmap v281: 129 pgs: 129 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:47 smithi100 ceph-mon[35055]: pgmap v281: 129 pgs: 129 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:49 smithi044 ceph-mon[31175]: pgmap v282: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:35:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:49 smithi070 ceph-mon[34502]: pgmap v282: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:35:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:49 smithi100 ceph-mon[35055]: pgmap v282: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:35:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 -- 172.21.15.44:0/3998551697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c104c30 msgr2=0x7f9a3c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/3998551697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c104c30 0x7f9a3c105010 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009930 tx=0x7f9a3802a420 comp rx=0 tx=0).stop 2024-09-13T17:35:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 -- 172.21.15.44:0/3998551697 shutdown_connections 2024-09-13T17:35:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/3998551697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c0feff0 0x7f9a3c0ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/3998551697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/3998551697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c104c30 0x7f9a3c105010 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 -- 172.21.15.44:0/3998551697 >> 172.21.15.44:0/3998551697 conn(0x7f9a3c074940 msgr2=0x7f9a3c074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 -- 172.21.15.44:0/3998551697 shutdown_connections 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.663+0000 7f9a43bb5640 1 -- 172.21.15.44:0/3998551697 wait complete. 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 Processor -- start 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 -- start start 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c0feff0 0x7f9a3c195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a3c104c30 0x7f9a3c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c19a010 con 0x7f9a3c0ff970 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c19a180 con 0x7f9a3c104c30 2024-09-13T17:35:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a43bb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c19a340 con 0x7f9a3c0feff0 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56038/0 (socket says 172.21.15.44:56038) 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 -- 172.21.15.44:0/137068946 learned_addr learned my addr 172.21.15.44:0/137068946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a42152640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a3c104c30 0x7f9a3c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41951640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c0feff0 0x7f9a3c195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 -- 172.21.15.44:0/137068946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c0feff0 msgr2=0x7f9a3c195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c0feff0 0x7f9a3c195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 -- 172.21.15.44:0/137068946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a3c104c30 msgr2=0x7f9a3c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.664+0000 7f9a41150640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a3c104c30 0x7f9a3c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a41150640 1 -- 172.21.15.44:0/137068946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38009560 con 0x7f9a3c0ff970 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a41150640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c195670 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d770 tx=0x7f9a2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a32ffd640 1 -- 172.21.15.44:0/137068946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c009c30 con 0x7f9a3c0ff970 2024-09-13T17:35:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a32ffd640 1 -- 172.21.15.44:0/137068946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c00de80 con 0x7f9a3c0ff970 2024-09-13T17:35:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a32ffd640 1 -- 172.21.15.44:0/137068946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c004d40 con 0x7f9a3c0ff970 2024-09-13T17:35:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c19a620 con 0x7f9a3c0ff970 2024-09-13T17:35:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.665+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c1a1ea0 con 0x7f9a3c0ff970 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.667+0000 7f9a32ffd640 1 -- 172.21.15.44:0/137068946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9a2c004ea0 con 0x7f9a3c0ff970 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 Processor -- start 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 -- start start 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80330 0x557c20c80710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c8d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20bebc30 con 0x557c20c8b4f0 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20be93c0 con 0x557c20c80c50 2024-09-13T17:35:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.668+0000 7f9a468d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20be4cb0 con 0x557c20c80330 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c8d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c8d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56042/0 (socket says 172.21.15.44:56042) 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 -- 172.21.15.44:0/3120522414 learned_addr learned my addr 172.21.15.44:0/3120522414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a32ffd640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9a280a1280 0x7f9a280a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41951640 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a42152640 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80330 0x557c20c80710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a42152640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9a280a1280 0x7f9a280a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a32ffd640 1 -- 172.21.15.44:0/137068946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9a2c0cc9f0 con 0x7f9a3c0ff970 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 -- 172.21.15.44:0/3120522414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80330 msgr2=0x557c20c80710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80330 0x557c20c80710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 -- 172.21.15.44:0/3120522414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 msgr2=0x557c20c8afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c8afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 -- 172.21.15.44:0/3120522414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c20b8b910 con 0x557c20c8b4f0 2024-09-13T17:35:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a42152640 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80330 0x557c20c80710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.669+0000 7f9a41150640 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c8d8e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0994e0 tx=0x7f9a2c099140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1fd4a0cb02ec93 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a27fff640 1 -- 172.21.15.44:0/3120522414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c09d560 con 0x557c20c8b4f0 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a42152640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9a280a1280 0x7f9a280a3740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009990 tx=0x7f9a340096a0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a27fff640 1 -- 172.21.15.44:0/3120522414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c0a1bb0 con 0x557c20c8b4f0 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a27fff640 1 -- 172.21.15.44:0/3120522414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0c9ff0 con 0x557c20c8b4f0 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 -- 172.21.15.44:0/3120522414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 msgr2=0x557c20c8d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c8d8e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0994e0 tx=0x7f9a2c099140 comp rx=0 tx=0).stop 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 -- 172.21.15.44:0/3120522414 shutdown_connections 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c8d8e0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c8afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3120522414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80330 0x557c20c80710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 -- 172.21.15.44:0/3120522414 >> 172.21.15.44:0/3120522414 conn(0x557c20c7c8c0 msgr2=0x557c20c7cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 -- 172.21.15.44:0/3120522414 shutdown_connections 2024-09-13T17:35:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.670+0000 7f9a468d7740 1 -- 172.21.15.44:0/3120522414 wait complete. 2024-09-13T17:35:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 -- start start 2024-09-13T17:35:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 Processor -- start 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c8a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c8aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c91900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20ba2e80 con 0x557c20c8b4f0 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20ba2b20 con 0x557c20c80330 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20b68260 con 0x557c20c80c50 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a468d7740 10 client.0 _ll_register_callbacks cb 0x557c20b9ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.671+0000 7f9a42152640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c91900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c91900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56046/0 (socket says 172.21.15.44:56046) 2024-09-13T17:35:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 -- 172.21.15.44:0/1531613333 learned_addr learned my addr 172.21.15.44:0/1531613333 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a41951640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c8a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a41150640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c8aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 -- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 msgr2=0x557c20c8aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c8aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 -- 172.21.15.44:0/1531613333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 msgr2=0x557c20c8a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c8a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 -- 172.21.15.44:0/1531613333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c1a1ea0 con 0x557c20c8b4f0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a41951640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c8a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a42152640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c91900 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9a34003df0 tx=0x7f9a34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411b8f97ec7351ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a257fa640 1 -- 172.21.15.44:0/1531613333 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34018070 con 0x557c20c8b4f0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a257fa640 1 -- 172.21.15.44:0/1531613333 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9a3400d990 con 0x557c20c8b4f0 2024-09-13T17:35:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.672+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c20c0d680 con 0x557c20c8b4f0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a257fa640 1 -- 172.21.15.44:0/1531613333 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a3400db50 con 0x557c20c8b4f0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 10 client.18489 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c20be3330 con 0x557c20c8b4f0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a257fa640 1 -- 172.21.15.44:0/1531613333 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c20c0d680 con 0x557c20c8b4f0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a257fa640 1 -- 172.21.15.44:0/1531613333 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9a3401e050 con 0x557c20c8b4f0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 10 client.18489 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 20 client.18489 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20c9d0f0 0x557c20c9f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 4 client.18489 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c20bd85d0 con 0x557c20c9d0f0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.673+0000 7f9a41951640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20c9d0f0 0x557c20c9f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.674+0000 7f9a41951640 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20c9d0f0 0x557c20c9f4e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3befc14906b678 server_cookie=3624e568880f848b in_seq=0 out_seq=0 2024-09-13T17:35:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.674+0000 7f9a257fa640 10 client.18489 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.674+0000 7f9a257fa640 1 -- 172.21.15.44:0/1531613333 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557c20bd85d0 con 0x557c20c9d0f0 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.674+0000 7f9a257fa640 10 client.18489 handle_command_reply: tid=0 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.674+0000 7f9a468d7740 1 client.18489 shutdown 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 msgr2=0x557c20c91900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c91900 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9a34003df0 tx=0x7f9a34004a40 comp rx=0 tx=0).stop 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 shutdown_connections 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20c9d0f0 0x557c20c9f4e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c8aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c91900 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 --2- 172.21.15.44:0/1531613333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c8a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 >> 172.21.15.44:0/1531613333 conn(0x557c20c7c8c0 msgr2=0x557c20c7bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a257fa640 0 client.18489 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 shutdown_connections 2024-09-13T17:35:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 1 -- 172.21.15.44:0/1531613333 wait complete. 2024-09-13T17:35:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.675+0000 7f9a468d7740 20 client.18489 trim_cache size 0 max 16384 2024-09-13T17:35:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.687+0000 7f9a468d7740 1 Processor -- start 2024-09-13T17:35:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 -- start start 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 0x557c20c7c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c8b4f0 0x557c20c8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20c91e30 con 0x557c20c80330 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20c99db0 con 0x557c20c80c50 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a468d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20c812d0 con 0x557c20c8b4f0 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 0x557c20c7c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 0x557c20c7c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56052/0 (socket says 172.21.15.44:56052) 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 -- 172.21.15.44:0/3014252076 learned_addr learned my addr 172.21.15.44:0/3014252076 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41150640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a42152640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c8b4f0 0x557c20c8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 -- 172.21.15.44:0/3014252076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c8b4f0 msgr2=0x557c20c8a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c8b4f0 0x557c20c8a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 -- 172.21.15.44:0/3014252076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 msgr2=0x557c20c918f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c918f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 -- 172.21.15.44:0/3014252076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c20b85300 con 0x557c20c80330 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41150640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a42152640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c8b4f0 0x557c20c8a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.688+0000 7f9a41951640 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 0x557c20c7c300 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009930 tx=0x7f9a38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad183d217074d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a30e05640 1 -- 172.21.15.44:0/3014252076 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38043040 con 0x557c20c80330 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a30e05640 1 -- 172.21.15.44:0/3014252076 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a38045e30 con 0x557c20c80330 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a30e05640 1 -- 172.21.15.44:0/3014252076 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38003d20 con 0x557c20c80330 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 -- 172.21.15.44:0/3014252076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 msgr2=0x557c20c7c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 0x557c20c7c300 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009930 tx=0x7f9a38000bc0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 -- 172.21.15.44:0/3014252076 shutdown_connections 2024-09-13T17:35:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c8b4f0 0x557c20c8a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80c50 0x557c20c918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 --2- 172.21.15.44:0/3014252076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c80330 0x557c20c7c300 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 -- 172.21.15.44:0/3014252076 >> 172.21.15.44:0/3014252076 conn(0x557c20c7c8c0 msgr2=0x557c20c869f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 -- 172.21.15.44:0/3014252076 shutdown_connections 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 -- 172.21.15.44:0/3014252076 wait complete. 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.689+0000 7f9a468d7740 1 -- start start 2024-09-13T17:35:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 Processor -- start 2024-09-13T17:35:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c94d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20c95440 con 0x557c20c8b4f0 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20cf35b0 con 0x557c20c80330 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20cf36f0 con 0x557c20c80c50 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a468d7740 10 client.0 _ll_register_callbacks cb 0x7f9a2c095ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a42152640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c94d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a42152640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c94d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56066/0 (socket says 172.21.15.44:56066) 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a42152640 1 -- 172.21.15.44:0/2090237385 learned_addr learned my addr 172.21.15.44:0/2090237385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a41951640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.690+0000 7f9a41150640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a42152640 1 -- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 msgr2=0x557c20c945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a42152640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c945b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a42152640 1 -- 172.21.15.44:0/2090237385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 msgr2=0x557c20c997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:35:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a42152640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a42152640 1 -- 172.21.15.44:0/2090237385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c10c340 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a42152640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c94d40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009960 tx=0x7f9a340125e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e764a83f676261 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a257fa640 1 -- 172.21.15.44:0/2090237385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34018050 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a257fa640 1 -- 172.21.15.44:0/2090237385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9a34021490 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a2c0a1bb0 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a257fa640 1 -- 172.21.15.44:0/2090237385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a340048a0 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a468d7740 10 client.18495 fetch_fsmap learned FSMap version 25 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c20baa8e0 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a257fa640 1 -- 172.21.15.44:0/2090237385 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c0a1bb0 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.691+0000 7f9a257fa640 1 -- 172.21.15.44:0/2090237385 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9a3401e040 con 0x557c20c8b4f0 2024-09-13T17:35:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a468d7740 10 client.18495 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a468d7740 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a468d7740 20 client.18495 populate_metadata read hostname 'smithi044' 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a468d7740 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20cf4d80 0x557c20cf71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a468d7740 4 client.18495 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c208d8300 con 0x557c20cf4d80 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a41951640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20cf4d80 0x557c20cf71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a41951640 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20cf4d80 0x557c20cf71a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25e35cae4c583e5 server_cookie=6ac50bafb1f1e4fe in_seq=0 out_seq=0 2024-09-13T17:35:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.692+0000 7f9a257fa640 10 client.18495 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a257fa640 1 -- 172.21.15.44:0/2090237385 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c208d8300 con 0x557c20cf4d80 2024-09-13T17:35:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a257fa640 10 client.18495 handle_command_reply: tid=0 2024-09-13T17:35:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 client.18495 shutdown 2024-09-13T17:35:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 msgr2=0x557c20c94d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c94d40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009960 tx=0x7f9a340125e0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 shutdown_connections 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x557c20cf4d80 0x557c20cf71a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c20c80c50 0x557c20c945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c20c8b4f0 0x557c20c94d40 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a257fa640 0 client.18495 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 --2- 172.21.15.44:0/2090237385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c20c80330 0x557c20c997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 >> 172.21.15.44:0/2090237385 conn(0x557c20c7c8c0 msgr2=0x557c20c8a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 shutdown_connections 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.706+0000 7f9a468d7740 1 -- 172.21.15.44:0/2090237385 wait complete. 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a468d7740 20 client.18495 trim_cache size 0 max 16384 2024-09-13T17:35:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:35:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:35:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:35:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9a280a1280 msgr2=0x7f9a280a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9a280a1280 0x7f9a280a3740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009990 tx=0x7f9a340096a0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 msgr2=0x7f9a3c195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c195670 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d770 tx=0x7f9a2c00dc30 comp rx=0 tx=0).stop 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 shutdown_connections 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9a280a1280 0x7f9a280a3740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a3c104c30 0x7f9a3c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c0ff970 0x7f9a3c195670 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 --2- 172.21.15.44:0/137068946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c0feff0 0x7f9a3c195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 >> 172.21.15.44:0/137068946 conn(0x7f9a3c074940 msgr2=0x7f9a3c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 shutdown_connections 2024-09-13T17:35:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:35:51.707+0000 7f9a43bb5640 1 -- 172.21.15.44:0/137068946 wait complete. 2024-09-13T17:35:51.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:51 smithi044 ceph-mon[31175]: pgmap v283: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:35:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:51 smithi070 ceph-mon[34502]: pgmap v283: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:35:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:51 smithi100 ceph-mon[35055]: pgmap v283: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:35:53.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:53 smithi044 ceph-mon[31175]: pgmap v284: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:35:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:53 smithi070 ceph-mon[34502]: pgmap v284: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:35:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:53 smithi100 ceph-mon[35055]: pgmap v284: 129 pgs: 129 active+clean; 4.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:35:55.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:55 smithi044 ceph-mon[31175]: pgmap v285: 129 pgs: 129 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-09-13T17:35:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:55 smithi070 ceph-mon[34502]: pgmap v285: 129 pgs: 129 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-09-13T17:35:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:55 smithi100 ceph-mon[35055]: pgmap v285: 129 pgs: 129 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-09-13T17:35:56.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:56 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:35:56.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:56 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:35:56.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:56 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:35:56.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:56 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:35:56.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:56 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:35:56.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:56 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:35:56.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:56 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:35:56.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:56 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:35:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:56 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:35:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:56 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:35:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:56 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:35:57.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:56 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:35:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:35:58 smithi044 ceph-mon[31175]: pgmap v286: 129 pgs: 129 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:35:58 smithi070 ceph-mon[34502]: pgmap v286: 129 pgs: 129 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:35:58.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:35:58 smithi100 ceph-mon[35055]: pgmap v286: 129 pgs: 129 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:00.576 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:00 smithi100 ceph-mon[35055]: pgmap v287: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:36:00.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:00 smithi044 ceph-mon[31175]: pgmap v287: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:36:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:00 smithi070 ceph-mon[34502]: pgmap v287: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:36:01.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:36:01.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 -- 172.21.15.44:0/926476528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f968010a330 msgr2=0x7f968010c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 --2- 172.21.15.44:0/926476528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f968010a330 0x7f968010c7b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9678009ca0 tx=0x7f967802a950 comp rx=0 tx=0).stop 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 -- 172.21.15.44:0/926476528 shutdown_connections 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 --2- 172.21.15.44:0/926476528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101880 0x7f9680109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 --2- 172.21.15.44:0/926476528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f968010a330 0x7f968010c7b0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 --2- 172.21.15.44:0/926476528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9680100f60 0x7f9680101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 -- 172.21.15.44:0/926476528 >> 172.21.15.44:0/926476528 conn(0x7f96800fa8d0 msgr2=0x7f96800fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 -- 172.21.15.44:0/926476528 shutdown_connections 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.169+0000 7f968869c640 1 -- 172.21.15.44:0/926476528 wait complete. 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 Processor -- start 2024-09-13T17:36:01.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 -- start start 2024-09-13T17:36:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 0x7f9680196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101880 0x7f9680196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968010a330 0x7f968019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019b050 con 0x7f9680100f60 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019b1c0 con 0x7f968010a330 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f968869c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019b380 con 0x7f9680101880 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 0x7f9680196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 0x7f9680196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40400/0 (socket says 172.21.15.44:40400) 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 -- 172.21.15.44:0/1910235027 learned_addr learned my addr 172.21.15.44:0/1910235027 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9685c37640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101880 0x7f9680196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686c39640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968010a330 0x7f968019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 -- 172.21.15.44:0/1910235027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101880 msgr2=0x7f9680196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101880 0x7f9680196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 -- 172.21.15.44:0/1910235027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968010a330 msgr2=0x7f968019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968010a330 0x7f968019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.170+0000 7f9686438640 1 -- 172.21.15.44:0/1910235027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009900 con 0x7f9680100f60 2024-09-13T17:36:01.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.171+0000 7f9686438640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 0x7f9680196120 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f967c00d950 tx=0x7f967c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.171+0000 7f96777fe640 1 -- 172.21.15.44:0/1910235027 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c009c30 con 0x7f9680100f60 2024-09-13T17:36:01.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.171+0000 7f96777fe640 1 -- 172.21.15.44:0/1910235027 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f967c009d90 con 0x7f9680100f60 2024-09-13T17:36:01.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.171+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968019b660 con 0x7f9680100f60 2024-09-13T17:36:01.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.171+0000 7f96777fe640 1 -- 172.21.15.44:0/1910235027 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00f620 con 0x7f9680100f60 2024-09-13T17:36:01.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.171+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96801a2ee0 con 0x7f9680100f60 2024-09-13T17:36:01.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.173+0000 7f96777fe640 1 -- 172.21.15.44:0/1910235027 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f967c0109f0 con 0x7f9680100f60 2024-09-13T17:36:01.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.173+0000 7f968b3c1740 1 Processor -- start 2024-09-13T17:36:01.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 -- start start 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea61531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea615dfd0 0x55dea61603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea60bdf20 con 0x55dea6152e10 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea60bb6b0 con 0x55dea6153730 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f968b3c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea60b6fa0 con 0x55dea615dfd0 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f9686c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea61531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f9686438640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f9686c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea61531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40414/0 (socket says 172.21.15.44:40414) 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f9686c39640 1 -- 172.21.15.44:0/2348019004 learned_addr learned my addr 172.21.15.44:0/2348019004 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f9685c37640 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea615dfd0 0x55dea61603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.174+0000 7f96777fe640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f966c0a1280 0x7f966c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 -- 172.21.15.44:0/2348019004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea615dfd0 msgr2=0x55dea61603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea615dfd0 0x55dea61603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 -- 172.21.15.44:0/2348019004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 msgr2=0x55dea615da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 -- 172.21.15.44:0/2348019004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dea6071840 con 0x55dea6152e10 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9685c37640 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea615dfd0 0x55dea61603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686438640 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f96777fe640 1 -- 172.21.15.44:0/1910235027 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f967c0ccc00 con 0x7f9680100f60 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f966c0a1280 0x7f966c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9686c39640 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea61531f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9678005e30 tx=0x7f967802ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0613116183a2aa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9674ff9640 1 -- 172.21.15.44:0/2348019004 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678003c30 con 0x55dea6152e10 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9674ff9640 1 -- 172.21.15.44:0/2348019004 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9678040070 con 0x55dea6152e10 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.175+0000 7f9674ff9640 1 -- 172.21.15.44:0/2348019004 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780042b0 con 0x55dea6152e10 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 -- 172.21.15.44:0/2348019004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 msgr2=0x55dea61531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f9686c39640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f966c0a1280 0x7f966c0a3740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9678004af0 tx=0x7f967804b000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea61531f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9678005e30 tx=0x7f967802ff30 comp rx=0 tx=0).stop 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 -- 172.21.15.44:0/2348019004 shutdown_connections 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea615dfd0 0x55dea61603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 --2- 172.21.15.44:0/2348019004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea61531f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 -- 172.21.15.44:0/2348019004 >> 172.21.15.44:0/2348019004 conn(0x55dea614f380 msgr2=0x55dea614f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 -- 172.21.15.44:0/2348019004 shutdown_connections 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 -- 172.21.15.44:0/2348019004 wait complete. 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 -- start start 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.176+0000 7f968b3c1740 1 Processor -- start 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6152e10 0x55dea615d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea615b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea60582a0 con 0x55dea615dfd0 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5d8f430 con 0x55dea6153730 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea603a680 con 0x55dea6152e10 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f968b3c1740 10 client.0 _ll_register_callbacks cb 0x55dea60b0610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea615b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea615b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40430/0 (socket says 172.21.15.44:40430) 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 -- 172.21.15.44:0/1606841233 learned_addr learned my addr 172.21.15.44:0/1606841233 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686438640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6152e10 0x55dea615d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9685c37640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 -- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6152e10 msgr2=0x55dea615d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6152e10 0x55dea615d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 -- 172.21.15.44:0/1606841233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 msgr2=0x55dea615ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 -- 172.21.15.44:0/1606841233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968010d430 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9686c39640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea615b1a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9678030040 tx=0x7f9678030b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69860f499a94bc3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.177+0000 7f9669ffb640 1 -- 172.21.15.44:0/1606841233 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678058070 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f9669ffb640 1 -- 172.21.15.44:0/1606841233 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f967802f600 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dea5daa4e0 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f9669ffb640 1 -- 172.21.15.44:0/1606841233 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96780044d0 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 10 client.18507 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dea60b5620 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f9669ffb640 1 -- 172.21.15.44:0/1606841233 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dea5daa4e0 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f9669ffb640 1 -- 172.21.15.44:0/1606841233 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9678050020 con 0x55dea615dfd0 2024-09-13T17:36:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 10 client.18507 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 20 client.18507 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea616db20 0x55dea616ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 4 client.18507 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dea5dbb7e0 con 0x55dea616db20 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.178+0000 7f9686438640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea616db20 0x55dea616ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.179+0000 7f9686438640 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea616db20 0x55dea616ff40 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ee4bd595b3f57c server_cookie=91cd8087ce2f206a in_seq=0 out_seq=0 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.179+0000 7f9669ffb640 10 client.18507 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.179+0000 7f9669ffb640 1 -- 172.21.15.44:0/1606841233 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dea5dbb7e0 con 0x55dea616db20 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.179+0000 7f9669ffb640 10 client.18507 handle_command_reply: tid=0 2024-09-13T17:36:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.179+0000 7f968b3c1740 1 client.18507 shutdown 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 msgr2=0x55dea615b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea615b1a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9678030040 tx=0x7f9678030b90 comp rx=0 tx=0).stop 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 shutdown_connections 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea616db20 0x55dea616ff40 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea615b1a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f9669ffb640 0 client.18507 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6153730 0x55dea615ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 --2- 172.21.15.44:0/1606841233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6152e10 0x55dea615d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 >> 172.21.15.44:0/1606841233 conn(0x55dea614f380 msgr2=0x55dea61550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 shutdown_connections 2024-09-13T17:36:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 1 -- 172.21.15.44:0/1606841233 wait complete. 2024-09-13T17:36:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.180+0000 7f968b3c1740 20 client.18507 trim_cache size 0 max 16384 2024-09-13T17:36:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 Processor -- start 2024-09-13T17:36:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 -- start start 2024-09-13T17:36:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea614ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6164d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea615dfd0 0x55dea6165270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5dbb7e0 con 0x55dea6152e10 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5daa4e0 con 0x55dea615dfd0 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f968b3c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea615c040 con 0x55dea6153730 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f9686438640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea614ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f9686438640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea614ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40434/0 (socket says 172.21.15.44:40434) 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.194+0000 7f9686438640 1 -- 172.21.15.44:0/3435628620 learned_addr learned my addr 172.21.15.44:0/3435628620 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9685c37640 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686c39640 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea615dfd0 0x55dea6165270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686438640 1 -- 172.21.15.44:0/3435628620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 msgr2=0x55dea6164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686438640 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686438640 1 -- 172.21.15.44:0/3435628620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea615dfd0 msgr2=0x55dea6165270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686438640 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea615dfd0 0x55dea6165270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686438640 1 -- 172.21.15.44:0/3435628620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dea6057790 con 0x55dea6152e10 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686c39640 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea615dfd0 0x55dea6165270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f9686438640 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea614ed00 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f967c009610 tx=0x7f967c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a6c8a564e5166a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f96757fa640 1 -- 172.21.15.44:0/3435628620 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c099300 con 0x55dea6152e10 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f96757fa640 1 -- 172.21.15.44:0/3435628620 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f967c09a0a0 con 0x55dea6152e10 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f96757fa640 1 -- 172.21.15.44:0/3435628620 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0960b0 con 0x55dea6152e10 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 -- 172.21.15.44:0/3435628620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 msgr2=0x55dea614ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea614ed00 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f967c009610 tx=0x7f967c0ce040 comp rx=0 tx=0).stop 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 -- 172.21.15.44:0/3435628620 shutdown_connections 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea615dfd0 0x55dea6165270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 --2- 172.21.15.44:0/3435628620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea6152e10 0x55dea614ed00 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 -- 172.21.15.44:0/3435628620 >> 172.21.15.44:0/3435628620 conn(0x55dea614f380 msgr2=0x55dea6155ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.195+0000 7f968b3c1740 1 -- 172.21.15.44:0/3435628620 shutdown_connections 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 -- 172.21.15.44:0/3435628620 wait complete. 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 -- start start 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 Processor -- start 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6152e10 0x55dea6157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea61c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea6158890 con 0x55dea615dfd0 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea6158a00 con 0x55dea6152e10 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea61c4560 con 0x55dea6153730 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f968b3c1740 10 client.0 _ll_register_callbacks cb 0x7f967c0d05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f9686c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea61c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f9686c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea61c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40444/0 (socket says 172.21.15.44:40444) 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f9686c39640 1 -- 172.21.15.44:0/719742220 learned_addr learned my addr 172.21.15.44:0/719742220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f9686438640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6152e10 0x55dea6157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.196+0000 7f9685c37640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9686c39640 1 -- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 msgr2=0x55dea6158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9686c39640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9686c39640 1 -- 172.21.15.44:0/719742220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6152e10 msgr2=0x55dea6157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9686c39640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6152e10 0x55dea6157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9686c39640 1 -- 172.21.15.44:0/719742220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968019b660 con 0x55dea615dfd0 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9685c37640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9686c39640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea61c4020 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f967802a990 tx=0x7f9678005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca5a55e0b814635 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9669ffb640 1 -- 172.21.15.44:0/719742220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678058060 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9669ffb640 1 -- 172.21.15.44:0/719742220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f967802f600 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9678040070 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9669ffb640 1 -- 172.21.15.44:0/719742220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9678039820 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f968b3c1740 10 client.18513 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dea607cea0 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9669ffb640 1 -- 172.21.15.44:0/719742220 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9678070020 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f9669ffb640 1 -- 172.21.15.44:0/719742220 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9678064070 con 0x55dea615dfd0 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f968b3c1740 10 client.18513 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.197+0000 7f968b3c1740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 20 client.18513 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea61c5850 0x55dea61c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 4 client.18513 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:36:01.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dea5e0ee30 con 0x55dea61c5850 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f9686438640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea61c5850 0x55dea61c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f9669ffb640 10 client.18513 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f9686438640 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea61c5850 0x55dea61c7c40 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d361181d496b7a57 server_cookie=4b7389576eaecdc9 in_seq=0 out_seq=0 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f9669ffb640 1 -- 172.21.15.44:0/719742220 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dea5e0ee30 con 0x55dea61c5850 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f9669ffb640 10 client.18513 handle_command_reply: tid=0 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 client.18513 shutdown 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 msgr2=0x55dea61c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea61c4020 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f967802a990 tx=0x7f9678005da0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 shutdown_connections 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55dea61c5850 0x55dea61c7c40 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dea6153730 0x55dea6158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dea615dfd0 0x55dea61c4020 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f9669ffb640 0 client.18513 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 --2- 172.21.15.44:0/719742220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55dea6152e10 0x55dea6157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.198+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 >> 172.21.15.44:0/719742220 conn(0x55dea614f380 msgr2=0x55dea61594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 shutdown_connections 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968b3c1740 1 -- 172.21.15.44:0/719742220 wait complete. 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968b3c1740 20 client.18513 trim_cache size 0 max 16384 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:36:01.210 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f966c0a1280 msgr2=0x7f966c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f966c0a1280 0x7f966c0a3740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9678004af0 tx=0x7f967804b000 comp rx=0 tx=0).stop 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 msgr2=0x7f9680196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 0x7f9680196120 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f967c00d950 tx=0x7f967c00de10 comp rx=0 tx=0).stop 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 shutdown_connections 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f966c0a1280 0x7f966c0a3740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9680101880 0x7f9680196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968010a330 0x7f968019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 --2- 172.21.15.44:0/1910235027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9680100f60 0x7f9680196120 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 >> 172.21.15.44:0/1910235027 conn(0x7f96800fa8d0 msgr2=0x7f96800fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 shutdown_connections 2024-09-13T17:36:01.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.199+0000 7f968869c640 1 -- 172.21.15.44:0/1910235027 wait complete. 2024-09-13T17:36:01.213 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:36:01.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:36:01.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 -- 172.21.15.44:0/3702970721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b40ff860 msgr2=0x7fc6b41038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 --2- 172.21.15.44:0/3702970721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b40ff860 0x7fc6b41038a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009930 tx=0x7fc6a802a420 comp rx=0 tx=0).stop 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 -- 172.21.15.44:0/3702970721 shutdown_connections 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 --2- 172.21.15.44:0/3702970721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b4104000 0x7fc6b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 --2- 172.21.15.44:0/3702970721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b40ff860 0x7fc6b41038a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 --2- 172.21.15.44:0/3702970721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6b40feeb0 0x7fc6b40ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 -- 172.21.15.44:0/3702970721 >> 172.21.15.44:0/3702970721 conn(0x7fc6b40fa900 msgr2=0x7fc6b40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 -- 172.21.15.44:0/3702970721 shutdown_connections 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.314+0000 7fc6b988f640 1 -- 172.21.15.44:0/3702970721 wait complete. 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 Processor -- start 2024-09-13T17:36:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 -- start start 2024-09-13T17:36:01.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b40feeb0 0x7fc6b4196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 0x7fc6b41967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6b419aba0 0x7fc6b419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b419b590 con 0x7fc6b4104000 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b419b700 con 0x7fc6b419aba0 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b988f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b419b870 con 0x7fc6b40feeb0 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 0x7fc6b41967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 0x7fc6b41967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40458/0 (socket says 172.21.15.44:40458) 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b27fc640 1 -- 172.21.15.44:0/4258847773 learned_addr learned my addr 172.21.15.44:0/4258847773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b37fe640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6b419aba0 0x7fc6b419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b2ffd640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b40feeb0 0x7fc6b4196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b27fc640 1 -- 172.21.15.44:0/4258847773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b40feeb0 msgr2=0x7fc6b4196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b27fc640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b40feeb0 0x7fc6b4196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.315+0000 7fc6b27fc640 1 -- 172.21.15.44:0/4258847773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6b419aba0 msgr2=0x7fc6b419b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b27fc640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6b419aba0 0x7fc6b419b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b27fc640 1 -- 172.21.15.44:0/4258847773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009560 con 0x7fc6b4104000 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b27fc640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 0x7fc6b41967b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005ab0 tx=0x7fc6a80329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8032df0 con 0x7fc6b4104000 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b41a2e50 con 0x7fc6b4104000 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6a8005490 con 0x7fc6b4104000 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80035f0 con 0x7fc6b4104000 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.316+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b41a3270 con 0x7fc6b4104000 2024-09-13T17:36:01.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.317+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4100380 con 0x7fc6b4104000 2024-09-13T17:36:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.319+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc6a80047c0 con 0x7fc6b4104000 2024-09-13T17:36:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.320+0000 7fc6b888d640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc69c0a1200 0x7fc69c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.320+0000 7fc6b2ffd640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc69c0a1200 0x7fc69c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.320+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fc6a80edef0 con 0x7fc6b4104000 2024-09-13T17:36:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.320+0000 7fc6b2ffd640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc69c0a1200 0x7fc69c0a36c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009bd0 tx=0x7fc6a4005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.323+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc6a80b7080 con 0x7fc6b4104000 2024-09-13T17:36:01.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.530+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6b40ff290 con 0x7fc6b4104000 2024-09-13T17:36:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.531+0000 7fc6b888d640 1 -- 172.21.15.44:0/4258847773 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fc6a80ba6a0 con 0x7fc6b4104000 2024-09-13T17:36:01.533 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:36:01.533 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:36:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc69c0a1200 msgr2=0x7fc69c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc69c0a1200 0x7fc69c0a36c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009bd0 tx=0x7fc6a4005f90 comp rx=0 tx=0).stop 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 msgr2=0x7fc6b41967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 0x7fc6b41967b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005ab0 tx=0x7fc6a80329f0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 shutdown_connections 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc69c0a1200 0x7fc69c0a36c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6b419aba0 0x7fc6b419b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4104000 0x7fc6b41967b0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 --2- 172.21.15.44:0/4258847773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b40feeb0 0x7fc6b4196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 >> 172.21.15.44:0/4258847773 conn(0x7fc6b40fa900 msgr2=0x7fc6b40689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 shutdown_connections 2024-09-13T17:36:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.533+0000 7fc6b988f640 1 -- 172.21.15.44:0/4258847773 wait complete. 2024-09-13T17:36:01.547 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:36:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 -- 172.21.15.44:0/3187869859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c0ff830 msgr2=0x7f9f2c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 --2- 172.21.15.44:0/3187869859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c0ff830 0x7f9f2c103870 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009930 tx=0x7f9f1402a420 comp rx=0 tx=0).stop 2024-09-13T17:36:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 -- 172.21.15.44:0/3187869859 shutdown_connections 2024-09-13T17:36:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 --2- 172.21.15.44:0/3187869859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 --2- 172.21.15.44:0/3187869859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c0ff830 0x7f9f2c103870 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 --2- 172.21.15.44:0/3187869859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f2c0fee80 0x7f9f2c0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 -- 172.21.15.44:0/3187869859 >> 172.21.15.44:0/3187869859 conn(0x7f9f2c0fa8d0 msgr2=0x7f9f2c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 -- 172.21.15.44:0/3187869859 shutdown_connections 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.649+0000 7f9f325bc640 1 -- 172.21.15.44:0/3187869859 wait complete. 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 Processor -- start 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 -- start start 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f2c0fee80 0x7f9f2c1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c19aae0 0x7f9f2c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19b4d0 con 0x7f9f2c103fd0 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19b640 con 0x7f9f2c19aae0 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f325bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19b7b0 con 0x7f9f2c0fee80 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40482/0 (socket says 172.21.15.44:40482) 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/2184941912 learned_addr learned my addr 172.21.15.44:0/2184941912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f2c0fee80 0x7f9f2c1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f30b59640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c19aae0 0x7f9f2c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/2184941912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f2c0fee80 msgr2=0x7f9f2c1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f2c0fee80 0x7f9f2c1961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/2184941912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c19aae0 msgr2=0x7f9f2c19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.650+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c19aae0 0x7f9f2c19af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/2184941912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009560 con 0x7f9f2c103fd0 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f30b59640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c19aae0 0x7f9f2c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c1966f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005ab0 tx=0x7f9f14032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f297fa640 1 -- 172.21.15.44:0/2184941912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14032e10 con 0x7f9f2c103fd0 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f297fa640 1 -- 172.21.15.44:0/2184941912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f140046c0 con 0x7f9f2c103fd0 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f297fa640 1 -- 172.21.15.44:0/2184941912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14003830 con 0x7f9f2c103fd0 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c1a2d90 con 0x7f9f2c103fd0 2024-09-13T17:36:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.651+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c1a3160 con 0x7f9f2c103fd0 2024-09-13T17:36:01.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.653+0000 7f9f297fa640 1 -- 172.21.15.44:0/2184941912 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9f14004830 con 0x7f9f2c103fd0 2024-09-13T17:36:01.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.653+0000 7f9f34f22740 1 Processor -- start 2024-09-13T17:36:01.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.653+0000 7f9f34f22740 1 -- start start 2024-09-13T17:36:01.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf55790 0x559ddbf55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf60410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf60950 0x559ddbf62d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f34f22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbebfcc0 con 0x559ddbf560b0 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f34f22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbebd450 con 0x559ddbf60950 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f34f22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbeb8d40 con 0x559ddbf55790 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf60410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf60410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40486/0 (socket says 172.21.15.44:40486) 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 -- 172.21.15.44:0/3673558013 learned_addr learned my addr 172.21.15.44:0/3673558013 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f30b59640 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf55790 0x559ddbf55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf60950 0x559ddbf62d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f297fa640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f180a1200 0x7f9f180a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 -- 172.21.15.44:0/3673558013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf55790 msgr2=0x559ddbf55b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf55790 0x559ddbf55b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 -- 172.21.15.44:0/3673558013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf60950 msgr2=0x559ddbf62d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf60950 0x559ddbf62d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 -- 172.21.15.44:0/3673558013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ddbe80620 con 0x559ddbf560b0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f30b59640 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf55790 0x559ddbf55b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f297fa640 1 -- 172.21.15.44:0/2184941912 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9f140edc90 con 0x7f9f2c103fd0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f30b59640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f180a1200 0x7f9f180a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.654+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf60410 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00ad00 tx=0x7f9f1c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465082bf86326e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f227fc640 1 -- 172.21.15.44:0/3673558013 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00a800 con 0x559ddbf560b0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f227fc640 1 -- 172.21.15.44:0/3673558013 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f1c018ad0 con 0x559ddbf560b0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f227fc640 1 -- 172.21.15.44:0/3673558013 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c010b40 con 0x559ddbf560b0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f30b59640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f180a1200 0x7f9f180a36c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f24005fd0 tx=0x7f9f24004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 -- 172.21.15.44:0/3673558013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 msgr2=0x559ddbf60410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf60410 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00ad00 tx=0x7f9f1c00f790 comp rx=0 tx=0).stop 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 -- 172.21.15.44:0/3673558013 shutdown_connections 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf60950 0x559ddbf62d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf60410 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3673558013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf55790 0x559ddbf55b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 -- 172.21.15.44:0/3673558013 >> 172.21.15.44:0/3673558013 conn(0x559ddbf51d00 msgr2=0x559ddbf520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 -- 172.21.15.44:0/3673558013 shutdown_connections 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.655+0000 7f9f34f22740 1 -- 172.21.15.44:0/3673558013 wait complete. 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 -- start start 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 Processor -- start 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 0x559ddbf5ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf560b0 0x559ddbf5d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf5d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbe77180 con 0x559ddbf55790 2024-09-13T17:36:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbb91430 con 0x559ddbf560b0 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbe3c5e0 con 0x559ddbf60950 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f34f22740 10 client.0 _ll_register_callbacks cb 0x559ddbea8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 0x559ddbf5ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 0x559ddbf5ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40494/0 (socket says 172.21.15.44:40494) 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f2bfff640 1 -- 172.21.15.44:0/1606736743 learned_addr learned my addr 172.21.15.44:0/1606736743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.656+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf560b0 0x559ddbf5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f30b59640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf5d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f2bfff640 1 -- 172.21.15.44:0/1606736743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 msgr2=0x559ddbf5d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf5d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f2bfff640 1 -- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf560b0 msgr2=0x559ddbf5d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf560b0 0x559ddbf5d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f2bfff640 1 -- 172.21.15.44:0/1606736743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c00a960 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 0x559ddbf5ff10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e59dcac81f8c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/1606736743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c010ae0 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/1606736743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9f1c010c40 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ddbee2b80 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/1606736743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f1c00a7e0 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f34f22740 10 client.18531 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559ddbeb73c0 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.657+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/1606736743 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559ddbee2b80 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/1606736743 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f1c0048c0 con 0x559ddbf55790 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f34f22740 10 client.18531 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f34f22740 10 client.18531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:36:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f34f22740 20 client.18531 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f34f22740 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbf72660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f34f22740 4 client.18531 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ddbeab1f0 con 0x559ddbf70270 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbf72660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbf72660 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b03b36ab3e4087b server_cookie=977738e74a975054 in_seq=0 out_seq=0 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.658+0000 7f9f0b7fe640 10 client.18531 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/1606736743 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559ddbeab1f0 con 0x559ddbf70270 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f0b7fe640 10 client.18531 handle_command_reply: tid=0 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 client.18531 shutdown 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 msgr2=0x559ddbf5ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 0x559ddbf5ff10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c018670 comp rx=0 tx=0).stop 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 shutdown_connections 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbf72660 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf5d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f0b7fe640 0 client.18531 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf560b0 0x559ddbf5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 --2- 172.21.15.44:0/1606736743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf55790 0x559ddbf5ff10 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 >> 172.21.15.44:0/1606736743 conn(0x559ddbf51d00 msgr2=0x559ddbf57a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 shutdown_connections 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 1 -- 172.21.15.44:0/1606736743 wait complete. 2024-09-13T17:36:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.659+0000 7f9f34f22740 20 client.18531 trim_cache size 0 max 16384 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.670+0000 7f9f34f22740 1 Processor -- start 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.670+0000 7f9f34f22740 1 -- start start 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf560b0 0x559ddbf67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 0x559ddbf58330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f34f22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbf6c750 con 0x559ddbf60950 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f34f22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbf63400 con 0x559ddbf55790 2024-09-13T17:36:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f34f22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbf5e3d0 con 0x559ddbf560b0 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 0x559ddbf58330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 0x559ddbf58330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40506/0 (socket says 172.21.15.44:40506) 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 -- 172.21.15.44:0/3862547394 learned_addr learned my addr 172.21.15.44:0/3862547394 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 -- 172.21.15.44:0/3862547394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf560b0 msgr2=0x559ddbf67a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf560b0 0x559ddbf67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf560b0 0x559ddbf67a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 -- 172.21.15.44:0/3862547394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 msgr2=0x559ddbf516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 -- 172.21.15.44:0/3862547394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ddbe59770 con 0x559ddbf60950 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f30b59640 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 0x559ddbf58330 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9f240128b0 tx=0x7f9f24014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac087fc413feec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f22ffd640 1 -- 172.21.15.44:0/3862547394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2401fbe0 con 0x559ddbf60950 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f22ffd640 1 -- 172.21.15.44:0/3862547394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f24012e30 con 0x559ddbf60950 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.671+0000 7f9f22ffd640 1 -- 172.21.15.44:0/3862547394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240286a0 con 0x559ddbf60950 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 -- 172.21.15.44:0/3862547394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 msgr2=0x559ddbf58330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 0x559ddbf58330 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9f240128b0 tx=0x7f9f24014000 comp rx=0 tx=0).stop 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 -- 172.21.15.44:0/3862547394 shutdown_connections 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf560b0 0x559ddbf67a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf60950 0x559ddbf58330 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3862547394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 -- 172.21.15.44:0/3862547394 >> 172.21.15.44:0/3862547394 conn(0x559ddbf51d00 msgr2=0x559ddbf5bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 -- 172.21.15.44:0/3862547394 shutdown_connections 2024-09-13T17:36:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 -- 172.21.15.44:0/3862547394 wait complete. 2024-09-13T17:36:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 -- start start 2024-09-13T17:36:01.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.672+0000 7f9f34f22740 1 Processor -- start 2024-09-13T17:36:01.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf5aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf59b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbf5a270 con 0x559ddbf560b0 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbfca760 con 0x559ddbf55790 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbfca8a0 con 0x559ddbf60950 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 10 client.0 _ll_register_callbacks cb 0x7f9f1c00a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40520/0 (socket says 172.21.15.44:40520) 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/3919796344 learned_addr learned my addr 172.21.15.44:0/3919796344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f30b59640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf59b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf5aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/3919796344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 msgr2=0x559ddbf59b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf59b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 msgr2=0x559ddbf5aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf5aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 -- 172.21.15.44:0/3919796344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24012590 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f2b7fe640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf594f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9f140eb120 tx=0x7f9f140ba640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9018c4fcd3f8c77c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/3919796344 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f140be860 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/3919796344 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9f140c2eb0 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.673+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f1c018ad0 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/3919796344 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f140d3220 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 10 client.18537 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559ddbe7ee80 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/3919796344 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f1c018ad0 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/3919796344 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f140eb2f0 con 0x559ddbf560b0 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 10 client.18537 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 10 client.18537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 20 client.18537 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbfcdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 4 client.18537 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ddbbac4e0 con 0x559ddbf70270 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.674+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbfcdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f0b7fe640 10 client.18537 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f2bfff640 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbfcdd70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=544c80ed0417e893 server_cookie=c0bce6da60bf9f34 in_seq=0 out_seq=0 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f0b7fe640 1 -- 172.21.15.44:0/3919796344 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ddbbac4e0 con 0x559ddbf70270 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f0b7fe640 10 client.18537 handle_command_reply: tid=0 2024-09-13T17:36:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 client.18537 shutdown 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 msgr2=0x559ddbf594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf594f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9f140eb120 tx=0x7f9f140ba640 comp rx=0 tx=0).stop 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 shutdown_connections 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x559ddbf70270 0x559ddbfcdd70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ddbf60950 0x559ddbf59b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f0b7fe640 0 client.18537 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ddbf560b0 0x559ddbf594f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 --2- 172.21.15.44:0/3919796344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559ddbf55790 0x559ddbf5aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 >> 172.21.15.44:0/3919796344 conn(0x559ddbf51d00 msgr2=0x559ddbf582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 shutdown_connections 2024-09-13T17:36:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 1 -- 172.21.15.44:0/3919796344 wait complete. 2024-09-13T17:36:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.675+0000 7f9f34f22740 20 client.18537 trim_cache size 0 max 16384 2024-09-13T17:36:01.685 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:36:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f180a1200 msgr2=0x7f9f180a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f180a1200 0x7f9f180a36c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f24005fd0 tx=0x7f9f24004a40 comp rx=0 tx=0).stop 2024-09-13T17:36:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 msgr2=0x7f9f2c1966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c1966f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005ab0 tx=0x7f9f14032c20 comp rx=0 tx=0).stop 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 shutdown_connections 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9f180a1200 0x7f9f180a36c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2c19aae0 0x7f9f2c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2c103fd0 0x7f9f2c1966f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 --2- 172.21.15.44:0/2184941912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f2c0fee80 0x7f9f2c1961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 >> 172.21.15.44:0/2184941912 conn(0x7f9f2c0fa8d0 msgr2=0x7f9f2c0688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 shutdown_connections 2024-09-13T17:36:01.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.676+0000 7f9f325bc640 1 -- 172.21.15.44:0/2184941912 wait complete. 2024-09-13T17:36:01.689 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:36:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2208331836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c28100900 msgr2=0x7f9c28104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2208331836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c28100900 0x7f9c28104940 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009930 tx=0x7f9c1c02a420 comp rx=0 tx=0).stop 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2208331836 shutdown_connections 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2208331836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c281050a0 0x7f9c2810cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2208331836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c28100900 0x7f9c28104940 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2208331836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c280fff50 0x7f9c28100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2208331836 >> 172.21.15.44:0/2208331836 conn(0x7f9c280fb9e0 msgr2=0x7f9c280fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2208331836 shutdown_connections 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2208331836 wait complete. 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 Processor -- start 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.793+0000 7f9c2e4d7640 1 -- start start 2024-09-13T17:36:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 0x7f9c28196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c281050a0 0x7f9c281967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2819ab60 0x7f9c2819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2819b550 con 0x7f9c280fff50 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2819b6c0 con 0x7f9c281050a0 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2819b830 con 0x7f9c2819ab60 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 0x7f9c28196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 0x7f9c28196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40542/0 (socket says 172.21.15.44:40542) 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 -- 172.21.15.44:0/2550165420 learned_addr learned my addr 172.21.15.44:0/2550165420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2819ab60 0x7f9c2819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c277fe640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c281050a0 0x7f9c281967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 -- 172.21.15.44:0/2550165420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2819ab60 msgr2=0x7f9c2819b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2819ab60 0x7f9c2819b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 -- 172.21.15.44:0/2550165420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c281050a0 msgr2=0x7f9c281967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c281050a0 0x7f9c281967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 -- 172.21.15.44:0/2550165420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c009560 con 0x7f9c280fff50 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c27fff640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 0x7f9c28196280 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9f0 tx=0x7f9c1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c257fa640 1 -- 172.21.15.44:0/2550165420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c18009700 con 0x7f9c280fff50 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c257fa640 1 -- 172.21.15.44:0/2550165420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c1800cd80 con 0x7f9c280fff50 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c281a2e10 con 0x7f9c280fff50 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.794+0000 7f9c257fa640 1 -- 172.21.15.44:0/2550165420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c180187e0 con 0x7f9c280fff50 2024-09-13T17:36:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.795+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c281a3280 con 0x7f9c280fff50 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c257fa640 1 -- 172.21.15.44:0/2550165420 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9c18010430 con 0x7f9c280fff50 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 Processor -- start 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 -- start start 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 0x5617d3d26b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d31950 0x5617d3d33d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3c90cc0 con 0x5617d3d26790 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3c8e450 con 0x5617d3d270b0 2024-09-13T17:36:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c311fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3c89d40 con 0x5617d3d31950 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c2ca74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 0x5617d3d26b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c2ca74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 0x5617d3d26b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40548/0 (socket says 172.21.15.44:40548) 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.797+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1714041497 learned_addr learned my addr 172.21.15.44:0/1714041497 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c277fe640 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d31950 0x5617d3d33d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c257fa640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9c100a1280 0x7f9c100a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c27fff640 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1714041497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d31950 msgr2=0x5617d3d33d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d31950 0x5617d3d33d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1714041497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 msgr2=0x5617d3d31410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d31410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1714041497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617d3c51620 con 0x5617d3d26790 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9c100a1280 0x7f9c100a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c257fa640 1 -- 172.21.15.44:0/2550165420 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9c18014070 con 0x7f9c280fff50 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 0x5617d3d26b70 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c20011590 tx=0x7f9c20011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f00d702447ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c0e7fc640 1 -- 172.21.15.44:0/1714041497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2001bb90 con 0x5617d3d26790 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c0e7fc640 1 -- 172.21.15.44:0/1714041497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c2001f070 con 0x5617d3d26790 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9c100a1280 0x7f9c100a3740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c20022000 tx=0x7f9c20023000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.798+0000 7f9c0e7fc640 1 -- 172.21.15.44:0/1714041497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20025cf0 con 0x5617d3d26790 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 -- 172.21.15.44:0/1714041497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 msgr2=0x5617d3d26b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 0x5617d3d26b70 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c20011590 tx=0x7f9c20011a50 comp rx=0 tx=0).stop 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 -- 172.21.15.44:0/1714041497 shutdown_connections 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d31950 0x5617d3d33d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d31410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1714041497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d26790 0x5617d3d26b70 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 -- 172.21.15.44:0/1714041497 >> 172.21.15.44:0/1714041497 conn(0x5617d3d22d00 msgr2=0x5617d3d230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 -- 172.21.15.44:0/1714041497 shutdown_connections 2024-09-13T17:36:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 -- 172.21.15.44:0/1714041497 wait complete. 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.799+0000 7f9c311fd740 1 -- start start 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 Processor -- start 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d30ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d2e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3c48180 con 0x5617d3d31950 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3962430 con 0x5617d3d270b0 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3c0d5e0 con 0x5617d3d26790 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c311fd740 10 client.0 _ll_register_callbacks cb 0x5617d3c79aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d2e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40564/0 (socket says 172.21.15.44:40564) 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 -- 172.21.15.44:0/38370312 learned_addr learned my addr 172.21.15.44:0/38370312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c27fff640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d30ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c277fe640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d2e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 -- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 msgr2=0x5617d3d30ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d30ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 -- 172.21.15.44:0/38370312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 msgr2=0x5617d3d2e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d2e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 -- 172.21.15.44:0/38370312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c20009c50 con 0x5617d3d31950 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d2e760 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9c20000f80 tx=0x7f9c2000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276333a5261b0865 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.800+0000 7f9c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c077fe640 1 -- 172.21.15.44:0/38370312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c200256f0 con 0x5617d3d31950 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c077fe640 1 -- 172.21.15.44:0/38370312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9c2001b990 con 0x5617d3d31950 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617d3cb3b80 con 0x5617d3d31950 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c077fe640 1 -- 172.21.15.44:0/38370312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c2001bb50 con 0x5617d3d31950 2024-09-13T17:36:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 10 client.18549 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5617d3c883c0 con 0x5617d3d31950 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c077fe640 1 -- 172.21.15.44:0/38370312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5617d3cb3b80 con 0x5617d3d31950 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c077fe640 1 -- 172.21.15.44:0/38370312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c2002eb70 con 0x5617d3d31950 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 10 client.18549 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 10 client.18549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 20 client.18549 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d41200 0x5617d3d435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 4 client.18549 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617d3c7c1f0 con 0x5617d3d41200 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.801+0000 7f9c27fff640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d41200 0x5617d3d435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.802+0000 7f9c27fff640 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d41200 0x5617d3d435f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8c7c33fa2aafb3b server_cookie=da90788d1b76b216 in_seq=0 out_seq=0 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.802+0000 7f9c077fe640 10 client.18549 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.802+0000 7f9c077fe640 1 -- 172.21.15.44:0/38370312 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5617d3c7c1f0 con 0x5617d3d41200 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.802+0000 7f9c077fe640 10 client.18549 handle_command_reply: tid=0 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 client.18549 shutdown 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 msgr2=0x5617d3d2e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d2e760 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9c20000f80 tx=0x7f9c2000f810 comp rx=0 tx=0).stop 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 shutdown_connections 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d41200 0x5617d3d435f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d2e760 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d270b0 0x5617d3d2e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c077fe640 0 client.18549 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 --2- 172.21.15.44:0/38370312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d30ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 >> 172.21.15.44:0/38370312 conn(0x5617d3d22d00 msgr2=0x5617d3d28a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 shutdown_connections 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 1 -- 172.21.15.44:0/38370312 wait complete. 2024-09-13T17:36:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.803+0000 7f9c311fd740 20 client.18549 trim_cache size 0 max 16384 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.814+0000 7f9c311fd740 1 Processor -- start 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.814+0000 7f9c311fd740 1 -- start start 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.814+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.814+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 0x5617d3d2cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.814+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d31950 0x5617d3d2d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c311fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3cb3b80 con 0x5617d3d270b0 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c311fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3d1d3b0 con 0x5617d3d31950 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c311fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3d314c0 con 0x5617d3d26790 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 0x5617d3d2cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 0x5617d3d2cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40566/0 (socket says 172.21.15.44:40566) 2024-09-13T17:36:01.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 -- 172.21.15.44:0/966954570 learned_addr learned my addr 172.21.15.44:0/966954570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d31950 0x5617d3d2d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c27fff640 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 -- 172.21.15.44:0/966954570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 msgr2=0x5617d3d226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d226a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 -- 172.21.15.44:0/966954570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d31950 msgr2=0x5617d3d2d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d31950 0x5617d3d2d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 -- 172.21.15.44:0/966954570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617d3c2a770 con 0x5617d3d270b0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d31950 0x5617d3d2d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c277fe640 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 0x5617d3d2cad0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9c28069630 tx=0x7f9c1c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c0bc447e0fe081 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c0effd640 1 -- 172.21.15.44:0/966954570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c004030 con 0x5617d3d270b0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c0effd640 1 -- 172.21.15.44:0/966954570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c1c031960 con 0x5617d3d270b0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.815+0000 7f9c0effd640 1 -- 172.21.15.44:0/966954570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c003b30 con 0x5617d3d270b0 2024-09-13T17:36:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 -- 172.21.15.44:0/966954570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 msgr2=0x5617d3d2cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 0x5617d3d2cad0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9c28069630 tx=0x7f9c1c0314e0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 -- 172.21.15.44:0/966954570 shutdown_connections 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d31950 0x5617d3d2d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d270b0 0x5617d3d2cad0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 --2- 172.21.15.44:0/966954570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d26790 0x5617d3d226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 -- 172.21.15.44:0/966954570 >> 172.21.15.44:0/966954570 conn(0x5617d3d22d00 msgr2=0x5617d3d230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 -- 172.21.15.44:0/966954570 shutdown_connections 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 -- 172.21.15.44:0/966954570 wait complete. 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 -- start start 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.816+0000 7f9c311fd740 1 Processor -- start 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d26790 0x5617d3d38840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d270b0 0x5617d3d36c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3d377d0 con 0x5617d3d31950 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3d37940 con 0x5617d3d26790 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d3d37b00 con 0x5617d3d270b0 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c311fd740 10 client.0 _ll_register_callbacks cb 0x7f9c20009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d26790 0x5617d3d38840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40582/0 (socket says 172.21.15.44:40582) 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1839870769 learned_addr learned my addr 172.21.15.44:0/1839870769 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c277fe640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d270b0 0x5617d3d36c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d270b0 msgr2=0x5617d3d36c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d270b0 0x5617d3d36c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1839870769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d26790 msgr2=0x5617d3d38840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d26790 0x5617d3d38840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 -- 172.21.15.44:0/1839870769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2810d430 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c277fe640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d270b0 0x5617d3d36c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c2ca74640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d371c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9c2002ef50 tx=0x7f9c20005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37935ee47a61c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.817+0000 7f9c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c077fe640 1 -- 172.21.15.44:0/1839870769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20037500 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c077fe640 1 -- 172.21.15.44:0/1839870769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9c20025700 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c2001f070 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c077fe640 1 -- 172.21.15.44:0/1839870769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c20011040 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 10 client.18555 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5617d3c4fe80 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c077fe640 1 -- 172.21.15.44:0/1839870769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c2001f070 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c077fe640 1 -- 172.21.15.44:0/1839870769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c2001ba00 con 0x5617d3d31950 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 10 client.18555 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 10 client.18555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:36:01.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 20 client.18555 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d989d0 0x5617d3d9adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 4 client.18555 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5617d397d4e0 con 0x5617d3d989d0 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.818+0000 7f9c27fff640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d989d0 0x5617d3d9adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.819+0000 7f9c077fe640 10 client.18555 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.819+0000 7f9c27fff640 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d989d0 0x5617d3d9adc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd55fc73fb0dc90e server_cookie=1b581e4dfefb0a3f in_seq=0 out_seq=0 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.819+0000 7f9c077fe640 1 -- 172.21.15.44:0/1839870769 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5617d397d4e0 con 0x5617d3d989d0 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.819+0000 7f9c077fe640 10 client.18555 handle_command_reply: tid=0 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.819+0000 7f9c311fd740 1 client.18555 shutdown 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 msgr2=0x5617d3d371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d371c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9c2002ef50 tx=0x7f9c20005cf0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 shutdown_connections 2024-09-13T17:36:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5617d3d989d0 0x5617d3d9adc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617d3d270b0 0x5617d3d36c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617d3d31950 0x5617d3d371c0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 --2- 172.21.15.44:0/1839870769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617d3d26790 0x5617d3d38840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c077fe640 0 client.18555 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 >> 172.21.15.44:0/1839870769 conn(0x5617d3d22d00 msgr2=0x5617d3d2ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 shutdown_connections 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 1 -- 172.21.15.44:0/1839870769 wait complete. 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c311fd740 20 client.18555 trim_cache size 0 max 16384 2024-09-13T17:36:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:36:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9c100a1280 msgr2=0x7f9c100a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9c100a1280 0x7f9c100a3740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c20022000 tx=0x7f9c20023000 comp rx=0 tx=0).stop 2024-09-13T17:36:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 msgr2=0x7f9c28196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 0x7f9c28196280 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9f0 tx=0x7f9c1800eeb0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 shutdown_connections 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9c100a1280 0x7f9c100a3740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2819ab60 0x7f9c2819b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c281050a0 0x7f9c281967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 --2- 172.21.15.44:0/2550165420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c280fff50 0x7f9c28196280 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 >> 172.21.15.44:0/2550165420 conn(0x7f9c280fb9e0 msgr2=0x7f9c280fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.820+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 shutdown_connections 2024-09-13T17:36:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.821+0000 7f9c2e4d7640 1 -- 172.21.15.44:0/2550165420 wait complete. 2024-09-13T17:36:01.833 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.935+0000 7ff274f65640 1 -- 172.21.15.44:0/983678192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff27010a350 msgr2=0x7ff27010c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.935+0000 7ff274f65640 1 --2- 172.21.15.44:0/983678192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff27010a350 0x7ff27010c7a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7ff268009ca0 tx=0x7ff26802a8c0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 -- 172.21.15.44:0/983678192 shutdown_connections 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 --2- 172.21.15.44:0/983678192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff27010a350 0x7ff27010c7a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 --2- 172.21.15.44:0/983678192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff270101800 0x7ff270109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 --2- 172.21.15.44:0/983678192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270100e50 0x7ff270101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 -- 172.21.15.44:0/983678192 >> 172.21.15.44:0/983678192 conn(0x7ff2700fa930 msgr2=0x7ff2700fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 -- 172.21.15.44:0/983678192 shutdown_connections 2024-09-13T17:36:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 -- 172.21.15.44:0/983678192 wait complete. 2024-09-13T17:36:01.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.936+0000 7ff274f65640 1 Processor -- start 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 -- start start 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270100e50 0x7ff2701960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 0x7ff270196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff27010a350 0x7ff27019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019b020 con 0x7ff270101800 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019b190 con 0x7ff27010a350 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff274f65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019b350 con 0x7ff270100e50 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 0x7ff270196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 0x7ff270196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40606/0 (socket says 172.21.15.44:40606) 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 -- 172.21.15.44:0/3535476505 learned_addr learned my addr 172.21.15.44:0/3535476505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff26e59c640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270100e50 0x7ff2701960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff27010a350 0x7ff27019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 -- 172.21.15.44:0/3535476505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270100e50 msgr2=0x7ff2701960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270100e50 0x7ff2701960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 -- 172.21.15.44:0/3535476505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff27010a350 msgr2=0x7ff27019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff27010a350 0x7ff27019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 -- 172.21.15.44:0/3535476505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268009900 con 0x7ff270101800 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff27010a350 0x7ff27019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.937+0000 7ff265d9b640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 0x7ff270196630 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c890 tx=0x7ff25800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.938+0000 7ff267fff640 1 -- 172.21.15.44:0/3535476505 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258004220 con 0x7ff270101800 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.938+0000 7ff267fff640 1 -- 172.21.15.44:0/3535476505 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff258007e40 con 0x7ff270101800 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.938+0000 7ff267fff640 1 -- 172.21.15.44:0/3535476505 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580047a0 con 0x7ff270101800 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.938+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27019b630 con 0x7ff270101800 2024-09-13T17:36:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.938+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2701a2eb0 con 0x7ff270101800 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.940+0000 7ff267fff640 1 -- 172.21.15.44:0/3535476505 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff2580079b0 con 0x7ff270101800 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.940+0000 7ff277c8e740 1 Processor -- start 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.940+0000 7ff277c8e740 1 -- start start 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afdea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff277c8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08af48cc0 con 0x55c08afdef90 2024-09-13T17:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff277c8e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08af46450 con 0x55c08afedd60 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff277c8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08af41d40 con 0x55c08afde670 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afed820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40620/0 (socket says 172.21.15.44:40620) 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 -- 172.21.15.44:0/2570629707 learned_addr learned my addr 172.21.15.44:0/2570629707 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afdea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff267fff640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff25c0a1200 0x7ff25c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff265d9b640 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 -- 172.21.15.44:0/2570629707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 msgr2=0x55c08afdea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afdea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 -- 172.21.15.44:0/2570629707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 msgr2=0x55c08aff0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 -- 172.21.15.44:0/2570629707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c08af09620 con 0x55c08afdef90 2024-09-13T17:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff25c0a1200 0x7ff25c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff267fff640 1 -- 172.21.15.44:0/3535476505 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff2580cd020 con 0x7ff270101800 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.941+0000 7ff26e59c640 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afed820 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ad30 tx=0x7ff26000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d984c77dd4e07f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff264d99640 1 -- 172.21.15.44:0/2570629707 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000a800 con 0x55c08afdef90 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff264d99640 1 -- 172.21.15.44:0/2570629707 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff26000ae20 con 0x55c08afdef90 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff264d99640 1 -- 172.21.15.44:0/2570629707 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260018b00 con 0x55c08afdef90 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 -- 172.21.15.44:0/2570629707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 msgr2=0x55c08afed820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff25c0a1200 0x7ff25c0a36c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098d0 tx=0x7ff26803f040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afed820 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ad30 tx=0x7ff26000f790 comp rx=0 tx=0).stop 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 -- 172.21.15.44:0/2570629707 shutdown_connections 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afed820 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2570629707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afdea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.942+0000 7ff277c8e740 1 -- 172.21.15.44:0/2570629707 >> 172.21.15.44:0/2570629707 conn(0x55c08afdad00 msgr2=0x55c08afdb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 -- 172.21.15.44:0/2570629707 shutdown_connections 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 -- 172.21.15.44:0/2570629707 wait complete. 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 -- start start 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 Processor -- start 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08afeabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08af00180 con 0x55c08afdef90 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08ac1a430 con 0x55c08afedd60 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08aec55e0 con 0x55c08afde670 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff277c8e740 10 client.0 _ll_register_callbacks cb 0x55c08af31aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff265d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff265d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40626/0 (socket says 172.21.15.44:40626) 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff265d9b640 1 -- 172.21.15.44:0/1434044547 learned_addr learned my addr 172.21.15.44:0/1434044547 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08afeabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.943+0000 7ff26e59c640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff265d9b640 1 -- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 msgr2=0x55c08afed3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff265d9b640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afed3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff265d9b640 1 -- 172.21.15.44:0/1434044547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 msgr2=0x55c08afeabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff265d9b640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08afeabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff265d9b640 1 -- 172.21.15.44:0/1434044547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26000a960 con 0x55c08afdef90 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff265d9b640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afea680 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff258096830 tx=0x7ff258099720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56255b1e846e5147 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff255ffb640 1 -- 172.21.15.44:0/1434044547 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25809dbf0 con 0x55c08afdef90 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff255ffb640 1 -- 172.21.15.44:0/1434044547 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff2580a2240 con 0x55c08afdef90 2024-09-13T17:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c08af6bb80 con 0x55c08afdef90 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff255ffb640 1 -- 172.21.15.44:0/1434044547 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff258099dc0 con 0x55c08afdef90 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff277c8e740 10 client.18567 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c08af403c0 con 0x55c08afdef90 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff255ffb640 1 -- 172.21.15.44:0/1434044547 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c08af6bb80 con 0x55c08afdef90 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff255ffb640 1 -- 172.21.15.44:0/1434044547 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff258020070 con 0x55c08afdef90 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff277c8e740 10 client.18567 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff277c8e740 10 client.18567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.944+0000 7ff277c8e740 20 client.18567 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.945+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08affd750 0x55c08afffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.945+0000 7ff277c8e740 4 client.18567 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.945+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c08af341f0 con 0x55c08affd750 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.945+0000 7ff26e59c640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08affd750 0x55c08afffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.945+0000 7ff255ffb640 10 client.18567 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.945+0000 7ff26e59c640 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08affd750 0x55c08afffb40 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6356d0802406aff server_cookie=cd34faebedbaaf33 in_seq=0 out_seq=0 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff255ffb640 1 -- 172.21.15.44:0/1434044547 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c08af341f0 con 0x55c08affd750 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff255ffb640 10 client.18567 handle_command_reply: tid=0 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 client.18567 shutdown 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 msgr2=0x55c08afea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afea680 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff258096830 tx=0x7ff258099720 comp rx=0 tx=0).stop 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 shutdown_connections 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08affd750 0x55c08afffb40 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08afeabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afdef90 0x55c08afea680 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff255ffb640 0 client.18567 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1434044547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08afed3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 >> 172.21.15.44:0/1434044547 conn(0x55c08afdad00 msgr2=0x55c08afe0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 shutdown_connections 2024-09-13T17:36:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 1 -- 172.21.15.44:0/1434044547 wait complete. 2024-09-13T17:36:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.946+0000 7ff277c8e740 20 client.18567 trim_cache size 0 max 16384 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 Processor -- start 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 -- start start 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08aff4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afdef90 0x55c08aff4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 0x55c08afece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08afdb680 con 0x55c08afedd60 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08afeaf30 con 0x55c08afdef90 2024-09-13T17:36:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff277c8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08afeb690 con 0x55c08afde670 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff26ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 0x55c08afece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff26ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 0x55c08afece50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40636/0 (socket says 172.21.15.44:40636) 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.957+0000 7ff26ed9d640 1 -- 172.21.15.44:0/1527181932 learned_addr learned my addr 172.21.15.44:0/1527181932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26e59c640 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08aff4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26ed9d640 1 -- 172.21.15.44:0/1527181932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 msgr2=0x55c08aff4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08aff4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff265d9b640 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afdef90 0x55c08aff4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26ed9d640 1 -- 172.21.15.44:0/1527181932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afdef90 msgr2=0x55c08aff4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afdef90 0x55c08aff4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26ed9d640 1 -- 172.21.15.44:0/1527181932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c08aee2770 con 0x55c08afedd60 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26ed9d640 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 0x55c08afece50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff268031ed0 tx=0x7ff268041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37b315a4be10d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26559a640 1 -- 172.21.15.44:0/1527181932 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26804e070 con 0x55c08afedd60 2024-09-13T17:36:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26559a640 1 -- 172.21.15.44:0/1527181932 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff268003e00 con 0x55c08afedd60 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff26559a640 1 -- 172.21.15.44:0/1527181932 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268004280 con 0x55c08afedd60 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff277c8e740 1 -- 172.21.15.44:0/1527181932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 msgr2=0x55c08afece50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.958+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 0x55c08afece50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff268031ed0 tx=0x7ff268041000 comp rx=0 tx=0).stop 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- 172.21.15.44:0/1527181932 shutdown_connections 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afedd60 0x55c08afece50 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afdef90 0x55c08aff4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 --2- 172.21.15.44:0/1527181932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afde670 0x55c08aff4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- 172.21.15.44:0/1527181932 >> 172.21.15.44:0/1527181932 conn(0x55c08afdad00 msgr2=0x55c08afe8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- 172.21.15.44:0/1527181932 shutdown_connections 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- 172.21.15.44:0/1527181932 wait complete. 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- start start 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 Processor -- start 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 0x55c08aff4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afdef90 0x55c08aff1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08aff1a80 con 0x55c08afde670 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08aff1bf0 con 0x55c08afedd60 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08aff1db0 con 0x55c08afdef90 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff277c8e740 10 client.0 _ll_register_callbacks cb 0x7ff26000a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 0x55c08aff4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 0x55c08aff4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40638/0 (socket says 172.21.15.44:40638) 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.959+0000 7ff26e59c640 1 -- 172.21.15.44:0/2875733065 learned_addr learned my addr 172.21.15.44:0/2875733065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff26e59c640 1 -- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afdef90 msgr2=0x55c08aff1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff265d9b640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afdef90 0x55c08aff1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff26e59c640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afdef90 0x55c08aff1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff26e59c640 1 -- 172.21.15.44:0/2875733065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 msgr2=0x55c08aff1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff26e59c640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff26e59c640 1 -- 172.21.15.44:0/2875733065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27010d420 con 0x55c08afde670 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff265d9b640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afdef90 0x55c08aff1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff26e59c640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 0x55c08aff4b20 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ad30 tx=0x7ff2600184c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d490b55f8221d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff255ffb640 1 -- 172.21.15.44:0/2875733065 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260029070 con 0x55c08afde670 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff255ffb640 1 -- 172.21.15.44:0/2875733065 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff2600189b0 con 0x55c08afde670 2024-09-13T17:36:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26000ae20 con 0x55c08afde670 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff255ffb640 1 -- 172.21.15.44:0/2875733065 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff260018b90 con 0x55c08afde670 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 10 client.18573 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c08af07e80 con 0x55c08afde670 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff255ffb640 1 -- 172.21.15.44:0/2875733065 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff26000ae20 con 0x55c08afde670 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff255ffb640 1 -- 172.21.15.44:0/2875733065 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff260020040 con 0x55c08afde670 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 10 client.18573 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 10 client.18573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 20 client.18573 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08b055050 0x55c08b057470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 4 client.18573 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.960+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c08ac354e0 con 0x55c08b055050 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff265d9b640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08b055050 0x55c08b057470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff255ffb640 10 client.18573 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff265d9b640 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08b055050 0x55c08b057470 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7437369e58eaa4c5 server_cookie=a7d7d55376901dfe in_seq=0 out_seq=0 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff255ffb640 1 -- 172.21.15.44:0/2875733065 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c08ac354e0 con 0x55c08b055050 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff255ffb640 10 client.18573 handle_command_reply: tid=0 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 client.18573 shutdown 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 msgr2=0x55c08aff4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 0x55c08aff4b20 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ad30 tx=0x7ff2600184c0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 shutdown_connections 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55c08b055050 0x55c08b057470 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08afdef90 0x55c08aff1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.961+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c08afedd60 0x55c08aff1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff277c8e740 1 --2- 172.21.15.44:0/2875733065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c08afde670 0x55c08aff4b20 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff255ffb640 0 client.18573 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 >> 172.21.15.44:0/2875733065 conn(0x55c08afdad00 msgr2=0x55c08afe8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 shutdown_connections 2024-09-13T17:36:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff277c8e740 1 -- 172.21.15.44:0/2875733065 wait complete. 2024-09-13T17:36:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff277c8e740 20 client.18573 trim_cache size 0 max 16384 2024-09-13T17:36:01.971 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:36:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff25c0a1200 msgr2=0x7ff25c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff25c0a1200 0x7ff25c0a36c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098d0 tx=0x7ff26803f040 comp rx=0 tx=0).stop 2024-09-13T17:36:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 msgr2=0x7ff270196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 0x7ff270196630 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c890 tx=0x7ff25800cc90 comp rx=0 tx=0).stop 2024-09-13T17:36:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 shutdown_connections 2024-09-13T17:36:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff25c0a1200 0x7ff25c0a36c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff27010a350 0x7ff27019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff270101800 0x7ff270196630 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 --2- 172.21.15.44:0/3535476505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270100e50 0x7ff2701960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 >> 172.21.15.44:0/3535476505 conn(0x7ff2700fa930 msgr2=0x7ff2700fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 shutdown_connections 2024-09-13T17:36:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:01.962+0000 7ff274f65640 1 -- 172.21.15.44:0/3535476505 wait complete. 2024-09-13T17:36:02.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:02 smithi044 ceph-mon[31175]: pgmap v288: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4258847773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:36:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:02 smithi070 ceph-mon[34502]: pgmap v288: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:02 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4258847773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:36:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:02 smithi100 ceph-mon[35055]: pgmap v288: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:02 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4258847773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:36:02.975 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:36:02.975 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 -- 172.21.15.44:0/912646318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103fb0 msgr2=0x7f527c10bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 --2- 172.21.15.44:0/912646318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103fb0 0x7f527c10bb00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f5274009cd0 tx=0x7f527402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 -- 172.21.15.44:0/912646318 shutdown_connections 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 --2- 172.21.15.44:0/912646318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c0ff810 0x7f527c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 --2- 172.21.15.44:0/912646318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103fb0 0x7f527c10bb00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 --2- 172.21.15.44:0/912646318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527c0fee60 0x7f527c0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 -- 172.21.15.44:0/912646318 >> 172.21.15.44:0/912646318 conn(0x7f527c075c60 msgr2=0x7f527c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 -- 172.21.15.44:0/912646318 shutdown_connections 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.094+0000 7f528439c640 1 -- 172.21.15.44:0/912646318 wait complete. 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 Processor -- start 2024-09-13T17:36:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 -- start start 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c0fee60 0x7f527c1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527c103fb0 0x7f527c196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 0x7f527c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c19b4f0 con 0x7f527c19ab00 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c19b660 con 0x7f527c103fb0 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f528439c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c19b7d0 con 0x7f527c0fee60 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f5282939640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 0x7f527c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f5282939640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 0x7f527c19afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40670/0 (socket says 172.21.15.44:40670) 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f5282939640 1 -- 172.21.15.44:0/2973135590 learned_addr learned my addr 172.21.15.44:0/2973135590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f5282138640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c0fee60 0x7f527c1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.095+0000 7f5281937640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527c103fb0 0x7f527c196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f5282939640 1 -- 172.21.15.44:0/2973135590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c0fee60 msgr2=0x7f527c1961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f5282939640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c0fee60 0x7f527c1961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f5282939640 1 -- 172.21.15.44:0/2973135590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527c103fb0 msgr2=0x7f527c196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f5282939640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527c103fb0 0x7f527c196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f5282939640 1 -- 172.21.15.44:0/2973135590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274009900 con 0x7f527c19ab00 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f5282939640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 0x7f527c19afb0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5274004af0 tx=0x7f5274030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f52737fe640 1 -- 172.21.15.44:0/2973135590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274031b80 con 0x7f527c19ab00 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f52737fe640 1 -- 172.21.15.44:0/2973135590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5274003e00 con 0x7f527c19ab00 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f52737fe640 1 -- 172.21.15.44:0/2973135590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274045410 con 0x7f527c19ab00 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c1a2db0 con 0x7f527c19ab00 2024-09-13T17:36:03.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.096+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c1a31e0 con 0x7f527c19ab00 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.098+0000 7f52737fe640 1 -- 172.21.15.44:0/2973135590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f527400a040 con 0x7f527c19ab00 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.099+0000 7f52870b9740 1 Processor -- start 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.099+0000 7f52870b9740 1 -- start start 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.099+0000 7f52870b9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289ed660 0x5562289eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52870b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289edf80 0x5562289fc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52870b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 0x5562289ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52870b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228958c60 con 0x5562289fcd90 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52870b9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562289563f0 con 0x5562289ed660 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52870b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228951ce0 con 0x5562289edf80 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52737fe640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f52680a1280 0x7f52680a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 0x5562289ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 0x5562289ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40684/0 (socket says 172.21.15.44:40684) 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 -- 172.21.15.44:0/1839530964 learned_addr learned my addr 172.21.15.44:0/1839530964 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289ed660 0x5562289eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5282138640 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289edf80 0x5562289fc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5282939640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f52680a1280 0x7f52680a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f52737fe640 1 -- 172.21.15.44:0/2973135590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f52740ee3c0 con 0x7f527c19ab00 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 -- 172.21.15.44:0/1839530964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289edf80 msgr2=0x5562289fc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289edf80 0x5562289fc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 -- 172.21.15.44:0/1839530964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289ed660 msgr2=0x5562289eda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289ed660 0x5562289eda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 -- 172.21.15.44:0/1839530964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55622890d970 con 0x5562289fcd90 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5281937640 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 0x5562289ff180 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f526c012700 tx=0x7f526c00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5da463be3433d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.100+0000 7f5282939640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f52680a1280 0x7f52680a3740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52740b7c20 tx=0x7f527402a8e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f5270e05640 1 -- 172.21.15.44:0/1839530964 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c012d40 con 0x5562289fcd90 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f5270e05640 1 -- 172.21.15.44:0/1839530964 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f526c00b9d0 con 0x5562289fcd90 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f5270e05640 1 -- 172.21.15.44:0/1839530964 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c010e90 con 0x5562289fcd90 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 -- 172.21.15.44:0/1839530964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 msgr2=0x5562289ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 0x5562289ff180 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f526c012700 tx=0x7f526c00b6e0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 -- 172.21.15.44:0/1839530964 shutdown_connections 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289edf80 0x5562289fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289fcd90 0x5562289ff180 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 --2- 172.21.15.44:0/1839530964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289ed660 0x5562289eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 -- 172.21.15.44:0/1839530964 >> 172.21.15.44:0/1839530964 conn(0x5562289e9d50 msgr2=0x5562289ea130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 -- 172.21.15.44:0/1839530964 shutdown_connections 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.101+0000 7f52870b9740 1 -- 172.21.15.44:0/1839530964 wait complete. 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 -- start start 2024-09-13T17:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 Processor -- start 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55622890ff80 con 0x5562289ed660 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55622890fc20 con 0x5562289edf80 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562288dcab0 con 0x5562289fcd90 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f52870b9740 10 client.0 _ll_register_callbacks cb 0x55622894b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40686/0 (socket says 172.21.15.44:40686) 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f5281937640 1 -- 172.21.15.44:0/2592717430 learned_addr learned my addr 172.21.15.44:0/2592717430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.102+0000 7f5282138640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 -- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 msgr2=0x5562289fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 -- 172.21.15.44:0/2592717430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 msgr2=0x5562289f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 -- 172.21.15.44:0/2592717430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c01a040 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5281937640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289fc7c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f526c001eb0 tx=0x7f526c010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71610060693d39f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5265ffb640 1 -- 172.21.15.44:0/2592717430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c023040 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5265ffb640 1 -- 172.21.15.44:0/2592717430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f526c000e40 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556228656860 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5265ffb640 1 -- 172.21.15.44:0/2592717430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f526c004230 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f52870b9740 10 client.18585 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556228950360 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.103+0000 7f5265ffb640 1 -- 172.21.15.44:0/2592717430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556228656860 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f5265ffb640 1 -- 172.21.15.44:0/2592717430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f526c004750 con 0x5562289ed660 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f52870b9740 10 client.18585 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f52870b9740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f52870b9740 20 client.18585 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f52870b9740 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a0ca00 0x556228a0ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f52870b9740 4 client.18585 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562286a9eb0 con 0x556228a0ca00 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.104+0000 7f5282138640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a0ca00 0x556228a0ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.105+0000 7f5282138640 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a0ca00 0x556228a0ee20 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f8c6c87fa36a23 server_cookie=59a9da64d5dc32c4 in_seq=0 out_seq=0 2024-09-13T17:36:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.105+0000 7f5265ffb640 10 client.18585 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.105+0000 7f5265ffb640 1 -- 172.21.15.44:0/2592717430 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5562286a9eb0 con 0x556228a0ca00 2024-09-13T17:36:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.105+0000 7f5265ffb640 10 client.18585 handle_command_reply: tid=0 2024-09-13T17:36:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.105+0000 7f52870b9740 1 client.18585 shutdown 2024-09-13T17:36:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 msgr2=0x5562289fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289fc7c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f526c001eb0 tx=0x7f526c010b30 comp rx=0 tx=0).stop 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 shutdown_connections 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a0ca00 0x556228a0ee20 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f5265ffb640 0 client.18585 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 --2- 172.21.15.44:0/2592717430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289fc7c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 >> 172.21.15.44:0/2592717430 conn(0x5562289e9d50 msgr2=0x5562289e9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 shutdown_connections 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 1 -- 172.21.15.44:0/2592717430 wait complete. 2024-09-13T17:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.106+0000 7f52870b9740 20 client.18585 trim_cache size 0 max 16384 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 Processor -- start 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 -- start start 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562289dbef0 con 0x5562289ed660 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562289f0220 con 0x5562289edf80 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f52870b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562289f7f70 con 0x5562289fcd90 2024-09-13T17:36:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289f8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40702/0 (socket says 172.21.15.44:40702) 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 -- 172.21.15.44:0/1694391540 learned_addr learned my addr 172.21.15.44:0/1694391540 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5282138640 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 -- 172.21.15.44:0/1694391540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 msgr2=0x5562289fe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 -- 172.21.15.44:0/1694391540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 msgr2=0x5562289fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 -- 172.21.15.44:0/1694391540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562288f21c0 con 0x5562289ed660 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.122+0000 7f5281937640 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289f8a00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f526c004e90 tx=0x7f526c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d41e5c32d9fd4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f5271606640 1 -- 172.21.15.44:0/1694391540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c023070 con 0x5562289ed660 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f5271606640 1 -- 172.21.15.44:0/1694391540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f526c02f040 con 0x5562289ed660 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f5271606640 1 -- 172.21.15.44:0/1694391540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c005490 con 0x5562289ed660 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 -- 172.21.15.44:0/1694391540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 msgr2=0x5562289f8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289f8a00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f526c004e90 tx=0x7f526c0107a0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 -- 172.21.15.44:0/1694391540 shutdown_connections 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x5562289fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x5562289fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 --2- 172.21.15.44:0/1694391540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x5562289f8a00 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.123+0000 7f52870b9740 1 -- 172.21.15.44:0/1694391540 >> 172.21.15.44:0/1694391540 conn(0x5562289e9d50 msgr2=0x5562289f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 -- 172.21.15.44:0/1694391540 shutdown_connections 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 -- 172.21.15.44:0/1694391540 wait complete. 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 -- start start 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 Processor -- start 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x556228a03e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x556228a01ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x556228a02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228a02770 con 0x5562289ed660 2024-09-13T17:36:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228a028e0 con 0x5562289edf80 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228a63ed0 con 0x5562289fcd90 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f52870b9740 10 client.0 _ll_register_callbacks cb 0x7f526c00bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x556228a03e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f5281937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x556228a03e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40710/0 (socket says 172.21.15.44:40710) 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f5281937640 1 -- 172.21.15.44:0/417857339 learned_addr learned my addr 172.21.15.44:0/417857339 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f5282138640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x556228a01ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.124+0000 7f5281937640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x556228a02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5281937640 1 -- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 msgr2=0x556228a02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5281937640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x556228a02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5281937640 1 -- 172.21.15.44:0/417857339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 msgr2=0x556228a01ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5281937640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x556228a01ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5281937640 1 -- 172.21.15.44:0/417857339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556228950360 con 0x5562289ed660 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5281937640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x556228a03e70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f526c006010 tx=0x7f526c01d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52246478ddf050d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5265ffb640 1 -- 172.21.15.44:0/417857339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c023070 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5265ffb640 1 -- 172.21.15.44:0/417857339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f526c03d070 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556228645590 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5265ffb640 1 -- 172.21.15.44:0/417857339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f526c004bb0 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f52870b9740 10 client.18591 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556228917990 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5265ffb640 1 -- 172.21.15.44:0/417857339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556228645590 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f5265ffb640 1 -- 172.21.15.44:0/417857339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f526c0035f0 con 0x5562289ed660 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f52870b9740 10 client.18591 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f52870b9740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.125+0000 7f52870b9740 20 client.18591 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.126+0000 7f52870b9740 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a651d0 0x556228a675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.126+0000 7f52870b9740 4 client.18591 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:36:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.126+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556228a67e40 con 0x556228a651d0 2024-09-13T17:36:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.126+0000 7f5282138640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a651d0 0x556228a675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.126+0000 7f5265ffb640 10 client.18591 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.126+0000 7f5282138640 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a651d0 0x556228a675c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5914f8cdfb62b785 server_cookie=e31863001fb95914 in_seq=0 out_seq=0 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f5265ffb640 1 -- 172.21.15.44:0/417857339 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556228a67e40 con 0x556228a651d0 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f5265ffb640 10 client.18591 handle_command_reply: tid=0 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 client.18591 shutdown 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 msgr2=0x556228a03e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x556228a03e70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f526c006010 tx=0x7f526c01d3f0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 shutdown_connections 2024-09-13T17:36:03.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556228a651d0 0x556228a675c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562289fcd90 0x556228a02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5562289edf80 0x556228a01ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 --2- 172.21.15.44:0/417857339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562289ed660 0x556228a03e70 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 >> 172.21.15.44:0/417857339 conn(0x5562289e9d50 msgr2=0x5562289f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f5265ffb640 0 client.18591 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 shutdown_connections 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 1 -- 172.21.15.44:0/417857339 wait complete. 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f52870b9740 20 client.18591 trim_cache size 0 max 16384 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:36:03.135 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "b26990a2-e23b-4437-b16c-ae1b196923f0", 2024-09-13T17:36:03.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:36:03.136 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:36:03.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f52680a1280 msgr2=0x7f52680a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f528439c640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f52680a1280 0x7f52680a3740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52740b7c20 tx=0x7f527402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 msgr2=0x7f527c19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.128+0000 7f528439c640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 0x7f527c19afb0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5274004af0 tx=0x7f5274030e50 comp rx=0 tx=0).stop 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 shutdown_connections 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f52680a1280 0x7f52680a3740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c19ab00 0x7f527c19afb0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527c103fb0 0x7f527c196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 --2- 172.21.15.44:0/2973135590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c0fee60 0x7f527c1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 >> 172.21.15.44:0/2973135590 conn(0x7f527c075c60 msgr2=0x7f527c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 shutdown_connections 2024-09-13T17:36:03.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.129+0000 7f528439c640 1 -- 172.21.15.44:0/2973135590 wait complete. 2024-09-13T17:36:03.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:36:03.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 -- 172.21.15.44:0/4026079232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 msgr2=0x7ff050101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 --2- 172.21.15.44:0/4026079232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050101230 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff04002a410 comp rx=0 tx=0).stop 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 -- 172.21.15.44:0/4026079232 shutdown_connections 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 --2- 172.21.15.44:0/4026079232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff050101800 0x7ff050109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 --2- 172.21.15.44:0/4026079232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff05010a350 0x7ff05010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 --2- 172.21.15.44:0/4026079232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050101230 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 -- 172.21.15.44:0/4026079232 >> 172.21.15.44:0/4026079232 conn(0x7ff0500fa930 msgr2=0x7ff0500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 -- 172.21.15.44:0/4026079232 shutdown_connections 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.244+0000 7ff056619640 1 -- 172.21.15.44:0/4026079232 wait complete. 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 Processor -- start 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 -- start start 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff050101800 0x7ff0501966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05010a350 0x7ff05019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019b0b0 con 0x7ff050100e50 2024-09-13T17:36:03.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019b220 con 0x7ff050101800 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff056619640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019b3e0 con 0x7ff05010a350 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40726/0 (socket says 172.21.15.44:40726) 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff04ffff640 1 -- 172.21.15.44:0/4174734202 learned_addr learned my addr 172.21.15.44:0/4174734202 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.245+0000 7ff054bb6640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05010a350 0x7ff05019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04ffff640 1 -- 172.21.15.44:0/4174734202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05010a350 msgr2=0x7ff05019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04ffff640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05010a350 0x7ff05019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04ffff640 1 -- 172.21.15.44:0/4174734202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff050101800 msgr2=0x7ff0501966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff047fff640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff050101800 0x7ff0501966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04ffff640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff050101800 0x7ff0501966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04ffff640 1 -- 172.21.15.44:0/4174734202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009560 con 0x7ff050100e50 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04ffff640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050196180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff040006010 tx=0x7ff040005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04dffb640 1 -- 172.21.15.44:0/4174734202 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040004c50 con 0x7ff050100e50 2024-09-13T17:36:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05019b660 con 0x7ff050100e50 2024-09-13T17:36:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04dffb640 1 -- 172.21.15.44:0/4174734202 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff040004db0 con 0x7ff050100e50 2024-09-13T17:36:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff04dffb640 1 -- 172.21.15.44:0/4174734202 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04003c930 con 0x7ff050100e50 2024-09-13T17:36:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.246+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501a2f40 con 0x7ff050100e50 2024-09-13T17:36:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.248+0000 7ff059419740 1 Processor -- start 2024-09-13T17:36:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.248+0000 7ff04dffb640 1 -- 172.21.15.44:0/4174734202 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff040003c80 con 0x7ff050100e50 2024-09-13T17:36:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.248+0000 7ff059419740 1 -- start start 2024-09-13T17:36:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff059419740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ce31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff059419740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37ceda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff059419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff059419740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37c4df20 con 0x55de37ce3730 2024-09-13T17:36:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff059419740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37c4b6b0 con 0x55de37ce2e10 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff059419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37c46fa0 con 0x55de37cedfd0 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37ceda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37ceda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40736/0 (socket says 172.21.15.44:40736) 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 -- 172.21.15.44:0/3885029946 learned_addr learned my addr 172.21.15.44:0/3885029946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff047fff640 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff054bb6640 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ce31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 -- 172.21.15.44:0/3885029946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 msgr2=0x55de37cf03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 -- 172.21.15.44:0/3885029946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 msgr2=0x55de37ce31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ce31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 -- 172.21.15.44:0/3885029946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de37c01840 con 0x55de37ce3730 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04dffb640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff03c0a1200 0x7ff03c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff054bb6640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff03c0a1200 0x7ff03c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04dffb640 1 -- 172.21.15.44:0/4174734202 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff0400edff0 con 0x7ff050100e50 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.249+0000 7ff04ffff640 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37ceda90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff0400eb2b0 tx=0x7ff0400f60e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b361dce66e569352 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff0467fc640 1 -- 172.21.15.44:0/3885029946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400bab00 con 0x55de37ce3730 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff0467fc640 1 -- 172.21.15.44:0/3885029946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0400ba870 con 0x55de37ce3730 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff0467fc640 1 -- 172.21.15.44:0/3885029946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400ba690 con 0x55de37ce3730 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff054bb6640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff03c0a1200 0x7ff03c0a36c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff0480096e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 -- 172.21.15.44:0/3885029946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 msgr2=0x55de37ceda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37ceda90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff0400eb2b0 tx=0x7ff0400f60e0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 -- 172.21.15.44:0/3885029946 shutdown_connections 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37ceda90 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 --2- 172.21.15.44:0/3885029946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ce31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 -- 172.21.15.44:0/3885029946 >> 172.21.15.44:0/3885029946 conn(0x55de37cdf380 msgr2=0x55de37cdf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 -- 172.21.15.44:0/3885029946 shutdown_connections 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.250+0000 7ff059419740 1 -- 172.21.15.44:0/3885029946 wait complete. 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 -- start start 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 Processor -- start 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37ce3730 0x55de37ceac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 0x55de37ceb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37be82a0 con 0x55de37cedfd0 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3791f430 con 0x55de37ce2e10 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37bca680 con 0x55de37ce3730 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff059419740 10 client.0 _ll_register_callbacks cb 0x55de37c40610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff054bb6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 0x55de37ceb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff054bb6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 0x55de37ceb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40740/0 (socket says 172.21.15.44:40740) 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff054bb6640 1 -- 172.21.15.44:0/1611907053 learned_addr learned my addr 172.21.15.44:0/1611907053 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff047fff640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37ce3730 0x55de37ceac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.251+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff054bb6640 1 -- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37ce3730 msgr2=0x55de37ceac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff054bb6640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37ce3730 0x55de37ceac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff054bb6640 1 -- 172.21.15.44:0/1611907053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 msgr2=0x55de37ced960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff054bb6640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff054bb6640 1 -- 172.21.15.44:0/1611907053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501a2f40 con 0x55de37cedfd0 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff054bb6640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 0x55de37ceb180 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff04801d850 tx=0x7ff048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c716ecbc47c7ff0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff02f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1611907053 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04801e820 con 0x55de37cedfd0 2024-09-13T17:36:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1611907053 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff048024070 con 0x55de37cedfd0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de3793a4e0 con 0x55de37cedfd0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1611907053 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff04801e980 con 0x55de37cedfd0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff059419740 10 client.18603 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de37c45620 con 0x55de37cedfd0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1611907053 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de3793a4e0 con 0x55de37cedfd0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.252+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1611907053 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff048029bd0 con 0x55de37cedfd0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff059419740 10 client.18603 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff059419740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff059419740 20 client.18603 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff059419740 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37cfda70 0x55de37cffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff059419740 4 client.18603 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de3794b7e0 con 0x55de37cfda70 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37cfda70 0x55de37cffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37cfda70 0x55de37cffe90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff11fa8d6e6cad2d server_cookie=66ba4b73067759dc in_seq=0 out_seq=0 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.253+0000 7ff02f7fe640 10 client.18603 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1611907053 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55de3794b7e0 con 0x55de37cfda70 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff02f7fe640 10 client.18603 handle_command_reply: tid=0 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 client.18603 shutdown 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 msgr2=0x55de37ceb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 0x55de37ceb180 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff04801d850 tx=0x7ff048004a40 comp rx=0 tx=0).stop 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 shutdown_connections 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37cfda70 0x55de37cffe90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37ce3730 0x55de37ceac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37cedfd0 0x55de37ceb180 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 --2- 172.21.15.44:0/1611907053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 >> 172.21.15.44:0/1611907053 conn(0x55de37cdf380 msgr2=0x55de37ce50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff02f7fe640 0 client.18603 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 shutdown_connections 2024-09-13T17:36:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 1 -- 172.21.15.44:0/1611907053 wait complete. 2024-09-13T17:36:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.254+0000 7ff059419740 20 client.18603 trim_cache size 0 max 16384 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.268+0000 7ff059419740 1 Processor -- start 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 -- start start 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37cf4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37ce2b90 con 0x55de37ce3730 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37ce3ca0 con 0x55de37ce2e10 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff059419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37ce4580 con 0x55de37cedfd0 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37cf4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37cf4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40748/0 (socket says 172.21.15.44:40748) 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 -- 172.21.15.44:0/1185490411 learned_addr learned my addr 172.21.15.44:0/1185490411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 -- 172.21.15.44:0/1185490411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 msgr2=0x55de37cf4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 -- 172.21.15.44:0/1185490411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 msgr2=0x55de37ced470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 -- 172.21.15.44:0/1185490411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de37be7790 con 0x55de37ce3730 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff047fff640 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37cf4a10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff050197720 tx=0x7ff038013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cede6c1e1da1ff5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff046ffd640 1 -- 172.21.15.44:0/1185490411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038018070 con 0x55de37ce3730 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff046ffd640 1 -- 172.21.15.44:0/1185490411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0380044b0 con 0x55de37ce3730 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.269+0000 7ff046ffd640 1 -- 172.21.15.44:0/1185490411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380047a0 con 0x55de37ce3730 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 -- 172.21.15.44:0/1185490411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 msgr2=0x55de37cf4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37cf4a10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff050197720 tx=0x7ff038013830 comp rx=0 tx=0).stop 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 -- 172.21.15.44:0/1185490411 shutdown_connections 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37cf4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce3730 0x55de37cf4a10 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 --2- 172.21.15.44:0/1185490411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce2e10 0x55de37ced470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 -- 172.21.15.44:0/1185490411 >> 172.21.15.44:0/1185490411 conn(0x55de37cdf380 msgr2=0x55de37ce5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 -- 172.21.15.44:0/1185490411 shutdown_connections 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 -- 172.21.15.44:0/1185490411 wait complete. 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.270+0000 7ff059419740 1 -- start start 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 Processor -- start 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 0x55de37cf4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce3730 0x55de37ce6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37ce73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37cf4f60 con 0x55de37ce2e10 2024-09-13T17:36:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37cf50d0 con 0x55de37ce3730 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de37cf5290 con 0x55de37cedfd0 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff059419740 10 client.0 _ll_register_callbacks cb 0x7ff04801d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 0x55de37cf4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 0x55de37cf4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40752/0 (socket says 172.21.15.44:40752) 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 -- 172.21.15.44:0/1494565726 learned_addr learned my addr 172.21.15.44:0/1494565726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff054bb6640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37ce73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff047fff640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce3730 0x55de37ce6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 -- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 msgr2=0x55de37ce73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37ce73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 -- 172.21.15.44:0/1494565726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce3730 msgr2=0x55de37ce6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce3730 0x55de37ce6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 -- 172.21.15.44:0/1494565726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0380098b0 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff04ffff640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 0x55de37cf4a20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff0400f6210 tx=0x7ff040009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4d4746c5d16329 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff02f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.271+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1494565726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400f1590 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1494565726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff0400bb430 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0400ba870 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1494565726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0400ba690 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 10 client.18609 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de37c0cea0 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1494565726 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0400d9030 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1494565726 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0400eb5f0 con 0x55de37ce2e10 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 10 client.18609 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 20 client.18609 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37d54a80 0x55de37d56ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 4 client.18609 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de3799ee30 con 0x55de37d54a80 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.272+0000 7ff047fff640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37d54a80 0x55de37d56ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff047fff640 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37d54a80 0x55de37d56ea0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5485d81bec482964 server_cookie=8d3a8a3e47ac0c54 in_seq=0 out_seq=0 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff02f7fe640 10 client.18609 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff02f7fe640 1 -- 172.21.15.44:0/1494565726 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55de3799ee30 con 0x55de37d54a80 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff02f7fe640 10 client.18609 handle_command_reply: tid=0 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 client.18609 shutdown 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 msgr2=0x55de37cf4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 0x55de37cf4a20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff0400f6210 tx=0x7ff040009260 comp rx=0 tx=0).stop 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 shutdown_connections 2024-09-13T17:36:03.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55de37d54a80 0x55de37d56ea0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de37cedfd0 0x55de37ce73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55de37ce3730 0x55de37ce6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff02f7fe640 0 client.18609 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 --2- 172.21.15.44:0/1494565726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de37ce2e10 0x55de37cf4a20 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 >> 172.21.15.44:0/1494565726 conn(0x55de37cdf380 msgr2=0x55de37ce9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 shutdown_connections 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 1 -- 172.21.15.44:0/1494565726 wait complete. 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.273+0000 7ff059419740 20 client.18609 trim_cache size 0 max 16384 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:36:03.283 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: "b26990a2-e23b-4437-b16c-ae1b196923f0": { 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "b26990a2-e23b-4437-b16c-ae1b196923f0", 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:36:03.284 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff03c0a1200 msgr2=0x7ff03c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff03c0a1200 0x7ff03c0a36c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff0480096e0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 msgr2=0x7ff050196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050196180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff040006010 tx=0x7ff040005620 comp rx=0 tx=0).stop 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 shutdown_connections 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff03c0a1200 0x7ff03c0a36c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05010a350 0x7ff05019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff050101800 0x7ff0501966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 --2- 172.21.15.44:0/4174734202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff050100e50 0x7ff050196180 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:03.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 >> 172.21.15.44:0/4174734202 conn(0x7ff0500fa930 msgr2=0x7ff0500fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:03.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 shutdown_connections 2024-09-13T17:36:03.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:03.274+0000 7ff056619640 1 -- 172.21.15.44:0/4174734202 wait complete. 2024-09-13T17:36:03.287 INFO:tasks.cephfs.filesystem:scrub status for tag:b26990a2-e23b-4437-b16c-ae1b196923f0 - {'path': '/', 'tag': 'b26990a2-e23b-4437-b16c-ae1b196923f0', 'options': 'recursive,force'} 2024-09-13T17:36:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:04 smithi044 ceph-mon[31175]: pgmap v289: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:04 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:36:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:04 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:04 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:36:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:04 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:04.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:04 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:36:04.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:04 smithi070 ceph-mon[34502]: pgmap v289: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:04.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:04 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:36:04.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:04 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:04.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:04 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:36:04.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:04 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:04.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:04 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:36:04.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:04 smithi100 ceph-mon[35055]: pgmap v289: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:04.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:04 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:36:04.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:04 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:04.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:04 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:36:04.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:04 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:04.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:04 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:36:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 -- 172.21.15.44:0/2905188951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190104ce0 msgr2=0x7f11901050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 --2- 172.21.15.44:0/2905188951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190104ce0 0x7f11901050c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002a410 comp rx=0 tx=0).stop 2024-09-13T17:36:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 -- 172.21.15.44:0/2905188951 shutdown_connections 2024-09-13T17:36:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 --2- 172.21.15.44:0/2905188951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11900681d0 0x7f119010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 --2- 172.21.15.44:0/2905188951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1190067830 0x7f1190067c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 --2- 172.21.15.44:0/2905188951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190104ce0 0x7f11901050c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 -- 172.21.15.44:0/2905188951 >> 172.21.15.44:0/2905188951 conn(0x7f1190074940 msgr2=0x7f1190074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 -- 172.21.15.44:0/2905188951 shutdown_connections 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.806+0000 7f1195072640 1 -- 172.21.15.44:0/2905188951 wait complete. 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 Processor -- start 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 -- start start 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 0x7f1190196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11900681d0 0x7f1190196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1190104ce0 0x7f119019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019b180 con 0x7f1190067830 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019b2f0 con 0x7f11900681d0 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.807+0000 7f1195072640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019b4b0 con 0x7f1190104ce0 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 0x7f1190196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 0x7f1190196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40782/0 (socket says 172.21.15.44:40782) 2024-09-13T17:36:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 -- 172.21.15.44:0/2294733045 learned_addr learned my addr 172.21.15.44:0/2294733045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118e59c640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11900681d0 0x7f1190196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118f59e640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1190104ce0 0x7f119019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 -- 172.21.15.44:0/2294733045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1190104ce0 msgr2=0x7f119019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1190104ce0 0x7f119019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 -- 172.21.15.44:0/2294733045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11900681d0 msgr2=0x7f1190196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11900681d0 0x7f1190196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 -- 172.21.15.44:0/2294733045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188009840 con 0x7f1190067830 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f118ed9d640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 0x7f1190196250 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f11800057c0 tx=0x7f1180032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f1177fff640 1 -- 172.21.15.44:0/2294733045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11800044d0 con 0x7f1190067830 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f1177fff640 1 -- 172.21.15.44:0/2294733045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1180032e10 con 0x7f1190067830 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1180009560 con 0x7f1190067830 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.808+0000 7f1177fff640 1 -- 172.21.15.44:0/2294733045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180003850 con 0x7f1190067830 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.809+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901a2e30 con 0x7f1190067830 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.811+0000 7f1177fff640 1 -- 172.21.15.44:0/2294733045 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1180004850 con 0x7f1190067830 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 Processor -- start 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 -- start start 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffec50 0x55d6f4008fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3f69c30 con 0x55d6f3ffe330 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3f673c0 con 0x55d6f3ffec50 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1197e71740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3f62cb0 con 0x55d6f40094f0 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40784/0 (socket says 172.21.15.44:40784) 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 -- 172.21.15.44:0/3494637922 learned_addr learned my addr 172.21.15.44:0/3494637922 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f1177fff640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11780a1200 0x7f11780a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118e59c640 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118ed9d640 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffec50 0x55d6f4008fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 -- 172.21.15.44:0/3494637922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 msgr2=0x55d6f400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 -- 172.21.15.44:0/3494637922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffec50 msgr2=0x55d6f4008fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.812+0000 7f118f59e640 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffec50 0x55d6f4008fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f118f59e640 1 -- 172.21.15.44:0/3494637922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6f3f09910 con 0x55d6f3ffe330 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f118e59c640 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f118f59e640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11780a1200 0x7f11780a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f1177fff640 1 -- 172.21.15.44:0/2294733045 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f11800edff0 con 0x7f1190067830 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f118f59e640 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffe710 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f1188010680 tx=0x7f1188010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c266c146c8c875 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f11757fa640 1 -- 172.21.15.44:0/3494637922 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188013b30 con 0x55d6f3ffe330 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f11757fa640 1 -- 172.21.15.44:0/3494637922 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1188013c90 con 0x55d6f3ffe330 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f118f59e640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11780a1200 0x7f11780a36c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1188021000 tx=0x7f118801f000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.813+0000 7f11757fa640 1 -- 172.21.15.44:0/3494637922 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880132d0 con 0x55d6f3ffe330 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 -- 172.21.15.44:0/3494637922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 msgr2=0x55d6f3ffe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffe710 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f1188010680 tx=0x7f1188010b40 comp rx=0 tx=0).stop 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 -- 172.21.15.44:0/3494637922 shutdown_connections 2024-09-13T17:36:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffec50 0x55d6f4008fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 --2- 172.21.15.44:0/3494637922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffe710 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 -- 172.21.15.44:0/3494637922 >> 172.21.15.44:0/3494637922 conn(0x55d6f3ffa8c0 msgr2=0x55d6f3ffaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 -- 172.21.15.44:0/3494637922 shutdown_connections 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 -- 172.21.15.44:0/3494637922 wait complete. 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 -- start start 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 Processor -- start 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.814+0000 7f1197e71740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f4008ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f3ffec50 0x55d6f4006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 0x55d6f4006770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3f20e80 con 0x55d6f40094f0 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3f20b20 con 0x55d6f3ffe330 2024-09-13T17:36:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3ee6260 con 0x55d6f3ffec50 2024-09-13T17:36:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 10 client.0 _ll_register_callbacks cb 0x55d6f3f1bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 0x55d6f4006770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 0x55d6f4006770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40792/0 (socket says 172.21.15.44:40792) 2024-09-13T17:36:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 -- 172.21.15.44:0/3690566920 learned_addr learned my addr 172.21.15.44:0/3690566920 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118ed9d640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f4008ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118e59c640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f3ffec50 0x55d6f4006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 -- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f3ffec50 msgr2=0x55d6f4006180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f3ffec50 0x55d6f4006180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 -- 172.21.15.44:0/3690566920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 msgr2=0x55d6f4008ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f4008ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 -- 172.21.15.44:0/3690566920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1188009dd0 con 0x55d6f40094f0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f118f59e640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 0x55d6f4006770 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f1188026a70 tx=0x7f11880049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a02599bd8967d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1172ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1172ff5640 1 -- 172.21.15.44:0/3690566920 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188033070 con 0x55d6f40094f0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1172ff5640 1 -- 172.21.15.44:0/3690566920 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f1188013070 con 0x55d6f40094f0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.815+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6f3f8b680 con 0x55d6f40094f0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1172ff5640 1 -- 172.21.15.44:0/3690566920 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1188013230 con 0x55d6f40094f0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 10 client.18621 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d6f3f61330 con 0x55d6f40094f0 2024-09-13T17:36:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1172ff5640 1 -- 172.21.15.44:0/3690566920 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d6f3f8b680 con 0x55d6f40094f0 2024-09-13T17:36:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1172ff5640 1 -- 172.21.15.44:0/3690566920 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1188036020 con 0x55d6f40094f0 2024-09-13T17:36:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 10 client.18621 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 10 client.18621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 20 client.18621 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4018ea0 0x55d6f401b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 4 client.18621 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6f3f565d0 con 0x55d6f4018ea0 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.816+0000 7f118ed9d640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4018ea0 0x55d6f401b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f1172ff5640 10 client.18621 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f118ed9d640 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4018ea0 0x55d6f401b2c0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21bd160582edd3ff server_cookie=38567c170a8060bc in_seq=0 out_seq=0 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f1172ff5640 1 -- 172.21.15.44:0/3690566920 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d6f3f565d0 con 0x55d6f4018ea0 2024-09-13T17:36:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f1172ff5640 10 client.18621 handle_command_reply: tid=0 2024-09-13T17:36:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f1197e71740 1 client.18621 shutdown 2024-09-13T17:36:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 msgr2=0x55d6f4006770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.817+0000 7f1197e71740 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 0x55d6f4006770 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f1188026a70 tx=0x7f11880049e0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 shutdown_connections 2024-09-13T17:36:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4018ea0 0x55d6f401b2c0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f3ffec50 0x55d6f4006180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f40094f0 0x55d6f4006770 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 --2- 172.21.15.44:0/3690566920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f4008ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1172ff5640 0 client.18621 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 >> 172.21.15.44:0/3690566920 conn(0x55d6f3ffa8c0 msgr2=0x55d6f3ff9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 shutdown_connections 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 1 -- 172.21.15.44:0/3690566920 wait complete. 2024-09-13T17:36:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.818+0000 7f1197e71740 20 client.18621 trim_cache size 0 max 16384 2024-09-13T17:36:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.830+0000 7f1197e71740 1 Processor -- start 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.830+0000 7f1197e71740 1 -- start start 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1197e71740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1197e71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f40105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1197e71740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1197e71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f3fff890 con 0x55d6f3ffec50 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1197e71740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f4010b10 con 0x55d6f3ffe330 2024-09-13T17:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1197e71740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f4006d70 con 0x55d6f40094f0 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f40105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f40105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40804/0 (socket says 172.21.15.44:40804) 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 -- 172.21.15.44:0/815387015 learned_addr learned my addr 172.21.15.44:0/815387015 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118ed9d640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118f59e640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 -- 172.21.15.44:0/815387015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 msgr2=0x55d6f40089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 -- 172.21.15.44:0/815387015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 msgr2=0x55d6f3ffa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 -- 172.21.15.44:0/815387015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6f3f03300 con 0x55d6f3ffec50 2024-09-13T17:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118ed9d640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118f59e640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f118e59c640 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f40105d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55d6f3ffb9b0 tx=0x7f1184009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfe7d80bbcbc748 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1175ffb640 1 -- 172.21.15.44:0/815387015 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184010e00 con 0x55d6f3ffec50 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1175ffb640 1 -- 172.21.15.44:0/815387015 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1184018040 con 0x55d6f3ffec50 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.831+0000 7f1175ffb640 1 -- 172.21.15.44:0/815387015 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118401c3a0 con 0x55d6f3ffec50 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 -- 172.21.15.44:0/815387015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 msgr2=0x55d6f40105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f40105d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55d6f3ffb9b0 tx=0x7f1184009f90 comp rx=0 tx=0).stop 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 -- 172.21.15.44:0/815387015 shutdown_connections 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f40105d0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 --2- 172.21.15.44:0/815387015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f3ffa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 -- 172.21.15.44:0/815387015 >> 172.21.15.44:0/815387015 conn(0x55d6f3ffa8c0 msgr2=0x55d6f4004a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 -- 172.21.15.44:0/815387015 shutdown_connections 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 -- 172.21.15.44:0/815387015 wait complete. 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 -- start start 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.832+0000 7f1197e71740 1 Processor -- start 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f406c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f4067eb0 con 0x55d6f3ffec50 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f4068020 con 0x55d6f3ffe330 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6f40681e0 con 0x55d6f40094f0 2024-09-13T17:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1197e71740 10 client.0 _ll_register_callbacks cb 0x7f1188009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40808/0 (socket says 172.21.15.44:40808) 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 -- 172.21.15.44:0/772776721 learned_addr learned my addr 172.21.15.44:0/772776721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118ed9d640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f406c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118f59e640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 -- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 msgr2=0x55d6f40678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 -- 172.21.15.44:0/772776721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 msgr2=0x55d6f406c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f406c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 -- 172.21.15.44:0/772776721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840097e0 con 0x55d6f3ffec50 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f118e59c640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f406c7f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1184003a30 tx=0x7f11840093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b0d393a3636106 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.833+0000 7f1172ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1172ff5640 1 -- 172.21.15.44:0/772776721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184004050 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1172ff5640 1 -- 172.21.15.44:0/772776721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f11840041b0 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1188013c90 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1172ff5640 1 -- 172.21.15.44:0/772776721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1184003ed0 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 10 client.18627 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d6f3f288e0 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1172ff5640 1 -- 172.21.15.44:0/772776721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1188013c90 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1172ff5640 1 -- 172.21.15.44:0/772776721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1184024070 con 0x55d6f3ffec50 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 10 client.18627 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 10 client.18627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 20 client.18627 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4075120 0x55d6f4077540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 4 client.18627 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:36:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.834+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d6f3c56300 con 0x55d6f4075120 2024-09-13T17:36:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.835+0000 7f118ed9d640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4075120 0x55d6f4077540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.835+0000 7f118ed9d640 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4075120 0x55d6f4077540 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6204e96a58de636d server_cookie=bc69051db3fca320 in_seq=0 out_seq=0 2024-09-13T17:36:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.835+0000 7f1172ff5640 10 client.18627 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1172ff5640 1 -- 172.21.15.44:0/772776721 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d6f3c56300 con 0x55d6f4075120 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1172ff5640 10 client.18627 handle_command_reply: tid=0 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 client.18627 shutdown 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 msgr2=0x55d6f406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f406c7f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1184003a30 tx=0x7f11840093f0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 shutdown_connections 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d6f4075120 0x55d6f4077540 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6f40094f0 0x55d6f40678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1172ff5640 0 client.18627 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6f3ffec50 0x55d6f406c7f0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 --2- 172.21.15.44:0/772776721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d6f3ffe330 0x55d6f406c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 >> 172.21.15.44:0/772776721 conn(0x55d6f3ffa8c0 msgr2=0x55d6f40058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 shutdown_connections 2024-09-13T17:36:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 1 -- 172.21.15.44:0/772776721 wait complete. 2024-09-13T17:36:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.848+0000 7f1197e71740 20 client.18627 trim_cache size 0 max 16384 2024-09-13T17:36:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:36:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:36:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:36:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11780a1200 msgr2=0x7f11780a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11780a1200 0x7f11780a36c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1188021000 tx=0x7f118801f000 comp rx=0 tx=0).stop 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 msgr2=0x7f1190196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 0x7f1190196250 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f11800057c0 tx=0x7f1180032c20 comp rx=0 tx=0).stop 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 shutdown_connections 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f11780a1200 0x7f11780a36c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1190104ce0 0x7f119019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11900681d0 0x7f1190196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 --2- 172.21.15.44:0/2294733045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1190067830 0x7f1190196250 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 >> 172.21.15.44:0/2294733045 conn(0x7f1190074940 msgr2=0x7f11900fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 shutdown_connections 2024-09-13T17:36:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:04.849+0000 7f1195072640 1 -- 172.21.15.44:0/2294733045 wait complete. 2024-09-13T17:36:05.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:05 smithi044 ceph-mon[31175]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T17:36:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:05 smithi070 ceph-mon[34502]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T17:36:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:05 smithi100 ceph-mon[35055]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T17:36:06.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:06 smithi044 ceph-mon[31175]: pgmap v291: 129 pgs: 129 active+clean; 5.6 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-13T17:36:06.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:06 smithi070 ceph-mon[34502]: pgmap v291: 129 pgs: 129 active+clean; 5.6 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-13T17:36:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:06 smithi100 ceph-mon[35055]: pgmap v291: 129 pgs: 129 active+clean; 5.6 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-13T17:36:07.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:07 smithi044 ceph-mon[31175]: pgmap v292: 129 pgs: 129 active+clean; 5.6 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-13T17:36:07.700 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:07 smithi070 ceph-mon[34502]: pgmap v292: 129 pgs: 129 active+clean; 5.6 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-13T17:36:07.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:07 smithi100 ceph-mon[35055]: pgmap v292: 129 pgs: 129 active+clean; 5.6 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-13T17:36:09.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:09 smithi044 ceph-mon[31175]: pgmap v293: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-09-13T17:36:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:09 smithi070 ceph-mon[34502]: pgmap v293: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-09-13T17:36:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:09 smithi100 ceph-mon[35055]: pgmap v293: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-09-13T17:36:12.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:12 smithi044 ceph-mon[31175]: pgmap v294: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-09-13T17:36:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:12 smithi044 ceph-mon[31175]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T17:36:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:12 smithi070 ceph-mon[34502]: pgmap v294: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-09-13T17:36:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:12 smithi070 ceph-mon[34502]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T17:36:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:12 smithi100 ceph-mon[35055]: pgmap v294: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-09-13T17:36:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:12 smithi100 ceph-mon[35055]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T17:36:14.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:14 smithi044 ceph-mon[31175]: pgmap v296: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-13T17:36:14.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:14 smithi070 ceph-mon[34502]: pgmap v296: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-13T17:36:14.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:14 smithi100 ceph-mon[35055]: pgmap v296: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-13T17:36:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:16 smithi044 ceph-mon[31175]: pgmap v297: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:36:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:16 smithi070 ceph-mon[34502]: pgmap v297: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:36:16.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:16 smithi100 ceph-mon[35055]: pgmap v297: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 -- 172.21.15.44:0/1124849083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 msgr2=0x7f00380fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 --2- 172.21.15.44:0/1124849083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f00380fe800 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0028009930 tx=0x7f002802a420 comp rx=0 tx=0).stop 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 -- 172.21.15.44:0/1124849083 shutdown_connections 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 --2- 172.21.15.44:0/1124849083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0038105090 0x7f003810cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 --2- 172.21.15.44:0/1124849083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00380fedd0 0x7f0038104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 --2- 172.21.15.44:0/1124849083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f00380fe800 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 -- 172.21.15.44:0/1124849083 >> 172.21.15.44:0/1124849083 conn(0x7f0038075340 msgr2=0x7f00380fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 -- 172.21.15.44:0/1124849083 shutdown_connections 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 -- 172.21.15.44:0/1124849083 wait complete. 2024-09-13T17:36:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 Processor -- start 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.950+0000 7f003e896640 1 -- start start 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003e896640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f0038196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003e896640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0038105090 0x7f00381966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003e896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003819aac0 0x7f003819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003e896640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003819b4b0 con 0x7f00380fe420 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003e896640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003819b620 con 0x7f003819aac0 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003e896640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003819b790 con 0x7f0038105090 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f0038196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f0038196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34262/0 (socket says 172.21.15.44:34262) 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 -- 172.21.15.44:0/531224633 learned_addr learned my addr 172.21.15.44:0/531224633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f002ffff640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0038105090 0x7f00381966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f003ce33640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003819aac0 0x7f003819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 -- 172.21.15.44:0/531224633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0038105090 msgr2=0x7f00381966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0038105090 0x7f00381966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 -- 172.21.15.44:0/531224633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003819aac0 msgr2=0x7f003819af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003819aac0 0x7f003819af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 -- 172.21.15.44:0/531224633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028009560 con 0x7f00380fe420 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.951+0000 7f0037fff640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f0038196190 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0028005ab0 tx=0x7f00280329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.952+0000 7f0035ffb640 1 -- 172.21.15.44:0/531224633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028032df0 con 0x7f00380fe420 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.952+0000 7f0035ffb640 1 -- 172.21.15.44:0/531224633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0028005490 con 0x7f00380fe420 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.952+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00381a2d70 con 0x7f00380fe420 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.952+0000 7f0035ffb640 1 -- 172.21.15.44:0/531224633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00280035f0 con 0x7f00380fe420 2024-09-13T17:36:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.952+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00381a3190 con 0x7f00380fe420 2024-09-13T17:36:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.954+0000 7f0035ffb640 1 -- 172.21.15.44:0/531224633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f00280046c0 con 0x7f00380fe420 2024-09-13T17:36:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 Processor -- start 2024-09-13T17:36:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 -- start start 2024-09-13T17:36:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e87710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87c50 0x5654a9e91fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9df2c30 con 0x5654a9e924f0 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9df03c0 con 0x5654a9e87c50 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f00411fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9debcb0 con 0x5654a9e87330 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f002ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f002ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34270/0 (socket says 172.21.15.44:34270) 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f002ffff640 1 -- 172.21.15.44:0/4216499195 learned_addr learned my addr 172.21.15.44:0/4216499195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f003ce33640 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e87710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f0037fff640 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87c50 0x5654a9e91fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f0035ffb640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00240a1200 0x7f00240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.955+0000 7f002ffff640 1 -- 172.21.15.44:0/4216499195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 msgr2=0x5654a9e87710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002ffff640 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e87710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002ffff640 1 -- 172.21.15.44:0/4216499195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87c50 msgr2=0x5654a9e91fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002ffff640 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87c50 0x5654a9e91fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002ffff640 1 -- 172.21.15.44:0/4216499195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654a9d92910 con 0x5654a9e924f0 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f003ce33640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00240a1200 0x7f00240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f0035ffb640 1 -- 172.21.15.44:0/531224633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f00280edb50 con 0x7f00380fe420 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002ffff640 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e948e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0020009c60 tx=0x7f0020011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61f255d75225d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002e7fc640 1 -- 172.21.15.44:0/4216499195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020004230 con 0x5654a9e924f0 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002e7fc640 1 -- 172.21.15.44:0/4216499195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0020004390 con 0x5654a9e924f0 2024-09-13T17:36:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.956+0000 7f002e7fc640 1 -- 172.21.15.44:0/4216499195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020004c40 con 0x5654a9e924f0 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 -- 172.21.15.44:0/4216499195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 msgr2=0x5654a9e948e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f003ce33640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00240a1200 0x7f00240a36c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0030005fd0 tx=0x7f0030014040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e948e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0020009c60 tx=0x7f0020011810 comp rx=0 tx=0).stop 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 -- 172.21.15.44:0/4216499195 shutdown_connections 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e948e0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87c50 0x5654a9e91fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 --2- 172.21.15.44:0/4216499195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e87710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 -- 172.21.15.44:0/4216499195 >> 172.21.15.44:0/4216499195 conn(0x5654a9e838c0 msgr2=0x5654a9e83ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 -- 172.21.15.44:0/4216499195 shutdown_connections 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 -- 172.21.15.44:0/4216499195 wait complete. 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 -- start start 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.957+0000 7f00411fd740 1 Processor -- start 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e91d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87c50 0x5654a9e8f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e8f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9da9e80 con 0x5654a9e924f0 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9da9b20 con 0x5654a9e87330 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9d6f260 con 0x5654a9e87c50 2024-09-13T17:36:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00411fd740 10 client.0 _ll_register_callbacks cb 0x5654a9da4df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e8f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f002ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87c50 0x5654a9e8f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e8f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34280/0 (socket says 172.21.15.44:34280) 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 -- 172.21.15.44:0/3711347088 learned_addr learned my addr 172.21.15.44:0/3711347088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f0037fff640 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e91d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 -- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87c50 msgr2=0x5654a9e8f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87c50 0x5654a9e8f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 -- 172.21.15.44:0/3711347088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 msgr2=0x5654a9e91d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e91d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 -- 172.21.15.44:0/3711347088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0020009890 con 0x5654a9e924f0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f003ce33640 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e8f590 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0030010890 tx=0x7f0030009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1ad762b1fc66ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.958+0000 7f00177fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00177fe640 1 -- 172.21.15.44:0/3711347088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003001c890 con 0x5654a9e924f0 2024-09-13T17:36:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00177fe640 1 -- 172.21.15.44:0/3711347088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f003001c9f0 con 0x5654a9e924f0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654a9e14680 con 0x5654a9e924f0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00177fe640 1 -- 172.21.15.44:0/3711347088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f003001cbb0 con 0x5654a9e924f0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 10 client.18639 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5654a9dea330 con 0x5654a9e924f0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00177fe640 1 -- 172.21.15.44:0/3711347088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5654a9e14680 con 0x5654a9e924f0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00177fe640 1 -- 172.21.15.44:0/3711347088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f003001b3e0 con 0x5654a9e924f0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 10 client.18639 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 10 client.18639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 20 client.18639 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ea1e60 0x5654a9ea4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 4 client.18639 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.959+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654a9ddf5d0 con 0x5654a9ea1e60 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.960+0000 7f0037fff640 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ea1e60 0x5654a9ea4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.960+0000 7f0037fff640 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ea1e60 0x5654a9ea4250 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0a5092b6384d06 server_cookie=fe235236894bbfb8 in_seq=0 out_seq=0 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.960+0000 7f00177fe640 10 client.18639 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.960+0000 7f00177fe640 1 -- 172.21.15.44:0/3711347088 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5654a9ddf5d0 con 0x5654a9ea1e60 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.960+0000 7f00177fe640 10 client.18639 handle_command_reply: tid=0 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.960+0000 7f00411fd740 1 client.18639 shutdown 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 msgr2=0x5654a9e8f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e8f590 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0030010890 tx=0x7f0030009500 comp rx=0 tx=0).stop 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 shutdown_connections 2024-09-13T17:36:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ea1e60 0x5654a9ea4250 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87c50 0x5654a9e8f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e924f0 0x5654a9e8f590 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00177fe640 0 client.18639 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 --2- 172.21.15.44:0/3711347088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e91d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 >> 172.21.15.44:0/3711347088 conn(0x5654a9e838c0 msgr2=0x5654a9e82b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 shutdown_connections 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 1 -- 172.21.15.44:0/3711347088 wait complete. 2024-09-13T17:36:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.961+0000 7f00411fd740 20 client.18639 trim_cache size 0 max 16384 2024-09-13T17:36:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 Processor -- start 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 -- start start 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e8c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e924f0 0x5654a9e8c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9e92360 con 0x5654a9e87c50 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9e870d0 con 0x5654a9e924f0 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f00411fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9e94fa0 con 0x5654a9e87330 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e8c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e8c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34290/0 (socket says 172.21.15.44:34290) 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 -- 172.21.15.44:0/297586421 learned_addr learned my addr 172.21.15.44:0/297586421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f0037fff640 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f003ce33640 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e924f0 0x5654a9e8c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 -- 172.21.15.44:0/297586421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 msgr2=0x5654a9e918e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e918e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 -- 172.21.15.44:0/297586421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e924f0 msgr2=0x5654a9e8c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e924f0 0x5654a9e8c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 -- 172.21.15.44:0/297586421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654a9d8c300 con 0x5654a9e87c50 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f003ce33640 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e924f0 0x5654a9e8c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.974+0000 7f002ffff640 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e8c2d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x5654a9e849b0 tx=0x7f00200048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f68b8972abc4ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f002effd640 1 -- 172.21.15.44:0/297586421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020024420 con 0x5654a9e87c50 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f002effd640 1 -- 172.21.15.44:0/297586421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0020024a10 con 0x5654a9e87c50 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f002effd640 1 -- 172.21.15.44:0/297586421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200199b0 con 0x5654a9e87c50 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 -- 172.21.15.44:0/297586421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 msgr2=0x5654a9e8c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e8c2d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x5654a9e849b0 tx=0x7f00200048e0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 -- 172.21.15.44:0/297586421 shutdown_connections 2024-09-13T17:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e924f0 0x5654a9e8c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e8c2d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 --2- 172.21.15.44:0/297586421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e87330 0x5654a9e918e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 -- 172.21.15.44:0/297586421 >> 172.21.15.44:0/297586421 conn(0x5654a9e838c0 msgr2=0x5654a9e8d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 -- 172.21.15.44:0/297586421 shutdown_connections 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 -- 172.21.15.44:0/297586421 wait complete. 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.975+0000 7f00411fd740 1 -- start start 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 Processor -- start 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e99560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e97a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e924f0 0x5654a9e97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9e98630 con 0x5654a9e87c50 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9e987a0 con 0x5654a9e87330 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a9ef8340 con 0x5654a9e924f0 2024-09-13T17:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f00411fd740 10 client.0 _ll_register_callbacks cb 0x7f00200097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e97a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e97a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34306/0 (socket says 172.21.15.44:34306) 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 -- 172.21.15.44:0/2023138541 learned_addr learned my addr 172.21.15.44:0/2023138541 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f0037fff640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e99560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f003ce33640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e924f0 0x5654a9e97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 -- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e924f0 msgr2=0x5654a9e97f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e924f0 0x5654a9e97f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 -- 172.21.15.44:0/2023138541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 msgr2=0x5654a9e99560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.976+0000 7f002ffff640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e99560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f002ffff640 1 -- 172.21.15.44:0/2023138541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654a9d92910 con 0x5654a9e87c50 2024-09-13T17:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f002ffff640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e97a20 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x5654a9e83830 tx=0x7f0020004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97ff344787b1a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00177fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00177fe640 1 -- 172.21.15.44:0/2023138541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020024620 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00177fe640 1 -- 172.21.15.44:0/2023138541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f0020024780 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0020004390 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00177fe640 1 -- 172.21.15.44:0/2023138541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0020024e50 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 10 client.18645 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5654a9db18e0 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00177fe640 1 -- 172.21.15.44:0/2023138541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0020004390 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00177fe640 1 -- 172.21.15.44:0/2023138541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f002002d7b0 con 0x5654a9e87c50 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 10 client.18645 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 20 client.18645 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ef9470 0x5654a9efb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 4 client.18645 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:36:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.977+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5654a9adf300 con 0x5654a9ef9470 2024-09-13T17:36:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.978+0000 7f0037fff640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ef9470 0x5654a9efb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.978+0000 7f0037fff640 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ef9470 0x5654a9efb860 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c845566f5faa9a server_cookie=e078069e4e187c0e in_seq=0 out_seq=0 2024-09-13T17:36:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.978+0000 7f00177fe640 10 client.18645 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00177fe640 1 -- 172.21.15.44:0/2023138541 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5654a9adf300 con 0x5654a9ef9470 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00177fe640 10 client.18645 handle_command_reply: tid=0 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 client.18645 shutdown 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 msgr2=0x5654a9e97a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e97a20 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x5654a9e83830 tx=0x7f0020004830 comp rx=0 tx=0).stop 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 shutdown_connections 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654a9ef9470 0x5654a9efb860 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654a9e924f0 0x5654a9e97f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654a9e87c50 0x5654a9e97a20 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00177fe640 0 client.18645 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 --2- 172.21.15.44:0/2023138541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654a9e87330 0x5654a9e99560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 >> 172.21.15.44:0/2023138541 conn(0x5654a9e838c0 msgr2=0x5654a9e88d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 shutdown_connections 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 1 -- 172.21.15.44:0/2023138541 wait complete. 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.992+0000 7f00411fd740 20 client.18645 trim_cache size 0 max 16384 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:36:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00240a1200 msgr2=0x7f00240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00240a1200 0x7f00240a36c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0030005fd0 tx=0x7f0030014040 comp rx=0 tx=0).stop 2024-09-13T17:36:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 msgr2=0x7f0038196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f0038196190 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0028005ab0 tx=0x7f00280329f0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 shutdown_connections 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f00240a1200 0x7f00240a36c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0038105090 0x7f00381966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003819aac0 0x7f003819af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 --2- 172.21.15.44:0/531224633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00380fe420 0x7f0038196190 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 >> 172.21.15.44:0/531224633 conn(0x7f0038075340 msgr2=0x7f0038103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 shutdown_connections 2024-09-13T17:36:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:17.993+0000 7f003e896640 1 -- 172.21.15.44:0/531224633 wait complete. 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:437/600: open fd = 3 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:438/600: open fd = 3 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.247 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:439/600: open fd = 3 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:440/600: open fd = 3 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:441/600: open fd = 3 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:442/600: open fd = 3 2024-09-13T17:36:18.248 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:443/600: open fd = 3 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:444/600: open fd = 3 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:445/600: open fd = 3 2024-09-13T17:36:18.249 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:446/600: open fd = 3 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:447/600: open fd = 3 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:448/600: open fd = 3 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.250 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:449/600: open fd = 3 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:450/600: open fd = 3 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:451/600: open fd = 3 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.251 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:452/600: open fd = 3 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:453/600: open fd = 3 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:454/600: open fd = 3 2024-09-13T17:36:18.252 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:455/600: open fd = 3 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:456/600: open fd = 3 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:457/600: open fd = 3 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.253 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:458/600: open fd = 3 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:459/600: open fd = 3 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:460/600: open fd = 3 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:461/600: open fd = 3 2024-09-13T17:36:18.254 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:462/600: open fd = 3 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:463/600: open fd = 3 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:464/600: open fd = 3 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.255 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:465/600: open fd = 3 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:466/600: open fd = 3 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:467/600: open fd = 3 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:468/600: open fd = 3 2024-09-13T17:36:18.256 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:469/600: open fd = 3 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:470/600: open fd = 3 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:471/600: open fd = 3 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.257 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:472/600: open fd = 3 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:473/600: open fd = 3 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:474/600: open fd = 3 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:475/600: open fd = 3 2024-09-13T17:36:18.258 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:476/600: open fd = 3 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:477/600: open fd = 3 2024-09-13T17:36:18.259 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:478/600: open fd = 3 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:479/600: open fd = 3 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:480/600: open fd = 3 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.260 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:481/600: open fd = 3 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:482/600: open fd = 3 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:483/600: open fd = 3 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.261 INFO:tasks.workunit.client.0.smithi044.stdout:484/600: open fd = 3 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:485/600: open fd = 3 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.262 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:486/600: open fd = 3 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:487/600: open fd = 3 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:488/600: open fd = 3 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.263 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:489/600: open fd = 3 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:490/600: open fd = 3 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.264 INFO:tasks.workunit.client.0.smithi044.stdout:491/600: open fd = 3 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:492/600: open fd = 3 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:493/600: open fd = 3 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.265 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:494/600: open fd = 3 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:495/600: open fd = 3 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:496/600: open fd = 3 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.266 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:497/600: open fd = 3 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:498/600: open fd = 3 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:36:18.267 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:36:18.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:18 smithi044 ceph-mon[31175]: pgmap v298: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:36:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:18 smithi070 ceph-mon[34502]: pgmap v298: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:36:18.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:18 smithi100 ceph-mon[35055]: pgmap v298: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:36:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:20 smithi044 ceph-mon[31175]: pgmap v299: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-13T17:36:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:20 smithi070 ceph-mon[34502]: pgmap v299: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-13T17:36:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:20 smithi100 ceph-mon[35055]: pgmap v299: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-13T17:36:22.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:22 smithi044 ceph-mon[31175]: pgmap v300: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-13T17:36:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:22 smithi070 ceph-mon[34502]: pgmap v300: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-13T17:36:22.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:22 smithi100 ceph-mon[35055]: pgmap v300: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-13T17:36:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:24 smithi044 ceph-mon[31175]: pgmap v301: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-13T17:36:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:24 smithi070 ceph-mon[34502]: pgmap v301: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-13T17:36:24.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:24 smithi100 ceph-mon[35055]: pgmap v301: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-13T17:36:26.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:26 smithi044 ceph-mon[31175]: pgmap v302: 129 pgs: 129 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:36:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:26 smithi070 ceph-mon[34502]: pgmap v302: 129 pgs: 129 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:36:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:26 smithi100 ceph-mon[35055]: pgmap v302: 129 pgs: 129 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:36:28.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:28 smithi044 ceph-mon[31175]: pgmap v303: 129 pgs: 129 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:28.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:28 smithi070 ceph-mon[34502]: pgmap v303: 129 pgs: 129 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:28.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:28 smithi100 ceph-mon[35055]: pgmap v303: 129 pgs: 129 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:30.576 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:30 smithi100 ceph-mon[35055]: pgmap v304: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:36:30.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:30 smithi044 ceph-mon[31175]: pgmap v304: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:36:30.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:30 smithi070 ceph-mon[34502]: pgmap v304: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:36:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 -- 172.21.15.44:0/655296787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 msgr2=0x7f946c10cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 --2- 172.21.15.44:0/655296787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c10cbc0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9464009cd0 tx=0x7f946402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 -- 172.21.15.44:0/655296787 shutdown_connections 2024-09-13T17:36:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 --2- 172.21.15.44:0/655296787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f946c1008d0 0x7f946c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 --2- 172.21.15.44:0/655296787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c10cbc0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 --2- 172.21.15.44:0/655296787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946c0fff20 0x7f946c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 -- 172.21.15.44:0/655296787 >> 172.21.15.44:0/655296787 conn(0x7f946c0fb9b0 msgr2=0x7f946c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 -- 172.21.15.44:0/655296787 shutdown_connections 2024-09-13T17:36:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.092+0000 7f9473612640 1 -- 172.21.15.44:0/655296787 wait complete. 2024-09-13T17:36:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 Processor -- start 2024-09-13T17:36:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 -- start start 2024-09-13T17:36:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946c0fff20 0x7f946c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f946c073230 0x7f946c1a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1a4910 con 0x7f946c105070 2024-09-13T17:36:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1a4a80 con 0x7f946c0fff20 2024-09-13T17:36:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9473612640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1a4c40 con 0x7f946c073230 2024-09-13T17:36:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.093+0000 7f9470bad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51036/0 (socket says 172.21.15.44:51036) 2024-09-13T17:36:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 -- 172.21.15.44:0/1208703706 learned_addr learned my addr 172.21.15.44:0/1208703706 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f94713ae640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946c0fff20 0x7f946c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9471baf640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f946c073230 0x7f946c1a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 -- 172.21.15.44:0/1208703706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f946c073230 msgr2=0x7f946c1a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f946c073230 0x7f946c1a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 -- 172.21.15.44:0/1208703706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946c0fff20 msgr2=0x7f946c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946c0fff20 0x7f946c0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 -- 172.21.15.44:0/1208703706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464009900 con 0x7f946c105070 2024-09-13T17:36:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9470bad640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c072cf0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f945c00e9f0 tx=0x7f945c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f94627fc640 1 -- 172.21.15.44:0/1208703706 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0096c0 con 0x7f946c105070 2024-09-13T17:36:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f94627fc640 1 -- 172.21.15.44:0/1208703706 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f945c00ccc0 con 0x7f946c105070 2024-09-13T17:36:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c1a4f20 con 0x7f946c105070 2024-09-13T17:36:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f94627fc640 1 -- 172.21.15.44:0/1208703706 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c004e60 con 0x7f946c105070 2024-09-13T17:36:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.094+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1a5350 con 0x7f946c105070 2024-09-13T17:36:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f94627fc640 1 -- 172.21.15.44:0/1208703706 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f945c005020 con 0x7f946c105070 2024-09-13T17:36:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 Processor -- start 2024-09-13T17:36:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 -- start start 2024-09-13T17:36:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060330 0x55e1be060710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060c50 0x55e1be06afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1bdfcbc30 con 0x55e1be06b4f0 2024-09-13T17:36:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1bdfc93c0 con 0x55e1be060330 2024-09-13T17:36:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9476413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1bdfc4cb0 con 0x55e1be060c50 2024-09-13T17:36:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9470bad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9471baf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060330 0x55e1be060710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9470bad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51050/0 (socket says 172.21.15.44:51050) 2024-09-13T17:36:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f9470bad640 1 -- 172.21.15.44:0/780200846 learned_addr learned my addr 172.21.15.44:0/780200846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f94713ae640 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060c50 0x55e1be06afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.098+0000 7f94627fc640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94580a1230 0x7f94580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9470bad640 1 -- 172.21.15.44:0/780200846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060c50 msgr2=0x55e1be06afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9470bad640 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060c50 0x55e1be06afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9470bad640 1 -- 172.21.15.44:0/780200846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060330 msgr2=0x55e1be060710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9470bad640 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060330 0x55e1be060710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9470bad640 1 -- 172.21.15.44:0/780200846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1bdf6b910 con 0x55e1be06b4f0 2024-09-13T17:36:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f94713ae640 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060c50 0x55e1be06afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9471baf640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94580a1230 0x7f94580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f94627fc640 1 -- 172.21.15.44:0/1208703706 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f945c014070 con 0x7f946c105070 2024-09-13T17:36:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9470bad640 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be06d8e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f945c0ca810 tx=0x7f945c0656f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c63e081520de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f94577fe640 1 -- 172.21.15.44:0/780200846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c099da0 con 0x55e1be06b4f0 2024-09-13T17:36:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f94577fe640 1 -- 172.21.15.44:0/780200846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f945c099bc0 con 0x55e1be06b4f0 2024-09-13T17:36:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f9471baf640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94580a1230 0x7f94580a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9464005e30 tx=0x7f946403f040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.099+0000 7f94577fe640 1 -- 172.21.15.44:0/780200846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c09a960 con 0x55e1be06b4f0 2024-09-13T17:36:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 -- 172.21.15.44:0/780200846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 msgr2=0x55e1be06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be06d8e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f945c0ca810 tx=0x7f945c0656f0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 -- 172.21.15.44:0/780200846 shutdown_connections 2024-09-13T17:36:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060c50 0x55e1be06afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be06d8e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 --2- 172.21.15.44:0/780200846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060330 0x55e1be060710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 -- 172.21.15.44:0/780200846 >> 172.21.15.44:0/780200846 conn(0x55e1be05c8c0 msgr2=0x55e1be05cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 -- 172.21.15.44:0/780200846 shutdown_connections 2024-09-13T17:36:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 -- 172.21.15.44:0/780200846 wait complete. 2024-09-13T17:36:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 -- start start 2024-09-13T17:36:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.100+0000 7f9476413740 1 Processor -- start 2024-09-13T17:36:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be06aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be068780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1bdf82e80 con 0x55e1be06b4f0 2024-09-13T17:36:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1bdf82b20 con 0x55e1be060c50 2024-09-13T17:36:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1bdf48260 con 0x55e1be060330 2024-09-13T17:36:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9476413740 10 client.0 _ll_register_callbacks cb 0x55e1bdf7ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be068780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51056/0 (socket says 172.21.15.44:51056) 2024-09-13T17:36:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 -- 172.21.15.44:0/1708963873 learned_addr learned my addr 172.21.15.44:0/1708963873 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9470bad640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f94713ae640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be06aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 -- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 msgr2=0x55e1be06aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be06aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 -- 172.21.15.44:0/1708963873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 msgr2=0x55e1be0681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be0681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9471baf640 1 -- 172.21.15.44:0/1708963873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1a5350 con 0x55e1be06b4f0 2024-09-13T17:36:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f94713ae640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.101+0000 7f9470bad640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be0681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9471baf640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be068780 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9464035000 tx=0x7f9464005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a155c34d884d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9454ff9640 1 -- 172.21.15.44:0/1708963873 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464004450 con 0x55e1be06b4f0 2024-09-13T17:36:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9454ff9640 1 -- 172.21.15.44:0/1708963873 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9464030ea0 con 0x55e1be06b4f0 2024-09-13T17:36:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1bdfed680 con 0x55e1be06b4f0 2024-09-13T17:36:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9454ff9640 1 -- 172.21.15.44:0/1708963873 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94640045b0 con 0x55e1be06b4f0 2024-09-13T17:36:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 10 client.18657 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e1bdfc3330 con 0x55e1be06b4f0 2024-09-13T17:36:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9454ff9640 1 -- 172.21.15.44:0/1708963873 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e1bdfed680 con 0x55e1be06b4f0 2024-09-13T17:36:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9454ff9640 1 -- 172.21.15.44:0/1708963873 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9464046cd0 con 0x55e1be06b4f0 2024-09-13T17:36:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 10 client.18657 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 10 client.18657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 20 client.18657 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be07d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 4 client.18657 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.102+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1bdfb85d0 con 0x55e1be07ae60 2024-09-13T17:36:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.103+0000 7f94713ae640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be07d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.103+0000 7f94713ae640 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be07d250 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4a60fb51236ad server_cookie=a2b01a45a45bdca8 in_seq=0 out_seq=0 2024-09-13T17:36:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.103+0000 7f9454ff9640 10 client.18657 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9454ff9640 1 -- 172.21.15.44:0/1708963873 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e1bdfb85d0 con 0x55e1be07ae60 2024-09-13T17:36:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9454ff9640 10 client.18657 handle_command_reply: tid=0 2024-09-13T17:36:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 client.18657 shutdown 2024-09-13T17:36:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 msgr2=0x55e1be068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be068780 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9464035000 tx=0x7f9464005d80 comp rx=0 tx=0).stop 2024-09-13T17:36:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 shutdown_connections 2024-09-13T17:36:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be07d250 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be068780 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9454ff9640 0 client.18657 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be0681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 --2- 172.21.15.44:0/1708963873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 >> 172.21.15.44:0/1708963873 conn(0x55e1be05c8c0 msgr2=0x55e1be05bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 shutdown_connections 2024-09-13T17:36:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.104+0000 7f9476413740 1 -- 172.21.15.44:0/1708963873 wait complete. 2024-09-13T17:36:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.105+0000 7f9476413740 20 client.18657 trim_cache size 0 max 16384 2024-09-13T17:36:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 Processor -- start 2024-09-13T17:36:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 -- start start 2024-09-13T17:36:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be05c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be0721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1be066760 con 0x55e1be06b4f0 2024-09-13T17:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1be060a40 con 0x55e1be060c50 2024-09-13T17:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9476413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1be063cd0 con 0x55e1be060330 2024-09-13T17:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9471baf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be0721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9471baf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be0721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51060/0 (socket says 172.21.15.44:51060) 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.117+0000 7f9471baf640 1 -- 172.21.15.44:0/3425175429 learned_addr learned my addr 172.21.15.44:0/3425175429 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f94713ae640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be05c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9470bad640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9471baf640 1 -- 172.21.15.44:0/3425175429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 msgr2=0x55e1be05c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9471baf640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be05c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9471baf640 1 -- 172.21.15.44:0/3425175429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 msgr2=0x55e1be06a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9471baf640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be06a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9471baf640 1 -- 172.21.15.44:0/3425175429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1bdf65300 con 0x55e1be06b4f0 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9470bad640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f94713ae640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be05c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9471baf640 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be0721e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f94640098d0 tx=0x7f9464030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caccc03965a18c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9457fff640 1 -- 172.21.15.44:0/3425175429 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464009b20 con 0x55e1be06b4f0 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9457fff640 1 -- 172.21.15.44:0/3425175429 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9464047e40 con 0x55e1be06b4f0 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.118+0000 7f9457fff640 1 -- 172.21.15.44:0/3425175429 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640047c0 con 0x55e1be06b4f0 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 -- 172.21.15.44:0/3425175429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 msgr2=0x55e1be0721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be0721e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f94640098d0 tx=0x7f9464030620 comp rx=0 tx=0).stop 2024-09-13T17:36:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 -- 172.21.15.44:0/3425175429 shutdown_connections 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be06b4f0 0x55e1be0721e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be060c50 0x55e1be06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 --2- 172.21.15.44:0/3425175429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be05c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 -- 172.21.15.44:0/3425175429 >> 172.21.15.44:0/3425175429 conn(0x55e1be05c8c0 msgr2=0x55e1be057a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 -- 172.21.15.44:0/3425175429 shutdown_connections 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 -- 172.21.15.44:0/3425175429 wait complete. 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 -- start start 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.119+0000 7f9476413740 1 Processor -- start 2024-09-13T17:36:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 0x55e1be06e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be06b4f0 0x55e1be06edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1be06f300 con 0x55e1be060c50 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1be06f470 con 0x55e1be06b4f0 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1be06f630 con 0x55e1be060330 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 10 client.0 _ll_register_callbacks cb 0x7f945c0d0ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 0x55e1be06e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 0x55e1be06e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51062/0 (socket says 172.21.15.44:51062) 2024-09-13T17:36:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 -- 172.21.15.44:0/2823552094 learned_addr learned my addr 172.21.15.44:0/2823552094 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f94713ae640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9471baf640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be06b4f0 0x55e1be06edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 -- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 msgr2=0x55e1be072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 -- 172.21.15.44:0/2823552094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be06b4f0 msgr2=0x55e1be06edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be06b4f0 0x55e1be06edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 -- 172.21.15.44:0/2823552094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1bdfc3330 con 0x55e1be060c50 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9471baf640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be06b4f0 0x55e1be06edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9470bad640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 0x55e1be06e880 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f945c09a1f0 tx=0x7f945c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57add33fcb5a3109 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9454ff9640 1 -- 172.21.15.44:0/2823552094 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c099da0 con 0x55e1be060c50 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9454ff9640 1 -- 172.21.15.44:0/2823552094 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f945c09a960 con 0x55e1be060c50 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.120+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f945c099bc0 con 0x55e1be060c50 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9454ff9640 1 -- 172.21.15.44:0/2823552094 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f945c09a260 con 0x55e1be060c50 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 10 client.18663 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e1bdf8a8e0 con 0x55e1be060c50 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9454ff9640 1 -- 172.21.15.44:0/2823552094 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f945c099bc0 con 0x55e1be060c50 2024-09-13T17:36:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9454ff9640 1 -- 172.21.15.44:0/2823552094 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f945c01f060 con 0x55e1be060c50 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 10 client.18663 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 10 client.18663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 20 client.18663 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be0d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 4 client.18663 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1bdcb8300 con 0x55e1be07ae60 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.121+0000 7f94713ae640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be0d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.122+0000 7f9454ff9640 10 client.18663 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.122+0000 7f94713ae640 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be0d4570 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84774f015820bb6 server_cookie=b741c4a920a81d04 in_seq=0 out_seq=0 2024-09-13T17:36:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9454ff9640 1 -- 172.21.15.44:0/2823552094 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1bdcb8300 con 0x55e1be07ae60 2024-09-13T17:36:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9454ff9640 10 client.18663 handle_command_reply: tid=0 2024-09-13T17:36:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9476413740 1 client.18663 shutdown 2024-09-13T17:36:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 msgr2=0x55e1be06e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9476413740 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 0x55e1be06e880 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f945c09a1f0 tx=0x7f945c0048b0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 shutdown_connections 2024-09-13T17:36:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.134+0000 7f9476413740 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e1be07ae60 0x55e1be0d4570 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1be06b4f0 0x55e1be06edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1be060c50 0x55e1be06e880 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 1 --2- 172.21.15.44:0/2823552094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1be060330 0x55e1be072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9454ff9640 0 client.18663 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 >> 172.21.15.44:0/2823552094 conn(0x55e1be05c8c0 msgr2=0x55e1be066a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 shutdown_connections 2024-09-13T17:36:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 1 -- 172.21.15.44:0/2823552094 wait complete. 2024-09-13T17:36:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9476413740 20 client.18663 trim_cache size 0 max 16384 2024-09-13T17:36:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:36:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:36:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:36:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:36:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94580a1230 msgr2=0x7f94580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94580a1230 0x7f94580a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9464005e30 tx=0x7f946403f040 comp rx=0 tx=0).stop 2024-09-13T17:36:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 msgr2=0x7f946c072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c072cf0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f945c00e9f0 tx=0x7f945c00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 shutdown_connections 2024-09-13T17:36:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f94580a1230 0x7f94580a36f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f946c073230 0x7f946c1a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c105070 0x7f946c072cf0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 --2- 172.21.15.44:0/1208703706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946c0fff20 0x7f946c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 >> 172.21.15.44:0/1208703706 conn(0x7f946c0fb9b0 msgr2=0x7f946c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 shutdown_connections 2024-09-13T17:36:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:31.135+0000 7f9473612640 1 -- 172.21.15.44:0/1208703706 wait complete. 2024-09-13T17:36:32.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:32 smithi044 ceph-mon[31175]: pgmap v305: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:36:32.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:32 smithi070 ceph-mon[34502]: pgmap v305: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:36:32.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:32 smithi100 ceph-mon[35055]: pgmap v305: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:36:33.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 -- 172.21.15.44:0/1352700971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fee60 msgr2=0x7f80240ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 --2- 172.21.15.44:0/1352700971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fee60 0x7f80240ff240 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8020009930 tx=0x7f802002a420 comp rx=0 tx=0).stop 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 -- 172.21.15.44:0/1352700971 shutdown_connections 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 --2- 172.21.15.44:0/1352700971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240ff810 0x7f8024103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 --2- 172.21.15.44:0/1352700971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8024103fb0 0x7f802410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 --2- 172.21.15.44:0/1352700971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fee60 0x7f80240ff240 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 -- 172.21.15.44:0/1352700971 >> 172.21.15.44:0/1352700971 conn(0x7f8024075c60 msgr2=0x7f80240fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 -- 172.21.15.44:0/1352700971 shutdown_connections 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 -- 172.21.15.44:0/1352700971 wait complete. 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 Processor -- start 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.410+0000 7f802c776640 1 -- start start 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802c776640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240fee60 0x7f80240fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802c776640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8024103fb0 0x7f80240fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802c776640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 0x7f80241a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802c776640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80241a49f0 con 0x7f80240fbb20 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802c776640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80241a4b60 con 0x7f8024103fb0 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802c776640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80241a4d20 con 0x7f80240fee60 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 0x7f80241a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 0x7f80241a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51086/0 (socket says 172.21.15.44:51086) 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 -- 172.21.15.44:0/1363777048 learned_addr learned my addr 172.21.15.44:0/1363777048 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802a512640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240fee60 0x7f80240fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f8029d11640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8024103fb0 0x7f80240fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 -- 172.21.15.44:0/1363777048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240fee60 msgr2=0x7f80240fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240fee60 0x7f80240fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 -- 172.21.15.44:0/1363777048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8024103fb0 msgr2=0x7f80240fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8024103fb0 0x7f80240fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 -- 172.21.15.44:0/1363777048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020009560 con 0x7f80240fbb20 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.411+0000 7f802ad13640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 0x7f80241a43a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f801c00cab0 tx=0x7f801c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.412+0000 7f801b7fe640 1 -- 172.21.15.44:0/1363777048 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00adc0 con 0x7f80240fbb20 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.412+0000 7f801b7fe640 1 -- 172.21.15.44:0/1363777048 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f801c00f8e0 con 0x7f80240fbb20 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.412+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80241a4fa0 con 0x7f80240fbb20 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.412+0000 7f801b7fe640 1 -- 172.21.15.44:0/1363777048 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c018830 con 0x7f80240fbb20 2024-09-13T17:36:33.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.412+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80241a5480 con 0x7f80240fbb20 2024-09-13T17:36:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.414+0000 7f801b7fe640 1 -- 172.21.15.44:0/1363777048 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f801c00fd30 con 0x7f80240fbb20 2024-09-13T17:36:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.414+0000 7f802f496740 1 Processor -- start 2024-09-13T17:36:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 -- start start 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c25a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122b85f20 con 0x563122c1ae10 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122b836b0 con 0x563122c1b730 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802f496740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122b7efa0 con 0x563122c25fd0 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c1b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51090/0 (socket says 172.21.15.44:51090) 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f8029d11640 1 -- 172.21.15.44:0/1370180891 learned_addr learned my addr 172.21.15.44:0/1370180891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802a512640 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c25a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f8029d11640 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f801b7fe640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80100a1200 0x7f80100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802a512640 1 -- 172.21.15.44:0/1370180891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 msgr2=0x563122c283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802a512640 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802a512640 1 -- 172.21.15.44:0/1370180891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 msgr2=0x563122c1b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802a512640 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c1b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802ad13640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80100a1200 0x7f80100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f802a512640 1 -- 172.21.15.44:0/1370180891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563122b39840 con 0x563122c1b730 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.415+0000 7f801b7fe640 1 -- 172.21.15.44:0/1363777048 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f801c0cd2d0 con 0x7f80240fbb20 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.416+0000 7f802a512640 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c25a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f802002aa10 tx=0x7f8020005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1de74663fe0dcc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.416+0000 7f8018ff9640 1 -- 172.21.15.44:0/1370180891 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020004230 con 0x563122c1b730 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.416+0000 7f8018ff9640 1 -- 172.21.15.44:0/1370180891 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8020031df0 con 0x563122c1b730 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.416+0000 7f8018ff9640 1 -- 172.21.15.44:0/1370180891 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020003c70 con 0x563122c1b730 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.416+0000 7f802ad13640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80100a1200 0x7f80100a36c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f801c099d10 tx=0x7f801c0659f0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- 172.21.15.44:0/1370180891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 msgr2=0x563122c25a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c25a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f802002aa10 tx=0x7f8020005fd0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- 172.21.15.44:0/1370180891 shutdown_connections 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c25a90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- 172.21.15.44:0/1370180891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c1b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- 172.21.15.44:0/1370180891 >> 172.21.15.44:0/1370180891 conn(0x563122c17380 msgr2=0x563122c17760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- 172.21.15.44:0/1370180891 shutdown_connections 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- 172.21.15.44:0/1370180891 wait complete. 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- start start 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 Processor -- start 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c25fd0 0x563122c23320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122b202a0 con 0x563122c1ae10 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122857430 con 0x563122c25fd0 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122b02680 con 0x563122c1b730 2024-09-13T17:36:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.417+0000 7f802f496740 10 client.0 _ll_register_callbacks cb 0x563122b78610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c259f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51098/0 (socket says 172.21.15.44:51098) 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 -- 172.21.15.44:0/211145015 learned_addr learned my addr 172.21.15.44:0/211145015 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f802a512640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c25fd0 0x563122c23320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 -- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 msgr2=0x563122c22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 -- 172.21.15.44:0/211145015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c25fd0 msgr2=0x563122c23320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c25fd0 0x563122c23320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 -- 172.21.15.44:0/211145015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802410c340 con 0x563122c1ae10 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f8029d11640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c259f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f801400b5f0 tx=0x7f8014014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2876a0ac6683642b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f800dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f800dffb640 1 -- 172.21.15.44:0/211145015 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014012950 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f800dffb640 1 -- 172.21.15.44:0/211145015 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8014012ab0 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.418+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631228724e0 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f800dffb640 1 -- 172.21.15.44:0/211145015 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8014012c70 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 10 client.18675 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563122b7d620 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f800dffb640 1 -- 172.21.15.44:0/211145015 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5631228724e0 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f800dffb640 1 -- 172.21.15.44:0/211145015 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f80140113e0 con 0x563122c1ae10 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 10 client.18675 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 20 client.18675 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c35d50 0x563122c38170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 4 client.18675 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631228837e0 con 0x563122c35d50 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.419+0000 7f802a512640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c35d50 0x563122c38170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.420+0000 7f802a512640 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c35d50 0x563122c38170 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5ed41c59f4968d server_cookie=8895f23fb650dc80 in_seq=0 out_seq=0 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.420+0000 7f800dffb640 10 client.18675 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.420+0000 7f800dffb640 1 -- 172.21.15.44:0/211145015 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5631228837e0 con 0x563122c35d50 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.420+0000 7f800dffb640 10 client.18675 handle_command_reply: tid=0 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 client.18675 shutdown 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 msgr2=0x563122c259f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c259f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f801400b5f0 tx=0x7f8014014040 comp rx=0 tx=0).stop 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 shutdown_connections 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c35d50 0x563122c38170 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c25fd0 0x563122c23320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f800dffb640 0 client.18675 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 --2- 172.21.15.44:0/211145015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c259f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 >> 172.21.15.44:0/211145015 conn(0x563122c17380 msgr2=0x563122c1d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 shutdown_connections 2024-09-13T17:36:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 1 -- 172.21.15.44:0/211145015 wait complete. 2024-09-13T17:36:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.421+0000 7f802f496740 20 client.18675 trim_cache size 0 max 16384 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.435+0000 7f802f496740 1 Processor -- start 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 -- start start 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c21ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c1abc0 con 0x563122c1ae10 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c1bfe0 con 0x563122c1b730 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802f496740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c21ff0 con 0x563122c25fd0 2024-09-13T17:36:33.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c25550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51112/0 (socket says 172.21.15.44:51112) 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 -- 172.21.15.44:0/578460284 learned_addr learned my addr 172.21.15.44:0/578460284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f802a512640 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c21ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 -- 172.21.15.44:0/578460284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 msgr2=0x563122c21ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c21ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 -- 172.21.15.44:0/578460284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 msgr2=0x563122c21570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c21570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 -- 172.21.15.44:0/578460284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563122b1f790 con 0x563122c1ae10 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.436+0000 7f8029d11640 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c25550 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8014012010 tx=0x7f80140043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286e6469ad4bafc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f80197fa640 1 -- 172.21.15.44:0/578460284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400bcc0 con 0x563122c1ae10 2024-09-13T17:36:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f80197fa640 1 -- 172.21.15.44:0/578460284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80140054d0 con 0x563122c1ae10 2024-09-13T17:36:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f80197fa640 1 -- 172.21.15.44:0/578460284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014004680 con 0x563122c1ae10 2024-09-13T17:36:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 -- 172.21.15.44:0/578460284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 msgr2=0x563122c25550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c25550 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8014012010 tx=0x7f80140043f0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 -- 172.21.15.44:0/578460284 shutdown_connections 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c25fd0 0x563122c21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1b730 0x563122c21570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 --2- 172.21.15.44:0/578460284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c1ae10 0x563122c25550 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 -- 172.21.15.44:0/578460284 >> 172.21.15.44:0/578460284 conn(0x563122c17380 msgr2=0x563122c1df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 -- 172.21.15.44:0/578460284 shutdown_connections 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.437+0000 7f802f496740 1 -- 172.21.15.44:0/578460284 wait complete. 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 -- start start 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 Processor -- start 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1ae10 0x563122c1fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 0x563122c20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c8c370 con 0x563122c25fd0 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c8c4e0 con 0x563122c1ae10 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c8c6a0 con 0x563122c1b730 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802f496740 10 client.0 _ll_register_callbacks cb 0x7f801400b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 0x563122c20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f8029d11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 0x563122c20660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51118/0 (socket says 172.21.15.44:51118) 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f8029d11640 1 -- 172.21.15.44:0/3574344146 learned_addr learned my addr 172.21.15.44:0/3574344146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f802a512640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.438+0000 7f8029d11640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1ae10 0x563122c1fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f8029d11640 1 -- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 msgr2=0x563122c20120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f8029d11640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c20120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f8029d11640 1 -- 172.21.15.44:0/3574344146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1ae10 msgr2=0x563122c1fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f8029d11640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1ae10 0x563122c1fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f8029d11640 1 -- 172.21.15.44:0/3574344146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563122b7d620 con 0x563122c25fd0 2024-09-13T17:36:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f8029d11640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 0x563122c20660 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x563122c1f450 tx=0x7f80140041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05340d4723feaec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f800dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f800dffb640 1 -- 172.21.15.44:0/3574344146 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014011e30 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8020031df0 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f800dffb640 1 -- 172.21.15.44:0/3574344146 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8014000e40 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f800dffb640 1 -- 172.21.15.44:0/3574344146 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8014012820 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f802f496740 10 client.18681 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563122b44ea0 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f800dffb640 1 -- 172.21.15.44:0/3574344146 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8020031df0 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.439+0000 7f800dffb640 1 -- 172.21.15.44:0/3574344146 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8014021770 con 0x563122c25fd0 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802f496740 10 client.18681 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802f496740 10 client.18681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802f496740 20 client.18681 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802f496740 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c8d960 0x563122c8fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802f496740 4 client.18681 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5631228d6e30 con 0x563122c8d960 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802a512640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c8d960 0x563122c8fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f802a512640 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c8d960 0x563122c8fd50 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e7712019ccc997 server_cookie=7af8d34be00ebe5a in_seq=0 out_seq=0 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f800dffb640 10 client.18681 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.440+0000 7f800dffb640 1 -- 172.21.15.44:0/3574344146 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5631228d6e30 con 0x563122c8d960 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f800dffb640 10 client.18681 handle_command_reply: tid=0 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 client.18681 shutdown 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 msgr2=0x563122c20660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 0x563122c20660 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x563122c1f450 tx=0x7f80140041e0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 shutdown_connections 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x563122c8d960 0x563122c8fd50 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563122c1b730 0x563122c20120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563122c25fd0 0x563122c20660 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f800dffb640 0 client.18681 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 --2- 172.21.15.44:0/3574344146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563122c1ae10 0x563122c1fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 >> 172.21.15.44:0/3574344146 conn(0x563122c17380 msgr2=0x563122c21140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 shutdown_connections 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 1 -- 172.21.15.44:0/3574344146 wait complete. 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802f496740 20 client.18681 trim_cache size 0 max 16384 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:36:33.449 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:36:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:36:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80100a1200 msgr2=0x7f80100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80100a1200 0x7f80100a36c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f801c099d10 tx=0x7f801c0659f0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 msgr2=0x7f80241a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 0x7f80241a43a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f801c00cab0 tx=0x7f801c00cf70 comp rx=0 tx=0).stop 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 shutdown_connections 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80100a1200 0x7f80100a36c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80240fbb20 0x7f80241a43a0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8024103fb0 0x7f80240fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 --2- 172.21.15.44:0/1363777048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240fee60 0x7f80240fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 >> 172.21.15.44:0/1363777048 conn(0x7f8024075c60 msgr2=0x7f80240677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 shutdown_connections 2024-09-13T17:36:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.441+0000 7f802c776640 1 -- 172.21.15.44:0/1363777048 wait complete. 2024-09-13T17:36:33.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:36:33.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 -- 172.21.15.44:0/3099984864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb82810a3a0 msgr2=0x7fb82810c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 --2- 172.21.15.44:0/3099984864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb82810a3a0 0x7fb82810c7f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb820009ca0 tx=0x7fb82002a950 comp rx=0 tx=0).stop 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 -- 172.21.15.44:0/3099984864 shutdown_connections 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 --2- 172.21.15.44:0/3099984864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8281018f0 0x7fb828109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 --2- 172.21.15.44:0/3099984864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb82810a3a0 0x7fb82810c7f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 --2- 172.21.15.44:0/3099984864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb828100fd0 0x7fb8281013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 -- 172.21.15.44:0/3099984864 >> 172.21.15.44:0/3099984864 conn(0x7fb8280fa900 msgr2=0x7fb8280fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 -- 172.21.15.44:0/3099984864 shutdown_connections 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 -- 172.21.15.44:0/3099984864 wait complete. 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.556+0000 7fb82fd25640 1 Processor -- start 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 -- start start 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb828100fd0 0x7fb828196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 0x7fb828196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82810a3a0 0x7fb82819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819b060 con 0x7fb8281018f0 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819b1d0 con 0x7fb82810a3a0 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82fd25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819b390 con 0x7fb828100fd0 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 0x7fb828196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 0x7fb828196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51148/0 (socket says 172.21.15.44:51148) 2024-09-13T17:36:33.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 -- 172.21.15.44:0/1397555981 learned_addr learned my addr 172.21.15.44:0/1397555981 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82e2c2640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82810a3a0 0x7fb82819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82dac1640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb828100fd0 0x7fb828196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 -- 172.21.15.44:0/1397555981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb828100fd0 msgr2=0x7fb828196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb828100fd0 0x7fb828196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 -- 172.21.15.44:0/1397555981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82810a3a0 msgr2=0x7fb82819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82810a3a0 0x7fb82819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 -- 172.21.15.44:0/1397555981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009900 con 0x7fb8281018f0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.557+0000 7fb82d2c0640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 0x7fb828196670 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb81800e9f0 tx=0x7fb81800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.558+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818009720 con 0x7fb8281018f0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.558+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81800cd80 con 0x7fb8281018f0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.558+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82819b670 con 0x7fb8281018f0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.558+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818010430 con 0x7fb8281018f0 2024-09-13T17:36:33.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.558+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8281a2ef0 con 0x7fb8281018f0 2024-09-13T17:36:33.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.559+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb828068e10 con 0x7fb8281018f0 2024-09-13T17:36:33.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.560+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb818004f90 con 0x7fb8281018f0 2024-09-13T17:36:33.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.561+0000 7fb81effd640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb8140a1200 0x7fb8140a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.561+0000 7fb82dac1640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb8140a1200 0x7fb8140a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.561+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fb818014070 con 0x7fb8281018f0 2024-09-13T17:36:33.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.561+0000 7fb82dac1640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb8140a1200 0x7fb8140a36c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb824009bd0 tx=0x7fb824005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.564+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb818096620 con 0x7fb8281018f0 2024-09-13T17:36:33.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:33 smithi044 ceph-mon[31175]: pgmap v306: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:36:33.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:33 smithi070 ceph-mon[34502]: pgmap v306: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:36:33.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.771+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8281013b0 con 0x7fb8281018f0 2024-09-13T17:36:33.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.772+0000 7fb81effd640 1 -- 172.21.15.44:0/1397555981 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fb818099c40 con 0x7fb8281018f0 2024-09-13T17:36:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:36:33.773 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:36:33.773 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:36:33.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:33 smithi100 ceph-mon[35055]: pgmap v306: 129 pgs: 129 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:36:33.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb8140a1200 msgr2=0x7fb8140a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb8140a1200 0x7fb8140a36c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb824009bd0 tx=0x7fb824005f90 comp rx=0 tx=0).stop 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 msgr2=0x7fb828196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 0x7fb828196670 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb81800e9f0 tx=0x7fb81800edf0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 shutdown_connections 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fb8140a1200 0x7fb8140a36c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82810a3a0 0x7fb82819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8281018f0 0x7fb828196670 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 --2- 172.21.15.44:0/1397555981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb828100fd0 0x7fb828196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 >> 172.21.15.44:0/1397555981 conn(0x7fb8280fa900 msgr2=0x7fb8280fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 shutdown_connections 2024-09-13T17:36:33.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.774+0000 7fb82fd25640 1 -- 172.21.15.44:0/1397555981 wait complete. 2024-09-13T17:36:33.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 -- 172.21.15.44:0/1010945071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 msgr2=0x7fba60101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 --2- 172.21.15.44:0/1010945071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60101230 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009900 tx=0x7fba5c02a3f0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 -- 172.21.15.44:0/1010945071 shutdown_connections 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 --2- 172.21.15.44:0/1010945071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6010a350 0x7fba6010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 --2- 172.21.15.44:0/1010945071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba60101800 0x7fba60109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 --2- 172.21.15.44:0/1010945071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60101230 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 -- 172.21.15.44:0/1010945071 >> 172.21.15.44:0/1010945071 conn(0x7fba600fa930 msgr2=0x7fba600fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 -- 172.21.15.44:0/1010945071 shutdown_connections 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.891+0000 7fba68290640 1 -- 172.21.15.44:0/1010945071 wait complete. 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 Processor -- start 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 -- start start 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba60101800 0x7fba60196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6010a350 0x7fba6019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6019b060 con 0x7fba60100e50 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6019b1d0 con 0x7fba60101800 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba68290640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6019b390 con 0x7fba6010a350 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51166/0 (socket says 172.21.15.44:51166) 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 -- 172.21.15.44:0/1685843812 learned_addr learned my addr 172.21.15.44:0/1685843812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 -- 172.21.15.44:0/1685843812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6010a350 msgr2=0x7fba6019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6582b640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba60101800 0x7fba60196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6010a350 0x7fba6019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 -- 172.21.15.44:0/1685843812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba60101800 msgr2=0x7fba60196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba60101800 0x7fba60196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.892+0000 7fba6602c640 1 -- 172.21.15.44:0/1685843812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c009560 con 0x7fba60100e50 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.893+0000 7fba6602c640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60196130 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0057c0 tx=0x7fba5c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.893+0000 7fba577fe640 1 -- 172.21.15.44:0/1685843812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c032e10 con 0x7fba60100e50 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.893+0000 7fba577fe640 1 -- 172.21.15.44:0/1685843812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba5c0046c0 con 0x7fba60100e50 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.893+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6019b610 con 0x7fba60100e50 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.893+0000 7fba577fe640 1 -- 172.21.15.44:0/1685843812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c003850 con 0x7fba60100e50 2024-09-13T17:36:33.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.893+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601a2ef0 con 0x7fba60100e50 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.895+0000 7fba577fe640 1 -- 172.21.15.44:0/1685843812 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fba5c005210 con 0x7fba60100e50 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.895+0000 7fba6b08c740 1 Processor -- start 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.895+0000 7fba6b08c740 1 -- start start 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d159410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6b08c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d0b8cc0 con 0x55c07d159950 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6b08c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d0b6450 con 0x55c07d14f0b0 2024-09-13T17:36:33.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6b08c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d0b1d40 con 0x55c07d14e790 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6582b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6602c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba577fe640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba4c0a1280 0x7fba4c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6602c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:33896/0 (socket says 172.21.15.44:33896) 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6602c640 1 -- 172.21.15.44:0/2744826696 learned_addr learned my addr 172.21.15.44:0/2744826696 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6682d640 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6682d640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba4c0a1280 0x7fba4c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6582b640 1 -- 172.21.15.44:0/2744826696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 msgr2=0x55c07d14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6582b640 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d14eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6582b640 1 -- 172.21.15.44:0/2744826696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 msgr2=0x55c07d159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6582b640 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba577fe640 1 -- 172.21.15.44:0/1685843812 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fba5c0ee130 con 0x7fba60100e50 2024-09-13T17:36:33.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.896+0000 7fba6582b640 1 -- 172.21.15.44:0/2744826696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07d079620 con 0x55c07d159950 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6582b640 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d15bd40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fba50009a90 tx=0x7fba50009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf70f4b523e26e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba54e05640 1 -- 172.21.15.44:0/2744826696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000ecb0 con 0x55c07d159950 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6682d640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba4c0a1280 0x7fba4c0a3740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fba58005c40 tx=0x7fba58005bd0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba54e05640 1 -- 172.21.15.44:0/2744826696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba5000ee10 con 0x55c07d159950 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba54e05640 1 -- 172.21.15.44:0/2744826696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50011470 con 0x55c07d159950 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6b08c740 1 -- 172.21.15.44:0/2744826696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 msgr2=0x55c07d15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6b08c740 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d15bd40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fba50009a90 tx=0x7fba50009f50 comp rx=0 tx=0).stop 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6b08c740 1 -- 172.21.15.44:0/2744826696 shutdown_connections 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6b08c740 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d15bd40 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6b08c740 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.897+0000 7fba6b08c740 1 --2- 172.21.15.44:0/2744826696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- 172.21.15.44:0/2744826696 >> 172.21.15.44:0/2744826696 conn(0x55c07d14ad00 msgr2=0x55c07d14b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- 172.21.15.44:0/2744826696 shutdown_connections 2024-09-13T17:36:33.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- 172.21.15.44:0/2744826696 wait complete. 2024-09-13T17:36:33.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- start start 2024-09-13T17:36:33.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 Processor -- start 2024-09-13T17:36:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 0x55c07d158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d159950 0x55c07d1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d070180 con 0x55c07d14e790 2024-09-13T17:36:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07cd8a430 con 0x55c07d14f0b0 2024-09-13T17:36:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d0355e0 con 0x55c07d159950 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6b08c740 10 client.0 _ll_register_callbacks cb 0x55c07d0a1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6602c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 0x55c07d158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6602c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 0x55c07d158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51180/0 (socket says 172.21.15.44:51180) 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6602c640 1 -- 172.21.15.44:0/1527658170 learned_addr learned my addr 172.21.15.44:0/1527658170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.898+0000 7fba6582b640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6682d640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d159950 0x55c07d1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6602c640 1 -- 172.21.15.44:0/1527658170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d159950 msgr2=0x55c07d1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6602c640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d159950 0x55c07d1563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6602c640 1 -- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 msgr2=0x55c07d1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6602c640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6602c640 1 -- 172.21.15.44:0/1527658170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba500096f0 con 0x55c07d14e790 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6602c640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 0x55c07d158e90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0b74d0 tx=0x7fba5c0bb6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e364ec7a62d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba49ffb640 1 -- 172.21.15.44:0/1527658170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0c3450 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba49ffb640 1 -- 172.21.15.44:0/1527658170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fba5c0eb770 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07d0dbb80 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba49ffb640 1 -- 172.21.15.44:0/1527658170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba5c0eb930 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6b08c740 10 client.18699 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c07d0b03c0 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba49ffb640 1 -- 172.21.15.44:0/1527658170 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c07d0dbb80 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.899+0000 7fba49ffb640 1 -- 172.21.15.44:0/1527658170 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba5c04d070 con 0x55c07d14e790 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6b08c740 10 client.18699 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6b08c740 10 client.18699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6b08c740 20 client.18699 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d168fe0 0x55c07d16b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6b08c740 4 client.18699 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c07d0a41f0 con 0x55c07d168fe0 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6582b640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d168fe0 0x55c07d16b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba49ffb640 10 client.18699 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.900+0000 7fba6582b640 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d168fe0 0x55c07d16b3d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=679348ff4eef4932 server_cookie=648c6858e9ffb30 in_seq=0 out_seq=0 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba49ffb640 1 -- 172.21.15.44:0/1527658170 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c07d0a41f0 con 0x55c07d168fe0 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba49ffb640 10 client.18699 handle_command_reply: tid=0 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 client.18699 shutdown 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 msgr2=0x55c07d158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 0x55c07d158e90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0b74d0 tx=0x7fba5c0bb6d0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 shutdown_connections 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d168fe0 0x55c07d16b3d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d159950 0x55c07d1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba49ffb640 0 client.18699 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1527658170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14e790 0x55c07d158e90 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 >> 172.21.15.44:0/1527658170 conn(0x55c07d14ad00 msgr2=0x55c07d150a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 shutdown_connections 2024-09-13T17:36:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.901+0000 7fba6b08c740 1 -- 172.21.15.44:0/1527658170 wait complete. 2024-09-13T17:36:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.902+0000 7fba6b08c740 20 client.18699 trim_cache size 0 max 16384 2024-09-13T17:36:33.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.912+0000 7fba6b08c740 1 Processor -- start 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.912+0000 7fba6b08c740 1 -- start start 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 0x55c07d160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d159950 0x55c07d160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6b08c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d168e00 con 0x55c07d14f0b0 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6b08c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d15c400 con 0x55c07d159950 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6b08c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d146910 con 0x55c07d14e790 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 0x55c07d160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 0x55c07d160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51182/0 (socket says 172.21.15.44:51182) 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 -- 172.21.15.44:0/934297764 learned_addr learned my addr 172.21.15.44:0/934297764 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6602c640 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6682d640 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d159950 0x55c07d160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 -- 172.21.15.44:0/934297764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 msgr2=0x55c07d155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 -- 172.21.15.44:0/934297764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d159950 msgr2=0x55c07d160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d159950 0x55c07d160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 -- 172.21.15.44:0/934297764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07d052770 con 0x55c07d14f0b0 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6682d640 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d159950 0x55c07d160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba6582b640 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 0x55c07d160650 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x55c07d151330 tx=0x7fba50004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475b479ed60a1f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba55606640 1 -- 172.21.15.44:0/934297764 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50021050 con 0x55c07d14f0b0 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba55606640 1 -- 172.21.15.44:0/934297764 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba50022040 con 0x55c07d14f0b0 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.913+0000 7fba55606640 1 -- 172.21.15.44:0/934297764 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500116d0 con 0x55c07d14f0b0 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 -- 172.21.15.44:0/934297764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 msgr2=0x55c07d160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 0x55c07d160650 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x55c07d151330 tx=0x7fba50004ac0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 -- 172.21.15.44:0/934297764 shutdown_connections 2024-09-13T17:36:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d159950 0x55c07d160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d14f0b0 0x55c07d160650 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 --2- 172.21.15.44:0/934297764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 -- 172.21.15.44:0/934297764 >> 172.21.15.44:0/934297764 conn(0x55c07d14ad00 msgr2=0x55c07d151360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 -- 172.21.15.44:0/934297764 shutdown_connections 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 -- 172.21.15.44:0/934297764 wait complete. 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 -- start start 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.914+0000 7fba6b08c740 1 Processor -- start 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d1b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d1b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d1b7cc0 con 0x55c07d159950 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d1b7e30 con 0x55c07d14f0b0 2024-09-13T17:36:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d1b7ff0 con 0x55c07d14e790 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6b08c740 10 client.0 _ll_register_callbacks cb 0x7fba500092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d1b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d1b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51184/0 (socket says 172.21.15.44:51184) 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 -- 172.21.15.44:0/1035320073 learned_addr learned my addr 172.21.15.44:0/1035320073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6602c640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d1b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6582b640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 -- 172.21.15.44:0/1035320073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 msgr2=0x55c07d1b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d1b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 -- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 msgr2=0x55c07d1b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 -- 172.21.15.44:0/1035320073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c07d079620 con 0x55c07d159950 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6602c640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d1b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba6682d640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d1b7750 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fba5801f910 tx=0x7fba580094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12090dedd8ec057 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.915+0000 7fba49ffb640 1 -- 172.21.15.44:0/1035320073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58024070 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba49ffb640 1 -- 172.21.15.44:0/1035320073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fba58017070 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07d1464f0 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba49ffb640 1 -- 172.21.15.44:0/1035320073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba58013c50 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 10 client.18705 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c07d077e80 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba49ffb640 1 -- 172.21.15.44:0/1035320073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c07d1464f0 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba49ffb640 1 -- 172.21.15.44:0/1035320073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba5801a050 con 0x55c07d159950 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 10 client.18705 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 10 client.18705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 20 client.18705 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d1c0b90 0x55c07d1c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 4 client.18705 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:36:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fba5000ee10 con 0x55c07d1c0b90 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.916+0000 7fba6602c640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d1c0b90 0x55c07d1c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6602c640 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d1c0b90 0x55c07d1c2fb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=789fb6a318a1f817 server_cookie=d1ddb45b1d641b10 in_seq=0 out_seq=0 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba49ffb640 10 client.18705 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba49ffb640 1 -- 172.21.15.44:0/1035320073 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fba5000ee10 con 0x55c07d1c0b90 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba49ffb640 10 client.18705 handle_command_reply: tid=0 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 client.18705 shutdown 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 msgr2=0x55c07d1b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d1b7750 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fba5801f910 tx=0x7fba580094d0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 shutdown_connections 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55c07d1c0b90 0x55c07d1c2fb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c07d159950 0x55c07d1b7750 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba49ffb640 0 client.18705 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:36:33.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c07d14f0b0 0x55c07d1b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 --2- 172.21.15.44:0/1035320073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c07d14e790 0x55c07d1b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 >> 172.21.15.44:0/1035320073 conn(0x55c07d14ad00 msgr2=0x55c07d1501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 shutdown_connections 2024-09-13T17:36:33.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 1 -- 172.21.15.44:0/1035320073 wait complete. 2024-09-13T17:36:33.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.917+0000 7fba6b08c740 20 client.18705 trim_cache size 0 max 16384 2024-09-13T17:36:33.925 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:36:33.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba4c0a1280 msgr2=0x7fba4c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba4c0a1280 0x7fba4c0a3740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fba58005c40 tx=0x7fba58005bd0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 msgr2=0x7fba60196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:33.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60196130 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0057c0 tx=0x7fba5c032c20 comp rx=0 tx=0).stop 2024-09-13T17:36:33.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 shutdown_connections 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fba4c0a1280 0x7fba4c0a3740 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6010a350 0x7fba6019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba60101800 0x7fba60196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 --2- 172.21.15.44:0/1685843812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba60100e50 0x7fba60196130 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 >> 172.21.15.44:0/1685843812 conn(0x7fba600fa930 msgr2=0x7fba600fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 shutdown_connections 2024-09-13T17:36:33.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:33.918+0000 7fba68290640 1 -- 172.21.15.44:0/1685843812 wait complete. 2024-09-13T17:36:33.929 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:36:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 -- 172.21.15.44:0/2254051463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20100f60 msgr2=0x7f8a20101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2254051463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20100f60 0x7f8a20101340 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009900 tx=0x7f8a1002a480 comp rx=0 tx=0).stop 2024-09-13T17:36:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 -- 172.21.15.44:0/2254051463 shutdown_connections 2024-09-13T17:36:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2254051463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a20101880 0x7f8a20109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2254051463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2010a330 0x7f8a2010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2254051463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20100f60 0x7f8a20101340 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 -- 172.21.15.44:0/2254051463 >> 172.21.15.44:0/2254051463 conn(0x7f8a20075c60 msgr2=0x7f8a200fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 -- 172.21.15.44:0/2254051463 shutdown_connections 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 -- 172.21.15.44:0/2254051463 wait complete. 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 Processor -- start 2024-09-13T17:36:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.032+0000 7f8a257ae640 1 -- start start 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a20100f60 0x7f8a200febb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 0x7f8a200ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a2010a330 0x7f8a200ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a201a45d0 con 0x7f8a20101880 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a201a4740 con 0x7f8a20100f60 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a201a4900 con 0x7f8a2010a330 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 0x7f8a200ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 0x7f8a200ff0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51206/0 (socket says 172.21.15.44:51206) 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/2933090097 learned_addr learned my addr 172.21.15.44:0/2933090097 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a2010a330 0x7f8a200ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1effd640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a20100f60 0x7f8a200febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/2933090097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a2010a330 msgr2=0x7f8a200ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a2010a330 0x7f8a200ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/2933090097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a20100f60 msgr2=0x7f8a200febb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a20100f60 0x7f8a200febb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/2933090097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10009560 con 0x7f8a20101880 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 0x7f8a200ff0f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d950 tx=0x7f8a1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a07fff640 1 -- 172.21.15.44:0/2933090097 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14009c30 con 0x7f8a20101880 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a201a4be0 con 0x7f8a20101880 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a07fff640 1 -- 172.21.15.44:0/2933090097 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a14009d90 con 0x7f8a20101880 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.033+0000 7f8a07fff640 1 -- 172.21.15.44:0/2933090097 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400f620 con 0x7f8a20101880 2024-09-13T17:36:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.034+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a201a5140 con 0x7f8a20101880 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a07fff640 1 -- 172.21.15.44:0/2933090097 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8a140109f0 con 0x7f8a20101880 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 Processor -- start 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 -- start start 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d50410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455cafcc0 con 0x562455d45790 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455cad450 con 0x562455d50950 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a284d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455ca8d40 con 0x562455d460b0 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d45b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51212/0 (socket says 172.21.15.44:51212) 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a1f7fe640 1 -- 172.21.15.44:0/3219253845 learned_addr learned my addr 172.21.15.44:0/3219253845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a1effd640 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d50410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.036+0000 7f8a07fff640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8a080a1200 0x7f8a080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 -- 172.21.15.44:0/3219253845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 msgr2=0x562455d50410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d50410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 -- 172.21.15.44:0/3219253845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 msgr2=0x562455d52d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d52d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 -- 172.21.15.44:0/3219253845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562455c70620 con 0x562455d45790 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a07fff640 1 -- 172.21.15.44:0/2933090097 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8a140ccc50 con 0x7f8a20101880 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8a080a1200 0x7f8a080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d45b70 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a180108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8afe2b4c3e17f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a057fa640 1 -- 172.21.15.44:0/3219253845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18022070 con 0x562455d45790 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a057fa640 1 -- 172.21.15.44:0/3219253845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a1801dda0 con 0x562455d45790 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.037+0000 7f8a057fa640 1 -- 172.21.15.44:0/3219253845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1801caf0 con 0x562455d45790 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 -- 172.21.15.44:0/3219253845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 msgr2=0x562455d45b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d45b70 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a180108d0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8a080a1200 0x7f8a080a36c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8a18021000 tx=0x7f8a1801e000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 -- 172.21.15.44:0/3219253845 shutdown_connections 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d50410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d52d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3219253845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d45b70 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 -- 172.21.15.44:0/3219253845 >> 172.21.15.44:0/3219253845 conn(0x562455d41d00 msgr2=0x562455d420e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 -- 172.21.15.44:0/3219253845 shutdown_connections 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 -- 172.21.15.44:0/3219253845 wait complete. 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 -- start start 2024-09-13T17:36:34.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.038+0000 7f8a284d1740 1 Processor -- start 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d45790 0x562455d4fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d4d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d50950 0x562455d4d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455c67180 con 0x562455d460b0 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455981430 con 0x562455d45790 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455c2c5e0 con 0x562455d50950 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 10 client.0 _ll_register_callbacks cb 0x562455c98aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d4d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d4d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51218/0 (socket says 172.21.15.44:51218) 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/3102565683 learned_addr learned my addr 172.21.15.44:0/3102565683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1effd640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d45790 0x562455d4fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d50950 0x562455d4d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d50950 msgr2=0x562455d4d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d50950 0x562455d4d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/3102565683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d45790 msgr2=0x562455d4fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d45790 0x562455d4fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/3102565683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a18009c10 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d4d270 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a14096460 tx=0x7f8a14069970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407765b3bb8f891c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a02ff5640 1 -- 172.21.15.44:0/3102565683 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140999f0 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a02ff5640 1 -- 172.21.15.44:0/3102565683 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8a1409d820 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562455cd2b80 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a02ff5640 1 -- 172.21.15.44:0/3102565683 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a140a1e70 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 10 client.18717 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.039+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562455ca73c0 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a02ff5640 1 -- 172.21.15.44:0/3102565683 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562455cd2b80 con 0x562455d460b0 2024-09-13T17:36:34.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a02ff5640 1 -- 172.21.15.44:0/3102565683 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a140989e0 con 0x562455d460b0 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a284d1740 10 client.18717 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a284d1740 10 client.18717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a284d1740 20 client.18717 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455d601e0 0x562455d62600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a284d1740 4 client.18717 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562455c9b1f0 con 0x562455d601e0 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.040+0000 7f8a1effd640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455d601e0 0x562455d62600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a02ff5640 10 client.18717 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a1effd640 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455d601e0 0x562455d62600 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edabaf3c8e5062ea server_cookie=9601706198eec618 in_seq=0 out_seq=0 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a02ff5640 1 -- 172.21.15.44:0/3102565683 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562455c9b1f0 con 0x562455d601e0 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a02ff5640 10 client.18717 handle_command_reply: tid=0 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 client.18717 shutdown 2024-09-13T17:36:34.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 msgr2=0x562455d4d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d4d270 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a14096460 tx=0x7f8a14069970 comp rx=0 tx=0).stop 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 shutdown_connections 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455d601e0 0x562455d62600 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d50950 0x562455d4d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d4d270 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a02ff5640 0 client.18717 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 --2- 172.21.15.44:0/3102565683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d45790 0x562455d4fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.041+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 >> 172.21.15.44:0/3102565683 conn(0x562455d41d00 msgr2=0x562455d40eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.042+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 shutdown_connections 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.042+0000 7f8a284d1740 1 -- 172.21.15.44:0/3102565683 wait complete. 2024-09-13T17:36:34.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.042+0000 7f8a284d1740 20 client.18717 trim_cache size 0 max 16384 2024-09-13T17:36:34.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 Processor -- start 2024-09-13T17:36:34.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 -- start start 2024-09-13T17:36:34.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d57610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d57b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455d5c8f0 con 0x562455d45790 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455d4b7a0 con 0x562455d50950 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a284d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455d45580 con 0x562455d460b0 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d57610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d57610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51234/0 (socket says 172.21.15.44:51234) 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 -- 172.21.15.44:0/430331017 learned_addr learned my addr 172.21.15.44:0/430331017 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d57b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 -- 172.21.15.44:0/430331017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 msgr2=0x562455d57b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d57b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 -- 172.21.15.44:0/430331017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 msgr2=0x562455d4f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1effd640 1 -- 172.21.15.44:0/430331017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562455c49770 con 0x562455d45790 2024-09-13T17:36:34.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.053+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a1effd640 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d57610 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8a1002aa40 tx=0x7f8a10005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce3d6f190da67a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a05ffb640 1 -- 172.21.15.44:0/430331017 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10042040 con 0x562455d45790 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a05ffb640 1 -- 172.21.15.44:0/430331017 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a10044d70 con 0x562455d45790 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a05ffb640 1 -- 172.21.15.44:0/430331017 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10003c20 con 0x562455d45790 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 -- 172.21.15.44:0/430331017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 msgr2=0x562455d57610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d57610 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8a1002aa40 tx=0x7f8a10005720 comp rx=0 tx=0).stop 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 -- 172.21.15.44:0/430331017 shutdown_connections 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d460b0 0x562455d57b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 --2- 172.21.15.44:0/430331017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d45790 0x562455d57610 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 -- 172.21.15.44:0/430331017 >> 172.21.15.44:0/430331017 conn(0x562455d41d00 msgr2=0x562455d66f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 -- 172.21.15.44:0/430331017 shutdown_connections 2024-09-13T17:36:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 -- 172.21.15.44:0/430331017 wait complete. 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.054+0000 7f8a284d1740 1 -- start start 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 Processor -- start 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d45790 0x562455d57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d49b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455d4a6b0 con 0x562455d460b0 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455d4a820 con 0x562455d50950 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562455d4a9e0 con 0x562455d45790 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a284d1740 10 client.0 _ll_register_callbacks cb 0x7f8a180099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d49b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d49b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51236/0 (socket says 172.21.15.44:51236) 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/418218364 learned_addr learned my addr 172.21.15.44:0/418218364 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1f7fe640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1effd640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d45790 0x562455d57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d45790 msgr2=0x562455d57880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d45790 0x562455d57880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/418218364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 msgr2=0x562455d4a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 -- 172.21.15.44:0/418218364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2010d420 con 0x562455d460b0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a1e7fc640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d49b60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8a14095ef0 tx=0x7f8a14098b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44296cdee6e6bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.055+0000 7f8a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a02ff5640 1 -- 172.21.15.44:0/418218364 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14014070 con 0x562455d460b0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a02ff5640 1 -- 172.21.15.44:0/418218364 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f8a1409d820 con 0x562455d460b0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562455d5cfa0 con 0x562455d460b0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a02ff5640 1 -- 172.21.15.44:0/418218364 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a14010830 con 0x562455d460b0 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 10 client.18723 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:34.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562455c6ee80 con 0x562455d460b0 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a02ff5640 1 -- 172.21.15.44:0/418218364 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562455d5cfa0 con 0x562455d460b0 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a02ff5640 1 -- 172.21.15.44:0/418218364 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a140a19e0 con 0x562455d460b0 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 10 client.18723 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 10 client.18723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 20 client.18723 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455db7a00 0x562455db9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 4 client.18723 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8a1801dda0 con 0x562455db7a00 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.056+0000 7f8a1effd640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455db7a00 0x562455db9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a1effd640 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455db7a00 0x562455db9e20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10af7a691e5d701e server_cookie=88477683f650257d in_seq=0 out_seq=0 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a02ff5640 10 client.18723 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a02ff5640 1 -- 172.21.15.44:0/418218364 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8a1801dda0 con 0x562455db7a00 2024-09-13T17:36:34.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a02ff5640 10 client.18723 handle_command_reply: tid=0 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 client.18723 shutdown 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 msgr2=0x562455d49b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d49b60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8a14095ef0 tx=0x7f8a14098b80 comp rx=0 tx=0).stop 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 shutdown_connections 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x562455db7a00 0x562455db9e20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562455d50950 0x562455d4a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562455d460b0 0x562455d49b60 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 --2- 172.21.15.44:0/418218364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562455d45790 0x562455d57880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a02ff5640 0 client.18723 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 >> 172.21.15.44:0/418218364 conn(0x562455d41d00 msgr2=0x562455d4cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 shutdown_connections 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 1 -- 172.21.15.44:0/418218364 wait complete. 2024-09-13T17:36:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.057+0000 7f8a284d1740 20 client.18723 trim_cache size 0 max 16384 2024-09-13T17:36:34.066 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:36:34.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8a080a1200 msgr2=0x7f8a080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8a080a1200 0x7f8a080a36c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8a18021000 tx=0x7f8a1801e000 comp rx=0 tx=0).stop 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 msgr2=0x7f8a200ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 0x7f8a200ff0f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d950 tx=0x7f8a1400de10 comp rx=0 tx=0).stop 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 shutdown_connections 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8a080a1200 0x7f8a080a36c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a2010a330 0x7f8a200ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a20101880 0x7f8a200ff0f0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 --2- 172.21.15.44:0/2933090097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a20100f60 0x7f8a200febb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 >> 172.21.15.44:0/2933090097 conn(0x7f8a20075c60 msgr2=0x7f8a200fb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 shutdown_connections 2024-09-13T17:36:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.058+0000 7f8a257ae640 1 -- 172.21.15.44:0/2933090097 wait complete. 2024-09-13T17:36:34.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 -- 172.21.15.44:0/4133412679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc103fd0 msgr2=0x7f04bc10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/4133412679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc103fd0 0x7f04bc10bb20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009930 tx=0x7f04b402a420 comp rx=0 tx=0).stop 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 -- 172.21.15.44:0/4133412679 shutdown_connections 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/4133412679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04bc0ff830 0x7f04bc103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/4133412679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc103fd0 0x7f04bc10bb20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/4133412679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc0fee80 0x7f04bc0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 -- 172.21.15.44:0/4133412679 >> 172.21.15.44:0/4133412679 conn(0x7f04bc0fa8d0 msgr2=0x7f04bc0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 -- 172.21.15.44:0/4133412679 shutdown_connections 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 -- 172.21.15.44:0/4133412679 wait complete. 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 Processor -- start 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.173+0000 7f04c2e3a640 1 -- start start 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c2e3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04bc0fee80 0x7f04bc197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c2e3a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc103fd0 0x7f04bc197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c2e3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 0x7f04bc1a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c2e3a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a2750 con 0x7f04bc197ab0 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c2e3a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a28c0 con 0x7f04bc103fd0 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c2e3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a2a80 con 0x7f04bc0fee80 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 0x7f04bc1a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 0x7f04bc1a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51264/0 (socket says 172.21.15.44:51264) 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 -- 172.21.15.44:0/1223249231 learned_addr learned my addr 172.21.15.44:0/1223249231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04bc0fee80 0x7f04bc197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04bbfff640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc103fd0 0x7f04bc197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 -- 172.21.15.44:0/1223249231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04bc0fee80 msgr2=0x7f04bc197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04bc0fee80 0x7f04bc197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 -- 172.21.15.44:0/1223249231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc103fd0 msgr2=0x7f04bc197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc103fd0 0x7f04bc197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 -- 172.21.15.44:0/1223249231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4009560 con 0x7f04bc197ab0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04bbfff640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc103fd0 0x7f04bc197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.174+0000 7f04c13d7640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 0x7f04bc1a20b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009900 tx=0x7f04b4032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.175+0000 7f04b9ffb640 1 -- 172.21.15.44:0/1223249231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40044f0 con 0x7f04bc197ab0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.175+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc1a2d00 con 0x7f04bc197ab0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.175+0000 7f04b9ffb640 1 -- 172.21.15.44:0/1223249231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b4005490 con 0x7f04bc197ab0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.175+0000 7f04b9ffb640 1 -- 172.21.15.44:0/1223249231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4003850 con 0x7f04bc197ab0 2024-09-13T17:36:34.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.175+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc1a3200 con 0x7f04bc197ab0 2024-09-13T17:36:34.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04b9ffb640 1 -- 172.21.15.44:0/1223249231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04b4004850 con 0x7f04bc197ab0 2024-09-13T17:36:34.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 Processor -- start 2024-09-13T17:36:34.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 -- start start 2024-09-13T17:36:34.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b618db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b6198410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b60f7cc0 con 0x5574b618d790 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b60f5450 con 0x5574b618e0b0 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c5c40740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b60f0d40 con 0x5574b6198950 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b618db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b618db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51274/0 (socket says 172.21.15.44:51274) 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04bbfff640 1 -- 172.21.15.44:0/3736833700 learned_addr learned my addr 172.21.15.44:0/3736833700 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b6198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.178+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04b9ffb640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a40a1200 0x7f04a40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04bbfff640 1 -- 172.21.15.44:0/3736833700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 msgr2=0x5574b619ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04bbfff640 1 -- 172.21.15.44:0/3736833700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 msgr2=0x5574b6198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b6198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04bbfff640 1 -- 172.21.15.44:0/3736833700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b60b8620 con 0x5574b618d790 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04c13d7640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a40a1200 0x7f04a40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b618db70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f04b00128a0 tx=0x7f04b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce3e486b7d3f2f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04b9ffb640 1 -- 172.21.15.44:0/1223249231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f04b40edc10 con 0x7f04bc197ab0 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04a2ffd640 1 -- 172.21.15.44:0/3736833700 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0003970 con 0x5574b618d790 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04a2ffd640 1 -- 172.21.15.44:0/3736833700 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b000ed60 con 0x5574b618d790 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.179+0000 7f04a2ffd640 1 -- 172.21.15.44:0/3736833700 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0010b70 con 0x5574b618d790 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 -- 172.21.15.44:0/3736833700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 msgr2=0x5574b618db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c13d7640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a40a1200 0x7f04a40a36c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04b40812f0 tx=0x7f04b40da040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b618db70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f04b00128a0 tx=0x7f04b0009f90 comp rx=0 tx=0).stop 2024-09-13T17:36:34.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 -- 172.21.15.44:0/3736833700 shutdown_connections 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b6198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3736833700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b618db70 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 -- 172.21.15.44:0/3736833700 >> 172.21.15.44:0/3736833700 conn(0x5574b6189d00 msgr2=0x5574b618a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 -- 172.21.15.44:0/3736833700 shutdown_connections 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 -- 172.21.15.44:0/3736833700 wait complete. 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 -- start start 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.180+0000 7f04c5c40740 1 Processor -- start 2024-09-13T17:36:34.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b6197e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b61983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6195400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b60af180 con 0x5574b618d790 2024-09-13T17:36:34.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b5dc9430 con 0x5574b6198950 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b60745e0 con 0x5574b618e0b0 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c5c40740 10 client.0 _ll_register_callbacks cb 0x5574b60e0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b6197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b6197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51286/0 (socket says 172.21.15.44:51286) 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 -- 172.21.15.44:0/3936156195 learned_addr learned my addr 172.21.15.44:0/3936156195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b61983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 -- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 msgr2=0x5574b61983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b61983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 -- 172.21.15.44:0/3936156195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 msgr2=0x5574b6195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 -- 172.21.15.44:0/3936156195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b00124d0 con 0x5574b618d790 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b6197e90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f04b000f010 tx=0x7f04b0005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b46e21ba8341aea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.181+0000 7f049bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f049bfff640 1 -- 172.21.15.44:0/3936156195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0003d80 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f049bfff640 1 -- 172.21.15.44:0/3936156195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f04b00108f0 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574b611ab80 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f049bfff640 1 -- 172.21.15.44:0/3936156195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b000ebd0 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 10 client.18735 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5574b60ef3c0 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f049bfff640 1 -- 172.21.15.44:0/3936156195 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5574b611ab80 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f049bfff640 1 -- 172.21.15.44:0/3936156195 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04b00045e0 con 0x5574b618d790 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 10 client.18735 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 10 client.18735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 20 client.18735 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61a8190 0x5574b61aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 4 client.18735 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:34.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.182+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574b60e31f0 con 0x5574b61a8190 2024-09-13T17:36:34.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.183+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61a8190 0x5574b61aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.183+0000 7f049bfff640 10 client.18735 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:34.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.183+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61a8190 0x5574b61aa5b0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd00049c4496661 server_cookie=847956136220edeb in_seq=0 out_seq=0 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f049bfff640 1 -- 172.21.15.44:0/3936156195 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5574b60e31f0 con 0x5574b61a8190 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f049bfff640 10 client.18735 handle_command_reply: tid=0 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 client.18735 shutdown 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 msgr2=0x5574b6197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b6197e90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f04b000f010 tx=0x7f04b0005a70 comp rx=0 tx=0).stop 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 shutdown_connections 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61a8190 0x5574b61aa5b0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b61983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f049bfff640 0 client.18735 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3936156195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b6197e90 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 >> 172.21.15.44:0/3936156195 conn(0x5574b6189d00 msgr2=0x5574b618fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 shutdown_connections 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 1 -- 172.21.15.44:0/3936156195 wait complete. 2024-09-13T17:36:34.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.184+0000 7f04c5c40740 20 client.18735 trim_cache size 0 max 16384 2024-09-13T17:36:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 Processor -- start 2024-09-13T17:36:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 -- start start 2024-09-13T17:36:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b619fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6193a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b618a8b0 con 0x5574b618d790 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b619b400 con 0x5574b6198950 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c5c40740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b618d510 con 0x5574b618e0b0 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51298/0 (socket says 172.21.15.44:51298) 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04bbfff640 1 -- 172.21.15.44:0/3529848467 learned_addr learned my addr 172.21.15.44:0/3529848467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.195+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b619fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04bbfff640 1 -- 172.21.15.44:0/3529848467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 msgr2=0x5574b619fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b619fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04bbfff640 1 -- 172.21.15.44:0/3529848467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 msgr2=0x5574b6193a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6193a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04bbfff640 1 -- 172.21.15.44:0/3529848467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b6091770 con 0x5574b618d790 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04bbfff640 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619f650 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f04b0005c70 tx=0x7f04b0010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12de6f9dc76a72ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04a37fe640 1 -- 172.21.15.44:0/3529848467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b002d560 con 0x5574b618d790 2024-09-13T17:36:34.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04a37fe640 1 -- 172.21.15.44:0/3529848467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b000e720 con 0x5574b618d790 2024-09-13T17:36:34.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.196+0000 7f04a37fe640 1 -- 172.21.15.44:0/3529848467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0003de0 con 0x5574b618d790 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 -- 172.21.15.44:0/3529848467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 msgr2=0x5574b619f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619f650 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f04b0005c70 tx=0x7f04b0010590 comp rx=0 tx=0).stop 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 -- 172.21.15.44:0/3529848467 shutdown_connections 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b618e0b0 0x5574b619fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b6198950 0x5574b6193a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 --2- 172.21.15.44:0/3529848467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619f650 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 -- 172.21.15.44:0/3529848467 >> 172.21.15.44:0/3529848467 conn(0x5574b6189d00 msgr2=0x5574b61aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 -- 172.21.15.44:0/3529848467 shutdown_connections 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 -- 172.21.15.44:0/3529848467 wait complete. 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 -- start start 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 Processor -- start 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.197+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b619e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c5c40740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c5c40740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b619f750 con 0x5574b618d790 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c5c40740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b619f8c0 con 0x5574b618e0b0 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c5c40740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b619fa80 con 0x5574b6198950 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c5c40740 10 client.0 _ll_register_callbacks cb 0x7f04b0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51306/0 (socket says 172.21.15.44:51306) 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 -- 172.21.15.44:0/396764406 learned_addr learned my addr 172.21.15.44:0/396764406 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b619e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 -- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 msgr2=0x5574b619f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 -- 172.21.15.44:0/396764406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 msgr2=0x5574b619e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b619e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 -- 172.21.15.44:0/396764406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574b60ef3c0 con 0x5574b618d790 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04bbfff640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619dd30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f04b000f010 tx=0x7f04b0009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60acd7e964744fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f049bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f049bfff640 1 -- 172.21.15.44:0/396764406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b002db60 con 0x5574b618d790 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f049bfff640 1 -- 172.21.15.44:0/396764406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f04b00126f0 con 0x5574b618d790 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b000ed60 con 0x5574b618d790 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.198+0000 7f049bfff640 1 -- 172.21.15.44:0/396764406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b0010600 con 0x5574b618d790 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 10 client.18741 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5574b60b6e80 con 0x5574b618d790 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f049bfff640 1 -- 172.21.15.44:0/396764406 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b000ed60 con 0x5574b618d790 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f049bfff640 1 -- 172.21.15.44:0/396764406 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04b00052f0 con 0x5574b618d790 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 10 client.18741 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 20 client.18741 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61ff430 0x5574b6201820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 4 client.18741 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574b5de44e0 con 0x5574b61ff430 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.199+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61ff430 0x5574b6201820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:34.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c0bd6640 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61ff430 0x5574b6201820 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f6d6f147597aa2 server_cookie=aedfaffb83b951c4 in_seq=0 out_seq=0 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f049bfff640 10 client.18741 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f049bfff640 1 -- 172.21.15.44:0/396764406 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574b5de44e0 con 0x5574b61ff430 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f049bfff640 10 client.18741 handle_command_reply: tid=0 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 client.18741 shutdown 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 msgr2=0x5574b619dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619dd30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f04b000f010 tx=0x7f04b0009fb0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 shutdown_connections 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5574b61ff430 0x5574b6201820 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574b6198950 0x5574b619f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f049bfff640 0 client.18741 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574b618e0b0 0x5574b619e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 --2- 172.21.15.44:0/396764406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574b618d790 0x5574b619dd30 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 >> 172.21.15.44:0/396764406 conn(0x5574b6189d00 msgr2=0x5574b6190310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 shutdown_connections 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 1 -- 172.21.15.44:0/396764406 wait complete. 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.200+0000 7f04c5c40740 20 client.18741 trim_cache size 0 max 16384 2024-09-13T17:36:34.210 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:36:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a40a1200 msgr2=0x7f04a40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a40a1200 0x7f04a40a36c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04b40812f0 tx=0x7f04b40da040 comp rx=0 tx=0).stop 2024-09-13T17:36:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 msgr2=0x7f04bc1a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 0x7f04bc1a20b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009900 tx=0x7f04b4032ba0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 shutdown_connections 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f04a40a1200 0x7f04a40a36c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04bc197ab0 0x7f04bc1a20b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04bc103fd0 0x7f04bc197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 --2- 172.21.15.44:0/1223249231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04bc0fee80 0x7f04bc197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 >> 172.21.15.44:0/1223249231 conn(0x7f04bc0fa8d0 msgr2=0x7f04bc0fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 shutdown_connections 2024-09-13T17:36:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:34.201+0000 7f04c2e3a640 1 -- 172.21.15.44:0/1223249231 wait complete. 2024-09-13T17:36:34.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:34 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1397555981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:36:34.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:34 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1397555981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:36:34.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:34 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1397555981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:36:35.214 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:36:35.214 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:36:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 -- 172.21.15.44:0/1466279101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c0fff90 msgr2=0x7fe51c100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 --2- 172.21.15.44:0/1466279101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c0fff90 0x7fe51c100370 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009930 tx=0x7fe50c02a420 comp rx=0 tx=0).stop 2024-09-13T17:36:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 -- 172.21.15.44:0/1466279101 shutdown_connections 2024-09-13T17:36:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 --2- 172.21.15.44:0/1466279101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c1050e0 0x7fe51c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 --2- 172.21.15.44:0/1466279101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c100940 0x7fe51c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 --2- 172.21.15.44:0/1466279101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c0fff90 0x7fe51c100370 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 -- 172.21.15.44:0/1466279101 >> 172.21.15.44:0/1466279101 conn(0x7fe51c0fba20 msgr2=0x7fe51c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 -- 172.21.15.44:0/1466279101 shutdown_connections 2024-09-13T17:36:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.333+0000 7fe52111f640 1 -- 172.21.15.44:0/1466279101 wait complete. 2024-09-13T17:36:35.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 Processor -- start 2024-09-13T17:36:35.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 -- start start 2024-09-13T17:36:35.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c0fff90 0x7fe51c196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 0x7fe51c196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c19ab00 0x7fe51c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19b4f0 con 0x7fe51c1050e0 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19b660 con 0x7fe51c19ab00 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe52111f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19b7d0 con 0x7fe51c0fff90 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe51a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 0x7fe51c196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe51a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 0x7fe51c196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51326/0 (socket says 172.21.15.44:51326) 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe51a59c640 1 -- 172.21.15.44:0/2683734834 learned_addr learned my addr 172.21.15.44:0/2683734834 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c0fff90 0x7fe51c196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.334+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c19ab00 0x7fe51c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51a59c640 1 -- 172.21.15.44:0/2683734834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c0fff90 msgr2=0x7fe51c196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51a59c640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c0fff90 0x7fe51c196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51a59c640 1 -- 172.21.15.44:0/2683734834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c19ab00 msgr2=0x7fe51c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51a59c640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c19ab00 0x7fe51c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51a59c640 1 -- 172.21.15.44:0/2683734834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c009560 con 0x7fe51c1050e0 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c0fff90 0x7fe51c196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c19ab00 0x7fe51c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe51a59c640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 0x7fe51c196760 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d770 tx=0x7fe51000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe503fff640 1 -- 172.21.15.44:0/2683734834 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510009c30 con 0x7fe51c1050e0 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe503fff640 1 -- 172.21.15.44:0/2683734834 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe51000de80 con 0x7fe51c1050e0 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe503fff640 1 -- 172.21.15.44:0/2683734834 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000f620 con 0x7fe51c1050e0 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c1a2db0 con 0x7fe51c1050e0 2024-09-13T17:36:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.335+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c1a31b0 con 0x7fe51c1050e0 2024-09-13T17:36:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe503fff640 1 -- 172.21.15.44:0/2683734834 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe510010460 con 0x7fe51c1050e0 2024-09-13T17:36:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 Processor -- start 2024-09-13T17:36:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 -- start start 2024-09-13T17:36:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7060a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac6fcbc60 con 0x562ac706fd90 2024-09-13T17:36:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac6fc93f0 con 0x562ac7060fc0 2024-09-13T17:36:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe523e47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac6fc4ce0 con 0x562ac70606a0 2024-09-13T17:36:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe51a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34046/0 (socket says 172.21.15.44:34046) 2024-09-13T17:36:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe51ad9d640 1 -- 172.21.15.44:0/412489898 learned_addr learned my addr 172.21.15.44:0/412489898 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe51a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51334/0 (socket says 172.21.15.44:51334) 2024-09-13T17:36:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe51b59e640 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7060a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.338+0000 7fe503fff640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5040a1200 0x7fe5040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51a59c640 1 -- 172.21.15.44:0/412489898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 msgr2=0x562ac7060a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51a59c640 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7060a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51a59c640 1 -- 172.21.15.44:0/412489898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 msgr2=0x562ac706f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51a59c640 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe503fff640 1 -- 172.21.15.44:0/2683734834 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe5100ccec0 con 0x7fe51c1050e0 2024-09-13T17:36:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51a59c640 1 -- 172.21.15.44:0/412489898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac6f80970 con 0x562ac706fd90 2024-09-13T17:36:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51b59e640 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7060a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:36:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:36:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5040a1200 0x7fe5040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe51a59c640 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7072180 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe5100ca2a0 tx=0x7fe510010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedc2088504a11ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe5017fa640 1 -- 172.21.15.44:0/412489898 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100995c0 con 0x562ac706fd90 2024-09-13T17:36:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe5017fa640 1 -- 172.21.15.44:0/412489898 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe510047630 con 0x562ac706fd90 2024-09-13T17:36:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.339+0000 7fe5017fa640 1 -- 172.21.15.44:0/412489898 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100a20e0 con 0x562ac706fd90 2024-09-13T17:36:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 -- 172.21.15.44:0/412489898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 msgr2=0x562ac7072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5040a1200 0x7fe5040a36c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe514005fd0 tx=0x7fe514004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7072180 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe5100ca2a0 tx=0x7fe510010bf0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 -- 172.21.15.44:0/412489898 shutdown_connections 2024-09-13T17:36:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7072180 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 --2- 172.21.15.44:0/412489898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7060a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 -- 172.21.15.44:0/412489898 >> 172.21.15.44:0/412489898 conn(0x562ac705cd50 msgr2=0x562ac705d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 -- 172.21.15.44:0/412489898 shutdown_connections 2024-09-13T17:36:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 -- 172.21.15.44:0/412489898 wait complete. 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 -- start start 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.340+0000 7fe523e47740 1 Processor -- start 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac706f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac706cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac6f82f80 con 0x562ac706fd90 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac6f82c20 con 0x562ac7060fc0 2024-09-13T17:36:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac6f4fab0 con 0x562ac70606a0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 10 client.0 _ll_register_callbacks cb 0x562ac6fbe350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac706cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac706cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51346/0 (socket says 172.21.15.44:51346) 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 -- 172.21.15.44:0/2059750658 learned_addr learned my addr 172.21.15.44:0/2059750658 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51a59c640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac706f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 -- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 msgr2=0x562ac706f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac706f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 -- 172.21.15.44:0/2059750658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 msgr2=0x562ac706c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 -- 172.21.15.44:0/2059750658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c1a31b0 con 0x562ac706fd90 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe51b59e640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac706cf70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe514015010 tx=0x7fe514004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af04b87c01ed7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe4feff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe4feff5640 1 -- 172.21.15.44:0/2059750658 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514022070 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe4feff5640 1 -- 172.21.15.44:0/2059750658 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe51401d0d0 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.341+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ac6cb8590 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe4feff5640 1 -- 172.21.15.44:0/2059750658 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51401d290 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 10 client.18753 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ac6fc3360 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe4feff5640 1 -- 172.21.15.44:0/2059750658 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562ac6cb8590 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe4feff5640 1 -- 172.21.15.44:0/2059750658 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe514005250 con 0x562ac706fd90 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 10 client.18753 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:35.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 20 client.18753 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac707fae0 0x562ac7081f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 4 client.18753 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ac6cc9860 con 0x562ac707fae0 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.342+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac707fae0 0x562ac7081f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe4feff5640 10 client.18753 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac707fae0 0x562ac7081f00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea056468680bd99 server_cookie=5f79461327eba4ab in_seq=0 out_seq=0 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe4feff5640 1 -- 172.21.15.44:0/2059750658 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562ac6cc9860 con 0x562ac707fae0 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe4feff5640 10 client.18753 handle_command_reply: tid=0 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 client.18753 shutdown 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 msgr2=0x562ac706cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac706cf70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe514015010 tx=0x7fe514004ab0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 shutdown_connections 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac707fae0 0x562ac7081f00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac706cf70 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac706c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 --2- 172.21.15.44:0/2059750658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac706f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 >> 172.21.15.44:0/2059750658 conn(0x562ac705cd50 msgr2=0x562ac705c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.343+0000 7fe4feff5640 0 client.18753 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.344+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 shutdown_connections 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.344+0000 7fe523e47740 1 -- 172.21.15.44:0/2059750658 wait complete. 2024-09-13T17:36:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.344+0000 7fe523e47740 20 client.18753 trim_cache size 0 max 16384 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 Processor -- start 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 -- start start 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7076a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7076f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7067cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7058830 con 0x562ac706fd90 2024-09-13T17:36:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac706fb60 con 0x562ac7060fc0 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe523e47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7061d40 con 0x562ac70606a0 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe51b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7067cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe51b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7067cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51362/0 (socket says 172.21.15.44:51362) 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.358+0000 7fe51b59e640 1 -- 172.21.15.44:0/671530442 learned_addr learned my addr 172.21.15.44:0/671530442 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51a59c640 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51b59e640 1 -- 172.21.15.44:0/671530442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 msgr2=0x562ac7076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51b59e640 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51b59e640 1 -- 172.21.15.44:0/671530442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 msgr2=0x562ac7076f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51b59e640 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7076f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51b59e640 1 -- 172.21.15.44:0/671530442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac6f651c0 con 0x562ac706fd90 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe51b59e640 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7067cb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fe514013ec0 tx=0x7fe514009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1541ac98c04e86af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe501ffb640 1 -- 172.21.15.44:0/671530442 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514022040 con 0x562ac706fd90 2024-09-13T17:36:35.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe501ffb640 1 -- 172.21.15.44:0/671530442 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe514005750 con 0x562ac706fd90 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.359+0000 7fe501ffb640 1 -- 172.21.15.44:0/671530442 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514005240 con 0x562ac706fd90 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- 172.21.15.44:0/671530442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 msgr2=0x562ac7067cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7067cb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fe514013ec0 tx=0x7fe514009930 comp rx=0 tx=0).stop 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- 172.21.15.44:0/671530442 shutdown_connections 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac706fd90 0x562ac7067cb0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7076f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- 172.21.15.44:0/671530442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac70606a0 0x562ac7076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- 172.21.15.44:0/671530442 >> 172.21.15.44:0/671530442 conn(0x562ac705cd50 msgr2=0x562ac7086860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- 172.21.15.44:0/671530442 shutdown_connections 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- 172.21.15.44:0/671530442 wait complete. 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- start start 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 Processor -- start 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 0x562ac7076a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7068c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac706fd90 0x562ac7069170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7076fa0 con 0x562ac70606a0 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7077110 con 0x562ac7060fc0 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac70772d0 con 0x562ac706fd90 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe523e47740 10 client.0 _ll_register_callbacks cb 0x7fe5140099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 0x562ac7076a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 0x562ac7076a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51372/0 (socket says 172.21.15.44:51372) 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.360+0000 7fe51ad9d640 1 -- 172.21.15.44:0/982583593 learned_addr learned my addr 172.21.15.44:0/982583593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51b59e640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac706fd90 0x562ac7069170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51a59c640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51ad9d640 1 -- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac706fd90 msgr2=0x562ac7069170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac706fd90 0x562ac7069170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51ad9d640 1 -- 172.21.15.44:0/982583593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 msgr2=0x562ac7068c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7068c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51ad9d640 1 -- 172.21.15.44:0/982583593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ac6fc3360 con 0x562ac70606a0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51a59c640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7068c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe51ad9d640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 0x562ac7076a60 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005ec0 tx=0x7fe50c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e684dadd9e6a5b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe4feff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe4feff5640 1 -- 172.21.15.44:0/982583593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c042050 con 0x562ac70606a0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe4feff5640 1 -- 172.21.15.44:0/982583593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe50c044d70 con 0x562ac70606a0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe510047630 con 0x562ac70606a0 2024-09-13T17:36:35.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe4feff5640 1 -- 172.21.15.44:0/982583593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50c031910 con 0x562ac70606a0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe523e47740 10 client.18759 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.361+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ac6f8a990 con 0x562ac70606a0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe4feff5640 1 -- 172.21.15.44:0/982583593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe510047630 con 0x562ac70606a0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe4feff5640 1 -- 172.21.15.44:0/982583593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe50c0035f0 con 0x562ac70606a0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe523e47740 10 client.18759 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe523e47740 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe523e47740 20 client.18759 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe523e47740 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac70d6fc0 0x562ac70d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe523e47740 4 client.18759 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562ac6d1ceb0 con 0x562ac70d6fc0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe51a59c640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac70d6fc0 0x562ac70d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe51a59c640 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac70d6fc0 0x562ac70d93e0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31c75ef56966aa5 server_cookie=5a654f1678c2c353 in_seq=0 out_seq=0 2024-09-13T17:36:35.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.362+0000 7fe4feff5640 10 client.18759 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.363+0000 7fe4feff5640 1 -- 172.21.15.44:0/982583593 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562ac6d1ceb0 con 0x562ac70d6fc0 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.363+0000 7fe4feff5640 10 client.18759 handle_command_reply: tid=0 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.363+0000 7fe523e47740 1 client.18759 shutdown 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.363+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 msgr2=0x562ac7076a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.363+0000 7fe523e47740 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 0x562ac7076a60 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005ec0 tx=0x7fe50c0041a0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 shutdown_connections 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x562ac70d6fc0 0x562ac70d93e0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ac706fd90 0x562ac7069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562ac7060fc0 0x562ac7068c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 --2- 172.21.15.44:0/982583593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac70606a0 0x562ac7076a60 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 >> 172.21.15.44:0/982583593 conn(0x562ac705cd50 msgr2=0x562ac7067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe4feff5640 0 client.18759 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 shutdown_connections 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 1 -- 172.21.15.44:0/982583593 wait complete. 2024-09-13T17:36:35.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe523e47740 20 client.18759 trim_cache size 0 max 16384 2024-09-13T17:36:35.372 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:36:35.372 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:36:35.372 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "a7f3a318-c55a-43bc-9e7e-e86ffa3c34db", 2024-09-13T17:36:35.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:36:35.372 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5040a1200 msgr2=0x7fe5040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5040a1200 0x7fe5040a36c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe514005fd0 tx=0x7fe514004a40 comp rx=0 tx=0).stop 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 msgr2=0x7fe51c196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 0x7fe51c196760 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d770 tx=0x7fe51000dc30 comp rx=0 tx=0).stop 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 shutdown_connections 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe5040a1200 0x7fe5040a36c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe51c19ab00 0x7fe51c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe51c1050e0 0x7fe51c196760 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 --2- 172.21.15.44:0/2683734834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe51c0fff90 0x7fe51c196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 >> 172.21.15.44:0/2683734834 conn(0x7fe51c0fba20 msgr2=0x7fe51c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 shutdown_connections 2024-09-13T17:36:35.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.364+0000 7fe52111f640 1 -- 172.21.15.44:0/2683734834 wait complete. 2024-09-13T17:36:35.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:36:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 -- 172.21.15.44:0/367588348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4100940 msgr2=0x7fe6a4104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 --2- 172.21.15.44:0/367588348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4100940 0x7fe6a4104980 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe694009930 tx=0x7fe69402a420 comp rx=0 tx=0).stop 2024-09-13T17:36:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 -- 172.21.15.44:0/367588348 shutdown_connections 2024-09-13T17:36:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 --2- 172.21.15.44:0/367588348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 0x7fe6a410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 --2- 172.21.15.44:0/367588348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4100940 0x7fe6a4104980 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 --2- 172.21.15.44:0/367588348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a40fff90 0x7fe6a4100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 -- 172.21.15.44:0/367588348 >> 172.21.15.44:0/367588348 conn(0x7fe6a40fba20 msgr2=0x7fe6a40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 -- 172.21.15.44:0/367588348 shutdown_connections 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.481+0000 7fe6ac525640 1 -- 172.21.15.44:0/367588348 wait complete. 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 Processor -- start 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 -- start start 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a40fff90 0x7fe6a40727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 0x7fe6a4072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 0x7fe6a41a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a49a0 con 0x7fe6a4073230 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a4b10 con 0x7fe6a41050e0 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6ac525640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a4cd0 con 0x7fe6a40fff90 2024-09-13T17:36:35.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 0x7fe6a41a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 0x7fe6a41a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51382/0 (socket says 172.21.15.44:51382) 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 -- 172.21.15.44:0/794729325 learned_addr learned my addr 172.21.15.44:0/794729325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 0x7fe6a4072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a40fff90 0x7fe6a40727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 -- 172.21.15.44:0/794729325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a40fff90 msgr2=0x7fe6a40727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a40fff90 0x7fe6a40727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 -- 172.21.15.44:0/794729325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 msgr2=0x7fe6a4072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 0x7fe6a4072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6aaac2640 1 -- 172.21.15.44:0/794729325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694009560 con 0x7fe6a4073230 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.482+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 0x7fe6a4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.483+0000 7fe6aaac2640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 0x7fe6a41a43d0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009fd0 tx=0x7fe69c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.483+0000 7fe69b7fe640 1 -- 172.21.15.44:0/794729325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00bea0 con 0x7fe6a4073230 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.483+0000 7fe69b7fe640 1 -- 172.21.15.44:0/794729325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe69c009940 con 0x7fe6a4073230 2024-09-13T17:36:35.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.483+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a41a4fb0 con 0x7fe6a4073230 2024-09-13T17:36:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.483+0000 7fe69b7fe640 1 -- 172.21.15.44:0/794729325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c010480 con 0x7fe6a4073230 2024-09-13T17:36:35.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.483+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a41a53e0 con 0x7fe6a4073230 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.485+0000 7fe6af252740 1 Processor -- start 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.485+0000 7fe69b7fe640 1 -- 172.21.15.44:0/794729325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe69c003970 con 0x7fe6a4073230 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.485+0000 7fe6af252740 1 -- start start 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6af252740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563583fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6af252740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x556358405a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6af252740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6af252740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556358365f20 con 0x556358405fd0 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6af252740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563583636b0 con 0x5563583fae10 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6af252740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635835efa0 con 0x5563583fb730 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51396/0 (socket says 172.21.15.44:51396) 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/743100347 learned_addr learned my addr 172.21.15.44:0/743100347 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x556358405a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563583fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe69b7fe640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6900a1230 0x7fe6900a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/743100347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 msgr2=0x556358405a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x556358405a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/743100347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 msgr2=0x5563583fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563583fb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/743100347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556358319840 con 0x556358405fd0 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6aaac2640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6900a1230 0x7fe6900a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584083c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40695f0 tx=0x7fe694005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8431c5e1b4cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.486+0000 7fe69b7fe640 1 -- 172.21.15.44:0/794729325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe69c0cccc0 con 0x7fe6a4073230 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe698ff9640 1 -- 172.21.15.44:0/743100347 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694044070 con 0x556358405fd0 2024-09-13T17:36:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe698ff9640 1 -- 172.21.15.44:0/743100347 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe694031960 con 0x556358405fd0 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe698ff9640 1 -- 172.21.15.44:0/743100347 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694003ae0 con 0x556358405fd0 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 -- 172.21.15.44:0/743100347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 msgr2=0x5563584083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584083c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40695f0 tx=0x7fe694005b50 comp rx=0 tx=0).stop 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6aaac2640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6900a1230 0x7fe6900a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0ca150 tx=0x7fe69c094040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 -- 172.21.15.44:0/743100347 shutdown_connections 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x556358405a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584083c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 --2- 172.21.15.44:0/743100347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563583fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.487+0000 7fe6af252740 1 -- 172.21.15.44:0/743100347 >> 172.21.15.44:0/743100347 conn(0x5563583f7380 msgr2=0x5563583f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 -- 172.21.15.44:0/743100347 shutdown_connections 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 -- 172.21.15.44:0/743100347 wait complete. 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 -- start start 2024-09-13T17:36:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 Processor -- start 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 0x556358402d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556358405fd0 0x556358403330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563583002a0 con 0x5563583fb730 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556358037430 con 0x5563583fae10 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563582e2680 con 0x556358405fd0 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6af252740 10 client.0 _ll_register_callbacks cb 0x556358358610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6aa2c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 0x556358402d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6aa2c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 0x556358402d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51412/0 (socket says 172.21.15.44:51412) 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6aa2c1640 1 -- 172.21.15.44:0/1207149471 learned_addr learned my addr 172.21.15.44:0/1207149471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.488+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556358405fd0 0x556358403330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6aa2c1640 1 -- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556358405fd0 msgr2=0x556358403330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556358405fd0 0x556358403330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6aa2c1640 1 -- 172.21.15.44:0/1207149471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 msgr2=0x5563584059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6aa2c1640 1 -- 172.21.15.44:0/1207149471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a410d470 con 0x5563583fb730 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 0x556358402d20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000bf80 tx=0x7fe6a00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eac16156d0cca6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe68dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe68dffb640 1 -- 172.21.15.44:0/1207149471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0010900 con 0x5563583fb730 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe68dffb640 1 -- 172.21.15.44:0/1207149471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe6a0010a60 con 0x5563583fb730 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563580524e0 con 0x5563583fb730 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe68dffb640 1 -- 172.21.15.44:0/1207149471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6a0010c20 con 0x5563583fb730 2024-09-13T17:36:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6af252740 10 client.18771 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55635835d620 con 0x5563583fb730 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.489+0000 7fe68dffb640 1 -- 172.21.15.44:0/1207149471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5563580524e0 con 0x5563583fb730 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe68dffb640 1 -- 172.21.15.44:0/1207149471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6a000d350 con 0x5563583fb730 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6af252740 10 client.18771 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6af252740 10 client.18771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6af252740 20 client.18771 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6af252740 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556358415eb0 0x5563584182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6af252740 4 client.18771 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563580637e0 con 0x556358415eb0 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556358415eb0 0x5563584182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556358415eb0 0x5563584182d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22930d10be31df08 server_cookie=1a1580368ad34fa9 in_seq=0 out_seq=0 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.490+0000 7fe68dffb640 10 client.18771 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe68dffb640 1 -- 172.21.15.44:0/1207149471 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5563580637e0 con 0x556358415eb0 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe68dffb640 10 client.18771 handle_command_reply: tid=0 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 client.18771 shutdown 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 msgr2=0x556358402d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 0x556358402d20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000bf80 tx=0x7fe6a00128e0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 shutdown_connections 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x556358415eb0 0x5563584182d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556358405fd0 0x556358403330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563583fb730 0x556358402d20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 --2- 172.21.15.44:0/1207149471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe68dffb640 0 client.18771 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 >> 172.21.15.44:0/1207149471 conn(0x5563583f7380 msgr2=0x5563583fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 shutdown_connections 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 1 -- 172.21.15.44:0/1207149471 wait complete. 2024-09-13T17:36:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.491+0000 7fe6af252740 20 client.18771 trim_cache size 0 max 16384 2024-09-13T17:36:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 Processor -- start 2024-09-13T17:36:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 -- start start 2024-09-13T17:36:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x5563583fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563583fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556358411cb0 con 0x556358405fd0 2024-09-13T17:36:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563583fab90 con 0x5563583fae10 2024-09-13T17:36:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6af252740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635841d510 con 0x5563583fb730 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563583fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563583fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51416/0 (socket says 172.21.15.44:51416) 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/2468854002 learned_addr learned my addr 172.21.15.44:0/2468854002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x5563583fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/2468854002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 msgr2=0x5563583fdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x5563583fdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/2468854002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 msgr2=0x5563584054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.506+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/2468854002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563582ff790 con 0x556358405fd0 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563583fe510 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe694009930 tx=0x7fe694001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfe217f629cbb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6997fa640 1 -- 172.21.15.44:0/2468854002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694050040 con 0x556358405fd0 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6997fa640 1 -- 172.21.15.44:0/2468854002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe694056870 con 0x556358405fd0 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6997fa640 1 -- 172.21.15.44:0/2468854002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694044040 con 0x556358405fd0 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 -- 172.21.15.44:0/2468854002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 msgr2=0x5563583fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563583fe510 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe694009930 tx=0x7fe694001e70 comp rx=0 tx=0).stop 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 -- 172.21.15.44:0/2468854002 shutdown_connections 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fb730 0x5563583fdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563583fe510 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 --2- 172.21.15.44:0/2468854002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fae10 0x5563584054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 -- 172.21.15.44:0/2468854002 >> 172.21.15.44:0/2468854002 conn(0x5563583f7380 msgr2=0x5563583f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 -- 172.21.15.44:0/2468854002 shutdown_connections 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.507+0000 7fe6af252740 1 -- 172.21.15.44:0/2468854002 wait complete. 2024-09-13T17:36:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 -- start start 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 Processor -- start 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fae10 0x556358408d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fb730 0x556358409280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635846b540 con 0x556358405fd0 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635846b6b0 con 0x5563583fb730 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635846b870 con 0x5563583fae10 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6af252740 10 client.0 _ll_register_callbacks cb 0x7fe6a000b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51424/0 (socket says 172.21.15.44:51424) 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/214910028 learned_addr learned my addr 172.21.15.44:0/214910028 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fb730 0x556358409280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fae10 0x556358408d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fae10 msgr2=0x556358408d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fae10 0x556358408d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/214910028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fb730 msgr2=0x556358409280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fb730 0x556358409280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.508+0000 7fe6a9ac0640 1 -- 172.21.15.44:0/214910028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556358319840 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6a9ac0640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584097e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe694009900 tx=0x7fe694031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b52d248478e4d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe68dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe68dffb640 1 -- 172.21.15.44:0/214910028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694050030 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe68dffb640 1 -- 172.21.15.44:0/214910028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe694004140 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe694031960 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe68dffb640 1 -- 172.21.15.44:0/214910028 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe694044070 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 10 client.18777 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556358324ea0 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe68dffb640 1 -- 172.21.15.44:0/214910028 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe69406c080 con 0x556358405fd0 2024-09-13T17:36:35.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe68dffb640 1 -- 172.21.15.44:0/214910028 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe694063440 con 0x556358405fd0 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 10 client.18777 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 20 client.18777 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55635846cbc0 0x55635846efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 4 client.18777 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.509+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5563580b6e30 con 0x55635846cbc0 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.510+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55635846cbc0 0x55635846efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.510+0000 7fe6aa2c1640 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55635846cbc0 0x55635846efb0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c88fc11f896c42f server_cookie=696530bbbf31bd0d in_seq=0 out_seq=0 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.510+0000 7fe68dffb640 10 client.18777 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.510+0000 7fe68dffb640 1 -- 172.21.15.44:0/214910028 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5563580b6e30 con 0x55635846cbc0 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.510+0000 7fe68dffb640 10 client.18777 handle_command_reply: tid=0 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.510+0000 7fe6af252740 1 client.18777 shutdown 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 msgr2=0x5563584097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584097e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe694009900 tx=0x7fe694031870 comp rx=0 tx=0).stop 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 shutdown_connections 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55635846cbc0 0x55635846efb0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556358405fd0 0x5563584097e0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563583fb730 0x556358409280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 --2- 172.21.15.44:0/214910028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563583fae10 0x556358408d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe68dffb640 0 client.18777 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 >> 172.21.15.44:0/214910028 conn(0x5563583f7380 msgr2=0x5563583f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 shutdown_connections 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 1 -- 172.21.15.44:0/214910028 wait complete. 2024-09-13T17:36:35.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6af252740 20 client.18777 trim_cache size 0 max 16384 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: "a7f3a318-c55a-43bc-9e7e-e86ffa3c34db": { 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "a7f3a318-c55a-43bc-9e7e-e86ffa3c34db", 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:36:35.523 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:36:35.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6900a1230 msgr2=0x7fe6900a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6900a1230 0x7fe6900a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0ca150 tx=0x7fe69c094040 comp rx=0 tx=0).stop 2024-09-13T17:36:35.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 msgr2=0x7fe6a41a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:35.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 0x7fe6a41a43d0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009fd0 tx=0x7fe69c00b260 comp rx=0 tx=0).stop 2024-09-13T17:36:35.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 shutdown_connections 2024-09-13T17:36:35.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6900a1230 0x7fe6900a36f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a4073230 0x7fe6a41a43d0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a41050e0 0x7fe6a4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 --2- 172.21.15.44:0/794729325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a40fff90 0x7fe6a40727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:35.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 >> 172.21.15.44:0/794729325 conn(0x7fe6a40fba20 msgr2=0x7fe6a40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:35.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 shutdown_connections 2024-09-13T17:36:35.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:36:35.511+0000 7fe6ac525640 1 -- 172.21.15.44:0/794729325 wait complete. 2024-09-13T17:36:35.525 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:35 smithi044 ceph-mon[31175]: pgmap v307: 129 pgs: 129 active+clean; 2.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:36:35.526 INFO:tasks.cephfs.filesystem:scrub status for tag:a7f3a318-c55a-43bc-9e7e-e86ffa3c34db - {'path': '/', 'tag': 'a7f3a318-c55a-43bc-9e7e-e86ffa3c34db', 'options': 'recursive,force'} 2024-09-13T17:36:35.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:35 smithi070 ceph-mon[34502]: pgmap v307: 129 pgs: 129 active+clean; 2.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:36:35.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:35 smithi100 ceph-mon[35055]: pgmap v307: 129 pgs: 129 active+clean; 2.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:36:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:36 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:36:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:36 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:36 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:36:36.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:36 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:36.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:36 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:36:36.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:36 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:36.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:36 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:36:36.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:36 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:36.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:36 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:36:36.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:36 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:36.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:36 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:36:36.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:36 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:36:37.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:37 smithi070 ceph-mon[34502]: pgmap v308: 129 pgs: 129 active+clean; 2.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:37.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:37 smithi100 ceph-mon[35055]: pgmap v308: 129 pgs: 129 active+clean; 2.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:37.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:37 smithi044 ceph-mon[31175]: pgmap v308: 129 pgs: 129 active+clean; 2.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:36:39.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:39 smithi044 ceph-mon[31175]: pgmap v309: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:36:39.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:39 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:36:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:39 smithi070 ceph-mon[34502]: pgmap v309: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:36:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:39 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:36:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:39 smithi100 ceph-mon[35055]: pgmap v309: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:36:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:39 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:36:41.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:41 smithi044 ceph-mon[31175]: pgmap v310: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:36:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:41 smithi070 ceph-mon[34502]: pgmap v310: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:36:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:41 smithi100 ceph-mon[35055]: pgmap v310: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:36:43.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:43 smithi044 ceph-mon[31175]: pgmap v311: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:36:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:43 smithi070 ceph-mon[34502]: pgmap v311: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:36:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:43 smithi100 ceph-mon[35055]: pgmap v311: 129 pgs: 129 active+clean; 3.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 -- 172.21.15.44:0/186880320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41008b0 msgr2=0x7f4ed41048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/186880320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41008b0 0x7f4ed41048f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009930 tx=0x7f4ec402a420 comp rx=0 tx=0).stop 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 -- 172.21.15.44:0/186880320 shutdown_connections 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/186880320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105050 0x7f4ed410cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/186880320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41008b0 0x7f4ed41048f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/186880320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 0x7f4ed41002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 -- 172.21.15.44:0/186880320 >> 172.21.15.44:0/186880320 conn(0x7f4ed40fb970 msgr2=0x7f4ed40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 -- 172.21.15.44:0/186880320 shutdown_connections 2024-09-13T17:36:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.233+0000 7f4edb5ce640 1 -- 172.21.15.44:0/186880320 wait complete. 2024-09-13T17:36:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 Processor -- start 2024-09-13T17:36:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 -- start start 2024-09-13T17:36:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 0x7f4ed4196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 0x7f4ed41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed419aac0 0x7f4ed419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed419b4b0 con 0x7f4ed4105050 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed419b620 con 0x7f4ed40fff00 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4edb5ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed419b790 con 0x7f4ed419aac0 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4ed8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 0x7f4ed41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4ed8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 0x7f4ed41966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55766/0 (socket says 172.21.15.44:55766) 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.234+0000 7f4ed8b69640 1 -- 172.21.15.44:0/602014654 learned_addr learned my addr 172.21.15.44:0/602014654 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed936a640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 0x7f4ed4196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed419aac0 0x7f4ed419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed8b69640 1 -- 172.21.15.44:0/602014654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed419aac0 msgr2=0x7f4ed419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed419aac0 0x7f4ed419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed8b69640 1 -- 172.21.15.44:0/602014654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 msgr2=0x7f4ed4196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 0x7f4ed4196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed8b69640 1 -- 172.21.15.44:0/602014654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4009560 con 0x7f4ed4105050 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed419aac0 0x7f4ed419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed936a640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 0x7f4ed4196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 0x7f4ed41966d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005ab0 tx=0x7f4ec4032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4eca7fc640 1 -- 172.21.15.44:0/602014654 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4032e10 con 0x7f4ed4105050 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4eca7fc640 1 -- 172.21.15.44:0/602014654 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec40046c0 con 0x7f4ed4105050 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed41a2d70 con 0x7f4ed4105050 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4eca7fc640 1 -- 172.21.15.44:0/602014654 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4003850 con 0x7f4ed4105050 2024-09-13T17:36:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.235+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a31a0 con 0x7f4ed4105050 2024-09-13T17:36:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.238+0000 7f4eca7fc640 1 -- 172.21.15.44:0/602014654 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4ec4004830 con 0x7f4ed4105050 2024-09-13T17:36:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 Processor -- start 2024-09-13T17:36:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 -- start start 2024-09-13T17:36:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fcb50 0x5654f140b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f140dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f1367c30 con 0x5654f140b900 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f13653c0 con 0x5654f13fcb50 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f1360cb0 con 0x5654f13fc230 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4eca7fc640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ec00a1280 0x7f4ec00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ed8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f140dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ec00a1280 0x7f4ec00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ed8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f140dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55776/0 (socket says 172.21.15.44:55776) 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ed8b69640 1 -- 172.21.15.44:0/3212736060 learned_addr learned my addr 172.21.15.44:0/3212736060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ed936a640 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fcb50 0x5654f140b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.239+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4eca7fc640 1 -- 172.21.15.44:0/602014654 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4ec40edd00 con 0x7f4ed4105050 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed8b69640 1 -- 172.21.15.44:0/3212736060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 msgr2=0x5654f13fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13fc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed8b69640 1 -- 172.21.15.44:0/3212736060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fcb50 msgr2=0x5654f140b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fcb50 0x5654f140b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed8b69640 1 -- 172.21.15.44:0/3212736060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654f1307910 con 0x5654f140b900 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ec00a1280 0x7f4ec00a3740 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005c40 tx=0x7f4ecc009d50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f140dcf0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec408dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b7c40ff0771dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ebf7fe640 1 -- 172.21.15.44:0/3212736060 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40a4240 con 0x5654f140b900 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ebf7fe640 1 -- 172.21.15.44:0/3212736060 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec40bb140 con 0x5654f140b900 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.240+0000 7f4ebf7fe640 1 -- 172.21.15.44:0/3212736060 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40b9610 con 0x5654f140b900 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 -- 172.21.15.44:0/3212736060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 msgr2=0x5654f140dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f140dcf0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec408dbd0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 -- 172.21.15.44:0/3212736060 shutdown_connections 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f140dcf0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fcb50 0x5654f140b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/3212736060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 -- 172.21.15.44:0/3212736060 >> 172.21.15.44:0/3212736060 conn(0x5654f13f88c0 msgr2=0x5654f13f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 -- 172.21.15.44:0/3212736060 shutdown_connections 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 -- 172.21.15.44:0/3212736060 wait complete. 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 -- start start 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.241+0000 7f4ede3ca740 1 Processor -- start 2024-09-13T17:36:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 0x5654f140b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f14084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1408a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f131ee80 con 0x5654f13fc230 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f131eb20 con 0x5654f140b900 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f12e4260 con 0x5654f13fcb50 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ede3ca740 10 client.0 _ll_register_callbacks cb 0x5654f1319df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 0x5654f140b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 0x5654f140b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55788/0 (socket says 172.21.15.44:55788) 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 -- 172.21.15.44:0/1501217832 learned_addr learned my addr 172.21.15.44:0/1501217832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1408a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f14084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 -- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 msgr2=0x5654f14084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f14084d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 -- 172.21.15.44:0/1501217832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 msgr2=0x5654f1408a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1408a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.242+0000 7f4ed936a640 1 -- 172.21.15.44:0/1501217832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a31a0 con 0x5654f13fc230 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ed936a640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 0x5654f140b230 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0012480 tx=0x7f4ed0012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e87d8d6573ad279 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ebcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ebcff9640 1 -- 172.21.15.44:0/1501217832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0004440 con 0x5654f13fc230 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ebcff9640 1 -- 172.21.15.44:0/1501217832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4ed00035f0 con 0x5654f13fc230 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654f1389680 con 0x5654f13fc230 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ebcff9640 1 -- 172.21.15.44:0/1501217832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed00037b0 con 0x5654f13fc230 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 10 client.18789 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5654f135f330 con 0x5654f13fc230 2024-09-13T17:36:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ebcff9640 1 -- 172.21.15.44:0/1501217832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5654f1389680 con 0x5654f13fc230 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ebcff9640 1 -- 172.21.15.44:0/1501217832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ed0010c60 con 0x5654f13fc230 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 10 client.18789 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 10 client.18789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 20 client.18789 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f141b0e0 0x5654f141d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 4 client.18789 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.243+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654f13545d0 con 0x5654f141b0e0 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.244+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f141b0e0 0x5654f141d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.244+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f141b0e0 0x5654f141d500 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba2d4aa4fe82a91 server_cookie=7afb1a980b333e54 in_seq=0 out_seq=0 2024-09-13T17:36:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.244+0000 7f4ebcff9640 10 client.18789 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ebcff9640 1 -- 172.21.15.44:0/1501217832 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5654f13545d0 con 0x5654f141b0e0 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ebcff9640 10 client.18789 handle_command_reply: tid=0 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 client.18789 shutdown 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 msgr2=0x5654f140b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 0x5654f140b230 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0012480 tx=0x7f4ed0012940 comp rx=0 tx=0).stop 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 shutdown_connections 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f141b0e0 0x5654f141d500 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f14084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1408a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ebcff9640 0 client.18789 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1501217832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fc230 0x5654f140b230 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 >> 172.21.15.44:0/1501217832 conn(0x5654f13f88c0 msgr2=0x5654f13fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 shutdown_connections 2024-09-13T17:36:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1501217832 wait complete. 2024-09-13T17:36:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.245+0000 7f4ede3ca740 20 client.18789 trim_cache size 0 max 16384 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 Processor -- start 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 -- start start 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13f4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 0x5654f140ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1405710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f13fede0 con 0x5654f13fcb50 2024-09-13T17:36:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f14198f0 con 0x5654f140b900 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f1417720 con 0x5654f13fc230 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ed8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 0x5654f140ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ed8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 0x5654f140ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55790/0 (socket says 172.21.15.44:55790) 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.258+0000 7f4ed8b69640 1 -- 172.21.15.44:0/1788513221 learned_addr learned my addr 172.21.15.44:0/1788513221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed936a640 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1405710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed8b69640 1 -- 172.21.15.44:0/1788513221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 msgr2=0x5654f13f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed8b69640 1 -- 172.21.15.44:0/1788513221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 msgr2=0x5654f1405710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1405710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed8b69640 1 -- 172.21.15.44:0/1788513221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654f1301300 con 0x5654f13fcb50 2024-09-13T17:36:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1405710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 0x5654f140ad40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4006010 tx=0x7f4ec40a5830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0211af65a61b734 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ebffff640 1 -- 172.21.15.44:0/1788513221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40a52b0 con 0x5654f13fcb50 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ebffff640 1 -- 172.21.15.44:0/1788513221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec40b6d80 con 0x5654f13fcb50 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ebffff640 1 -- 172.21.15.44:0/1788513221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4004b10 con 0x5654f13fcb50 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.259+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1788513221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 msgr2=0x5654f140ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 0x5654f140ad40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4006010 tx=0x7f4ec40a5830 comp rx=0 tx=0).stop 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1788513221 shutdown_connections 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f140b900 0x5654f1405710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f13fcb50 0x5654f140ad40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/1788513221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fc230 0x5654f13f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1788513221 >> 172.21.15.44:0/1788513221 conn(0x5654f13f88c0 msgr2=0x5654f13f3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1788513221 shutdown_connections 2024-09-13T17:36:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- 172.21.15.44:0/1788513221 wait complete. 2024-09-13T17:36:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- start start 2024-09-13T17:36:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 Processor -- start 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fc230 0x5654f14120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f1412630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f14106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f1410dc0 con 0x5654f140b900 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f1410f30 con 0x5654f13fc230 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.260+0000 7f4ede3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f14110f0 con 0x5654f13fcb50 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ede3ca740 10 client.0 _ll_register_callbacks cb 0x7f4ec4005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f14106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f14106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55794/0 (socket says 172.21.15.44:55794) 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 -- 172.21.15.44:0/54346711 learned_addr learned my addr 172.21.15.44:0/54346711 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed8b69640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f1412630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed936a640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fc230 0x5654f14120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 -- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 msgr2=0x5654f1412630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f1412630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 -- 172.21.15.44:0/54346711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fc230 msgr2=0x5654f14120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fc230 0x5654f14120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 -- 172.21.15.44:0/54346711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654f1307910 con 0x5654f140b900 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ed9b6b640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f14106c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc010d20 tx=0x7f4ecc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3390c5760059e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ebcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ebcff9640 1 -- 172.21.15.44:0/54346711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc02b400 con 0x5654f140b900 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ebcff9640 1 -- 172.21.15.44:0/54346711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4ecc01e3f0 con 0x5654f140b900 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec40bb140 con 0x5654f140b900 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ebcff9640 1 -- 172.21.15.44:0/54346711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc01e5f0 con 0x5654f140b900 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ede3ca740 10 client.18795 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.261+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5654f13268e0 con 0x5654f140b900 2024-09-13T17:36:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ebcff9640 1 -- 172.21.15.44:0/54346711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec40bb140 con 0x5654f140b900 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ebcff9640 1 -- 172.21.15.44:0/54346711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ecc02b780 con 0x5654f140b900 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ede3ca740 10 client.18795 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ede3ca740 10 client.18795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ede3ca740 20 client.18795 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f1472850 0x5654f1474c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ede3ca740 4 client.18795 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5654f1054300 con 0x5654f1472850 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.262+0000 7f4ed936a640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f1472850 0x5654f1474c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.263+0000 7f4ed936a640 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f1472850 0x5654f1474c70 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e488e8eb9ec79321 server_cookie=8617a7f8ddcd0c6b in_seq=0 out_seq=0 2024-09-13T17:36:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.263+0000 7f4ebcff9640 10 client.18795 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ebcff9640 1 -- 172.21.15.44:0/54346711 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5654f1054300 con 0x5654f1472850 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ebcff9640 10 client.18795 handle_command_reply: tid=0 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 client.18795 shutdown 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 msgr2=0x5654f14106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f14106c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc010d20 tx=0x7f4ecc005bd0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 shutdown_connections 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5654f1472850 0x5654f1474c70 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654f13fcb50 0x5654f1412630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654f140b900 0x5654f14106c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 --2- 172.21.15.44:0/54346711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5654f13fc230 0x5654f14120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ebcff9640 0 client.18795 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 >> 172.21.15.44:0/54346711 conn(0x5654f13f88c0 msgr2=0x5654f13fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 shutdown_connections 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.275+0000 7f4ede3ca740 1 -- 172.21.15.44:0/54346711 wait complete. 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4ede3ca740 20 client.18795 trim_cache size 0 max 16384 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:36:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:36:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ec00a1280 msgr2=0x7f4ec00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ec00a1280 0x7f4ec00a3740 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005c40 tx=0x7f4ecc009d50 comp rx=0 tx=0).stop 2024-09-13T17:36:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 msgr2=0x7f4ed41966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 0x7f4ed41966d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005ab0 tx=0x7f4ec4032c20 comp rx=0 tx=0).stop 2024-09-13T17:36:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 shutdown_connections 2024-09-13T17:36:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4ec00a1280 0x7f4ec00a3740 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed419aac0 0x7f4ed419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed4105050 0x7f4ed41966d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 --2- 172.21.15.44:0/602014654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed40fff00 0x7f4ed4196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 >> 172.21.15.44:0/602014654 conn(0x7f4ed40fb970 msgr2=0x7f4ed40fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 shutdown_connections 2024-09-13T17:36:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:44.276+0000 7f4edb5ce640 1 -- 172.21.15.44:0/602014654 wait complete. 2024-09-13T17:36:45.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:45 smithi044 ceph-mon[31175]: pgmap v312: 129 pgs: 129 active+clean; 3.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-09-13T17:36:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:45 smithi070 ceph-mon[34502]: pgmap v312: 129 pgs: 129 active+clean; 3.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-09-13T17:36:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:45 smithi100 ceph-mon[35055]: pgmap v312: 129 pgs: 129 active+clean; 3.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-09-13T17:36:47.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:47 smithi044 ceph-mon[31175]: pgmap v313: 129 pgs: 129 active+clean; 3.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-09-13T17:36:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:47 smithi070 ceph-mon[34502]: pgmap v313: 129 pgs: 129 active+clean; 3.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-09-13T17:36:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:47 smithi100 ceph-mon[35055]: pgmap v313: 129 pgs: 129 active+clean; 3.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-09-13T17:36:49.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:49 smithi044 ceph-mon[31175]: pgmap v314: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-13T17:36:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:49 smithi070 ceph-mon[34502]: pgmap v314: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-13T17:36:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:49 smithi100 ceph-mon[35055]: pgmap v314: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-13T17:36:51.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:51 smithi044 ceph-mon[31175]: pgmap v315: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:36:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:51 smithi070 ceph-mon[34502]: pgmap v315: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:36:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:51 smithi100 ceph-mon[35055]: pgmap v315: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:36:53.949 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:53 smithi070 ceph-mon[34502]: pgmap v316: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:36:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:53 smithi100 ceph-mon[35055]: pgmap v316: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:36:54.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:53 smithi044 ceph-mon[31175]: pgmap v316: 129 pgs: 129 active+clean; 3.3 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-13T17:36:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:55 smithi070 ceph-mon[34502]: pgmap v317: 129 pgs: 129 active+clean; 3.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:36:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:55 smithi100 ceph-mon[35055]: pgmap v317: 129 pgs: 129 active+clean; 3.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:36:56.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:55 smithi044 ceph-mon[31175]: pgmap v317: 129 pgs: 129 active+clean; 3.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 -- 172.21.15.44:0/3390445946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9441008a0 msgr2=0x7fe9441048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 --2- 172.21.15.44:0/3390445946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9441008a0 0x7fe9441048e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe938009930 tx=0x7fe93802a420 comp rx=0 tx=0).stop 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 -- 172.21.15.44:0/3390445946 shutdown_connections 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 --2- 172.21.15.44:0/3390445946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe944105040 0x7fe94410cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 --2- 172.21.15.44:0/3390445946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9441008a0 0x7fe9441048e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 --2- 172.21.15.44:0/3390445946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9440ffef0 0x7fe9441002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 -- 172.21.15.44:0/3390445946 >> 172.21.15.44:0/3390445946 conn(0x7fe9440fb960 msgr2=0x7fe9440fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 -- 172.21.15.44:0/3390445946 shutdown_connections 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.368+0000 7fe94c672640 1 -- 172.21.15.44:0/3390445946 wait complete. 2024-09-13T17:36:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 Processor -- start 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 -- start start 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9440ffef0 0x7fe9441961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 0x7fe9441966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe94419aad0 0x7fe94419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94419b4c0 con 0x7fe944105040 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94419b630 con 0x7fe94419aad0 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94c672640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94419b7a0 con 0x7fe9440ffef0 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe949c0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 0x7fe9441966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe949c0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 0x7fe9441966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47092/0 (socket says 172.21.15.44:47092) 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe949c0d640 1 -- 172.21.15.44:0/827359006 learned_addr learned my addr 172.21.15.44:0/827359006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.369+0000 7fe94a40e640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9440ffef0 0x7fe9441961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe94419aad0 0x7fe94419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe949c0d640 1 -- 172.21.15.44:0/827359006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9440ffef0 msgr2=0x7fe9441961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe949c0d640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9440ffef0 0x7fe9441961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe949c0d640 1 -- 172.21.15.44:0/827359006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe94419aad0 msgr2=0x7fe94419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe949c0d640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe94419aad0 0x7fe94419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe949c0d640 1 -- 172.21.15.44:0/827359006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938009560 con 0x7fe944105040 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe94419aad0 0x7fe94419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe949c0d640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 0x7fe9441966e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe938005ab0 tx=0x7fe9380329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe9377fe640 1 -- 172.21.15.44:0/827359006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938032df0 con 0x7fe944105040 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe9377fe640 1 -- 172.21.15.44:0/827359006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe938005490 con 0x7fe944105040 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9441a2d80 con 0x7fe944105040 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe9377fe640 1 -- 172.21.15.44:0/827359006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380035f0 con 0x7fe944105040 2024-09-13T17:36:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.370+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9441a31a0 con 0x7fe944105040 2024-09-13T17:36:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.372+0000 7fe9377fe640 1 -- 172.21.15.44:0/827359006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe9380047c0 con 0x7fe944105040 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 Processor -- start 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 -- start start 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0201ec30 con 0x559c020be4f0 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0201c3c0 con 0x559c020b3c50 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94f474740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c02017cb0 con 0x559c020b3330 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe9377fe640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9300a1200 0x7fe9300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe949c0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9300a1200 0x7fe9300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe949c0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47096/0 (socket says 172.21.15.44:47096) 2024-09-13T17:36:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe949c0d640 1 -- 172.21.15.44:0/2467650406 learned_addr learned my addr 172.21.15.44:0/2467650406 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94a40e640 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.373+0000 7fe9377fe640 1 -- 172.21.15.44:0/827359006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe9380edba0 con 0x7fe944105040 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94a40e640 1 -- 172.21.15.44:0/2467650406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 msgr2=0x559c020b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94a40e640 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94a40e640 1 -- 172.21.15.44:0/2467650406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 msgr2=0x559c020c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94a40e640 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94a40e640 1 -- 172.21.15.44:0/2467650406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c01fbe910 con 0x559c020b3c50 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9300a1200 0x7fe9300a36c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009fd0 tx=0x7fe93c0091e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe94a40e640 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020bdfb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe940009c00 tx=0x7fe940011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c9a399f102d88e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe934ff9640 1 -- 172.21.15.44:0/2467650406 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940004230 con 0x559c020b3c50 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe934ff9640 1 -- 172.21.15.44:0/2467650406 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe940009e70 con 0x559c020b3c50 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.374+0000 7fe934ff9640 1 -- 172.21.15.44:0/2467650406 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940004b20 con 0x559c020b3c50 2024-09-13T17:36:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 -- 172.21.15.44:0/2467650406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 msgr2=0x559c020bdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020bdfb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe940009c00 tx=0x7fe940011810 comp rx=0 tx=0).stop 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 -- 172.21.15.44:0/2467650406 shutdown_connections 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020bdfb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 --2- 172.21.15.44:0/2467650406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 -- 172.21.15.44:0/2467650406 >> 172.21.15.44:0/2467650406 conn(0x559c020af8c0 msgr2=0x559c020afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 -- 172.21.15.44:0/2467650406 shutdown_connections 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 -- 172.21.15.44:0/2467650406 wait complete. 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 -- start start 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.375+0000 7fe94f474740 1 Processor -- start 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 0x559c020bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3c50 0x559c020bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020be4f0 0x559c020bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c01fd5e80 con 0x559c020b3330 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c01fd5b20 con 0x559c020be4f0 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c01f9b260 con 0x559c020b3c50 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 10 client.0 _ll_register_callbacks cb 0x559c01fd0df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 0x559c020bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 0x559c020bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47108/0 (socket says 172.21.15.44:47108) 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 -- 172.21.15.44:0/226207465 learned_addr learned my addr 172.21.15.44:0/226207465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe949c0d640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3c50 0x559c020bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020be4f0 0x559c020bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 -- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3c50 msgr2=0x559c020bb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3c50 0x559c020bb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 -- 172.21.15.44:0/226207465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020be4f0 msgr2=0x559c020bb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020be4f0 0x559c020bb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 -- 172.21.15.44:0/226207465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe940009830 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94a40e640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 0x559c020bde50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe940004620 tx=0x7fe940019ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fc47472b6ad0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe92dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe92dffb640 1 -- 172.21.15.44:0/226207465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940024470 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c02040680 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.376+0000 7fe92dffb640 1 -- 172.21.15.44:0/226207465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe9400245d0 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe92dffb640 1 -- 172.21.15.44:0/226207465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe940024790 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 10 client.18807 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c02016330 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe92dffb640 1 -- 172.21.15.44:0/226207465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c02040680 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe92dffb640 1 -- 172.21.15.44:0/226207465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9400238a0 con 0x559c020b3330 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 10 client.18807 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 10 client.18807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 20 client.18807 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c020cdea0 0x559c020d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 4 client.18807 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c0200b5d0 con 0x559c020cdea0 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.377+0000 7fe949c0d640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c020cdea0 0x559c020d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe92dffb640 10 client.18807 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe949c0d640 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c020cdea0 0x559c020d0290 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b36570b2c59495 server_cookie=4960dd7cc7580d5a in_seq=0 out_seq=0 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe92dffb640 1 -- 172.21.15.44:0/226207465 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559c0200b5d0 con 0x559c020cdea0 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe92dffb640 10 client.18807 handle_command_reply: tid=0 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe94f474740 1 client.18807 shutdown 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 msgr2=0x559c020bde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe94f474740 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 0x559c020bde50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe940004620 tx=0x7fe940019ad0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.378+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 shutdown_connections 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c020cdea0 0x559c020d0290 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3c50 0x559c020bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe92dffb640 0 client.18807 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020be4f0 0x559c020bb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 --2- 172.21.15.44:0/226207465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3330 0x559c020bde50 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 >> 172.21.15.44:0/226207465 conn(0x559c020af8c0 msgr2=0x559c020aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 shutdown_connections 2024-09-13T17:36:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 1 -- 172.21.15.44:0/226207465 wait complete. 2024-09-13T17:36:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.379+0000 7fe94f474740 20 client.18807 trim_cache size 0 max 16384 2024-09-13T17:36:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 Processor -- start 2024-09-13T17:36:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 -- start start 2024-09-13T17:36:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3330 0x559c020bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 0x559c020c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020be4f0 0x559c020c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c020cc740 con 0x559c020b3c50 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c020b4780 con 0x559c020b3330 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94f474740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c020cab90 con 0x559c020be4f0 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe949c0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 0x559c020c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe949c0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 0x559c020c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47118/0 (socket says 172.21.15.44:47118) 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe949c0d640 1 -- 172.21.15.44:0/2496639069 learned_addr learned my addr 172.21.15.44:0/2496639069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.391+0000 7fe94a40e640 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3330 0x559c020bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020be4f0 0x559c020c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe949c0d640 1 -- 172.21.15.44:0/2496639069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020be4f0 msgr2=0x559c020c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe949c0d640 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020be4f0 0x559c020c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe949c0d640 1 -- 172.21.15.44:0/2496639069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3330 msgr2=0x559c020bd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe949c0d640 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3330 0x559c020bd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe949c0d640 1 -- 172.21.15.44:0/2496639069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c01fb8300 con 0x559c020b3c50 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe94a40e640 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3330 0x559c020bd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe949c0d640 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 0x559c020c51f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe9380eb030 tx=0x7fe9380da040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4381e043f94eeb80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe9357fa640 1 -- 172.21.15.44:0/2496639069 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380ba940 con 0x559c020b3c50 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe9357fa640 1 -- 172.21.15.44:0/2496639069 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9380a5040 con 0x559c020b3c50 2024-09-13T17:36:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.392+0000 7fe9357fa640 1 -- 172.21.15.44:0/2496639069 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380eb390 con 0x559c020b3c50 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- 172.21.15.44:0/2496639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 msgr2=0x559c020c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 0x559c020c51f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe9380eb030 tx=0x7fe9380da040 comp rx=0 tx=0).stop 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- 172.21.15.44:0/2496639069 shutdown_connections 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020be4f0 0x559c020c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020b3c50 0x559c020c51f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- 172.21.15.44:0/2496639069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3330 0x559c020bd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- 172.21.15.44:0/2496639069 >> 172.21.15.44:0/2496639069 conn(0x559c020af8c0 msgr2=0x559c020b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- 172.21.15.44:0/2496639069 shutdown_connections 2024-09-13T17:36:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- 172.21.15.44:0/2496639069 wait complete. 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- start start 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 Processor -- start 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c02124490 con 0x559c020be4f0 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c02124600 con 0x559c020b3c50 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c021247c0 con 0x559c020b3330 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94f474740 10 client.0 _ll_register_callbacks cb 0x7fe940009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94ac0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94ac0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47124/0 (socket says 172.21.15.44:47124) 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94ac0f640 1 -- 172.21.15.44:0/457872050 learned_addr learned my addr 172.21.15.44:0/457872050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe949c0d640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.393+0000 7fe94a40e640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94ac0f640 1 -- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 msgr2=0x559c020c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94ac0f640 1 -- 172.21.15.44:0/457872050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 msgr2=0x559c020c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94ac0f640 1 -- 172.21.15.44:0/457872050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c02016330 con 0x559c020be4f0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94ac0f640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c4820 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe93c003bf0 tx=0x7fe93c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a20f770ec27bcb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe92dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe92dffb640 1 -- 172.21.15.44:0/457872050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c004300 con 0x559c020be4f0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe92dffb640 1 -- 172.21.15.44:0/457872050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe93c004460 con 0x559c020be4f0 2024-09-13T17:36:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe940009e70 con 0x559c020be4f0 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe92dffb640 1 -- 172.21.15.44:0/457872050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe93c004620 con 0x559c020be4f0 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94f474740 10 client.18813 fetch_fsmap learned FSMap version 25 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe9441a31a0 con 0x559c020be4f0 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe92dffb640 1 -- 172.21.15.44:0/457872050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe940009e70 con 0x559c020be4f0 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.394+0000 7fe92dffb640 1 -- 172.21.15.44:0/457872050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe93c02a020 con 0x559c020be4f0 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94f474740 10 client.18813 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94f474740 10 client.18813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:36:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94f474740 20 client.18813 populate_metadata read hostname 'smithi044' 2024-09-13T17:36:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94f474740 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c02126050 0x559c02128470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:36:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94f474740 4 client.18813 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:36:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c01d0b300 con 0x559c02126050 2024-09-13T17:36:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94a40e640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c02126050 0x559c02128470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:36:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe92dffb640 10 client.18813 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.395+0000 7fe94a40e640 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c02126050 0x559c02128470 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=911df855c8c02b42 server_cookie=84cc2fe8df8c2677 in_seq=0 out_seq=0 2024-09-13T17:36:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe92dffb640 1 -- 172.21.15.44:0/457872050 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c01d0b300 con 0x559c02126050 2024-09-13T17:36:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe92dffb640 10 client.18813 handle_command_reply: tid=0 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 client.18813 shutdown 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 msgr2=0x559c020c4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c4820 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe93c003bf0 tx=0x7fe93c004ab0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 shutdown_connections 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x559c02126050 0x559c02128470 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c020be4f0 0x559c020c4820 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559c020b3c50 0x559c020c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 --2- 172.21.15.44:0/457872050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c020b3330 0x559c020c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe92dffb640 0 client.18813 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 >> 172.21.15.44:0/457872050 conn(0x559c020af8c0 msgr2=0x559c020b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 shutdown_connections 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 1 -- 172.21.15.44:0/457872050 wait complete. 2024-09-13T17:36:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.406+0000 7fe94f474740 20 client.18813 trim_cache size 0 max 16384 2024-09-13T17:36:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:36:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:36:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:36:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9300a1200 msgr2=0x7fe9300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9300a1200 0x7fe9300a36c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009fd0 tx=0x7fe93c0091e0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 msgr2=0x7fe9441966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 0x7fe9441966e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe938005ab0 tx=0x7fe9380329f0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 shutdown_connections 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe9300a1200 0x7fe9300a36c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe94419aad0 0x7fe94419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe944105040 0x7fe9441966e0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 --2- 172.21.15.44:0/827359006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9440ffef0 0x7fe9441961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:36:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 >> 172.21.15.44:0/827359006 conn(0x7fe9440fb960 msgr2=0x7fe9440fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:36:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 shutdown_connections 2024-09-13T17:36:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:36:57.407+0000 7fe94c672640 1 -- 172.21.15.44:0/827359006 wait complete. 2024-09-13T17:36:57.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:57 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:36:57.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:57 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:36:57.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:57 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:36:58.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: pgmap v318: 129 pgs: 129 active+clean; 3.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:36:58.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:36:58.611 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:58 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: pgmap v318: 129 pgs: 129 active+clean; 3.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:36:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:36:58.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:36:58.700 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:58 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: pgmap v318: 129 pgs: 129 active+clean; 3.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:36:58.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T17:36:58.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:36:58.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:36:58.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:58 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:36:59.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:59 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi070 to 176.0M 2024-09-13T17:36:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:59 smithi044 ceph-mon[31175]: Unable to set osd_memory_target on smithi070 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-13T17:36:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:59 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-13T17:36:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:59 smithi044 ceph-mon[31175]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-13T17:36:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:59 smithi044 ceph-mon[31175]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-09-13T17:36:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:36:59 smithi044 ceph-mon[31175]: Unable to set osd_memory_target on smithi044 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-13T17:36:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:59 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi070 to 176.0M 2024-09-13T17:36:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:59 smithi070 ceph-mon[34502]: Unable to set osd_memory_target on smithi070 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-13T17:36:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:59 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-13T17:36:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:59 smithi070 ceph-mon[34502]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-13T17:36:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:59 smithi070 ceph-mon[34502]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-09-13T17:36:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:36:59 smithi070 ceph-mon[34502]: Unable to set osd_memory_target on smithi044 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-13T17:36:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:59 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi070 to 176.0M 2024-09-13T17:36:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:59 smithi100 ceph-mon[35055]: Unable to set osd_memory_target on smithi070 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-13T17:36:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:59 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-13T17:36:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:59 smithi100 ceph-mon[35055]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-13T17:36:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:59 smithi100 ceph-mon[35055]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-09-13T17:36:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:36:59 smithi100 ceph-mon[35055]: Unable to set osd_memory_target on smithi044 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-13T17:37:00.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:00 smithi044 ceph-mon[31175]: pgmap v319: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-13T17:37:00.675 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:00 smithi100 ceph-mon[35055]: pgmap v319: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-13T17:37:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:00 smithi070 ceph-mon[34502]: pgmap v319: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-13T17:37:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:02 smithi044 ceph-mon[31175]: pgmap v320: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:02 smithi070 ceph-mon[34502]: pgmap v320: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:02 smithi100 ceph-mon[35055]: pgmap v320: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:03.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:03 smithi044 ceph-mon[31175]: pgmap v321: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:03.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:03 smithi070 ceph-mon[34502]: pgmap v321: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:03.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:03 smithi100 ceph-mon[35055]: pgmap v321: 129 pgs: 129 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:05.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:37:05.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:05 smithi044 ceph-mon[31175]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T17:37:05.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:05 smithi044 ceph-mon[31175]: pgmap v323: 129 pgs: 129 active+clean; 4.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 -- 172.21.15.44:0/1847500185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 msgr2=0x7fcb70109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1847500185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb70109d40 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009900 tx=0x7fcb5c02a480 comp rx=0 tx=0).stop 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 -- 172.21.15.44:0/1847500185 shutdown_connections 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1847500185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7010a310 0x7fcb7010c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1847500185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb70109d40 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1847500185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100f40 0x7fcb70101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.635+0000 7fcb771ae640 1 -- 172.21.15.44:0/1847500185 >> 172.21.15.44:0/1847500185 conn(0x7fcb70075c60 msgr2=0x7fcb700fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 -- 172.21.15.44:0/1847500185 shutdown_connections 2024-09-13T17:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 -- 172.21.15.44:0/1847500185 wait complete. 2024-09-13T17:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 Processor -- start 2024-09-13T17:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 -- start start 2024-09-13T17:37:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100f40 0x7fcb700feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb700ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7010a310 0x7fcb700ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb701a4530 con 0x7fcb70101860 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb701a46a0 con 0x7fcb7010a310 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb771ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb701a4860 con 0x7fcb70100f40 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb700ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb700ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40284/0 (socket says 172.21.15.44:40284) 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.636+0000 7fcb6ffff640 1 -- 172.21.15.44:0/1689432198 learned_addr learned my addr 172.21.15.44:0/1689432198 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100f40 0x7fcb700feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb7574b640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7010a310 0x7fcb700ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6ffff640 1 -- 172.21.15.44:0/1689432198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100f40 msgr2=0x7fcb700feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100f40 0x7fcb700feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6ffff640 1 -- 172.21.15.44:0/1689432198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7010a310 msgr2=0x7fcb700ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7010a310 0x7fcb700ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6ffff640 1 -- 172.21.15.44:0/1689432198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009560 con 0x7fcb70101860 2024-09-13T17:37:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb700ff0d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0057c0 tx=0x7fcb5c032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6dffb640 1 -- 172.21.15.44:0/1689432198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c005000 con 0x7fcb70101860 2024-09-13T17:37:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb701a4ae0 con 0x7fcb70101860 2024-09-13T17:37:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6dffb640 1 -- 172.21.15.44:0/1689432198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb5c0046c0 con 0x7fcb70101860 2024-09-13T17:37:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb6dffb640 1 -- 172.21.15.44:0/1689432198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c003850 con 0x7fcb70101860 2024-09-13T17:37:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.637+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb701a4f60 con 0x7fcb70101860 2024-09-13T17:37:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.640+0000 7fcb6dffb640 1 -- 172.21.15.44:0/1689432198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb5c005160 con 0x7fcb70101860 2024-09-13T17:37:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.640+0000 7fcb79ece740 1 Processor -- start 2024-09-13T17:37:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 -- start start 2024-09-13T17:37:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b1301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b13aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b13d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b09af20 con 0x55725b130730 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b0986b0 con 0x55725b12fe10 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb79ece740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b093fa0 con 0x55725b13afd0 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b13aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b13aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40292/0 (socket says 172.21.15.44:40292) 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1196988569 learned_addr learned my addr 172.21.15.44:0/1196988569 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b13d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb7574b640 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b1301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb6dffb640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb600a1200 0x7fcb600a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1196988569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 msgr2=0x55725b13d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b13d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1196988569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 msgr2=0x55725b1301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b1301f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1196988569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725b04e840 con 0x55725b130730 2024-09-13T17:37:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.641+0000 7fcb7574b640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb600a1200 0x7fcb600a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.642+0000 7fcb6dffb640 1 -- 172.21.15.44:0/1689432198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fcb5c0edc30 con 0x7fcb70101860 2024-09-13T17:37:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.642+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b13aa90 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400bd10 tx=0x7fcb640107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8477c20d12ed122e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.642+0000 7fcb5affd640 1 -- 172.21.15.44:0/1196988569 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64004030 con 0x55725b130730 2024-09-13T17:37:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.642+0000 7fcb5affd640 1 -- 172.21.15.44:0/1196988569 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb64015070 con 0x55725b130730 2024-09-13T17:37:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.642+0000 7fcb5affd640 1 -- 172.21.15.44:0/1196988569 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640116a0 con 0x55725b130730 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- 172.21.15.44:0/1196988569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 msgr2=0x55725b13aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb7574b640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb600a1200 0x7fcb600a36c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcb68005fd0 tx=0x7fcb68004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b13aa90 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400bd10 tx=0x7fcb640107a0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- 172.21.15.44:0/1196988569 shutdown_connections 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b13d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b13aa90 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1196988569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b1301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- 172.21.15.44:0/1196988569 >> 172.21.15.44:0/1196988569 conn(0x55725b12c380 msgr2=0x55725b12c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- 172.21.15.44:0/1196988569 shutdown_connections 2024-09-13T17:37:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- 172.21.15.44:0/1196988569 wait complete. 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- start start 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 Processor -- start 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b13a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b137c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b0352a0 con 0x55725b130730 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725ad6c430 con 0x55725b12fe10 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.643+0000 7fcb79ece740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b017680 con 0x55725b13afd0 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb79ece740 10 client.0 _ll_register_callbacks cb 0x55725b08d610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40294/0 (socket says 172.21.15.44:40294) 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 -- 172.21.15.44:0/3837474990 learned_addr learned my addr 172.21.15.44:0/3837474990 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb7574b640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b13a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 -- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 msgr2=0x55725b138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 -- 172.21.15.44:0/3837474990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 msgr2=0x55725b13a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b13a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 -- 172.21.15.44:0/3837474990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6400b940 con 0x55725b130730 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b137c40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c02aa70 tx=0x7fcb5c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91911132cfbe729a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb4ffff640 1 -- 172.21.15.44:0/3837474990 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0bb980 con 0x55725b130730 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb4ffff640 1 -- 172.21.15.44:0/3837474990 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fcb5c0bb070 con 0x55725b130730 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55725ad874e0 con 0x55725b130730 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb4ffff640 1 -- 172.21.15.44:0/3837474990 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5c005410 con 0x55725b130730 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb79ece740 10 client.18825 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.644+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55725b092620 con 0x55725b130730 2024-09-13T17:37:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb4ffff640 1 -- 172.21.15.44:0/3837474990 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55725ad874e0 con 0x55725b130730 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb4ffff640 1 -- 172.21.15.44:0/3837474990 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb5c0b7080 con 0x55725b130730 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb79ece740 10 client.18825 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb79ece740 10 client.18825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb79ece740 20 client.18825 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb79ece740 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b14ab70 0x55725b14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb79ece740 4 client.18825 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55725ad987e0 con 0x55725b14ab70 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.645+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b14ab70 0x55725b14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b14ab70 0x55725b14cf60 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f035f4588ba5b39d server_cookie=e752e236c7bb06ab in_seq=0 out_seq=0 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb4ffff640 10 client.18825 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb4ffff640 1 -- 172.21.15.44:0/3837474990 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55725ad987e0 con 0x55725b14ab70 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb4ffff640 10 client.18825 handle_command_reply: tid=0 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb79ece740 1 client.18825 shutdown 2024-09-13T17:37:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 msgr2=0x55725b137c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.646+0000 7fcb79ece740 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b137c40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c02aa70 tx=0x7fcb5c005720 comp rx=0 tx=0).stop 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 shutdown_connections 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b14ab70 0x55725b14cf60 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b137c40 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 --2- 172.21.15.44:0/3837474990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b13a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 >> 172.21.15.44:0/3837474990 conn(0x55725b12c380 msgr2=0x55725b1320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb4ffff640 0 client.18825 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 shutdown_connections 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 1 -- 172.21.15.44:0/3837474990 wait complete. 2024-09-13T17:37:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.647+0000 7fcb79ece740 20 client.18825 trim_cache size 0 max 16384 2024-09-13T17:37:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 Processor -- start 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 -- start start 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b12bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b12c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b126690 con 0x55725b130730 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b136170 con 0x55725b12fe10 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb79ece740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b13ace0 con 0x55725b13afd0 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b12c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b12c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40302/0 (socket says 172.21.15.44:40302) 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b12bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.662+0000 7fcb6ffff640 1 -- 172.21.15.44:0/1981425974 learned_addr learned my addr 172.21.15.44:0/1981425974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb7574b640 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1981425974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 msgr2=0x55725b1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1981425974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 msgr2=0x55725b12c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b12c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb74f4a640 1 -- 172.21.15.44:0/1981425974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725b034790 con 0x55725b12fe10 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b12bd00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400bce0 tx=0x7fcb6400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130b9ffef43c11e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb5b7fe640 1 -- 172.21.15.44:0/1981425974 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64020050 con 0x55725b12fe10 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb5b7fe640 1 -- 172.21.15.44:0/1981425974 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb64021040 con 0x55725b12fe10 2024-09-13T17:37:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.663+0000 7fcb5b7fe640 1 -- 172.21.15.44:0/1981425974 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64028110 con 0x55725b12fe10 2024-09-13T17:37:05.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 -- 172.21.15.44:0/1981425974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 msgr2=0x55725b12bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b12bd00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400bce0 tx=0x7fcb6400b390 comp rx=0 tx=0).stop 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 -- 172.21.15.44:0/1981425974 shutdown_connections 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b13afd0 0x55725b1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b130730 0x55725b12c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 --2- 172.21.15.44:0/1981425974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b12bd00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 -- 172.21.15.44:0/1981425974 >> 172.21.15.44:0/1981425974 conn(0x55725b12c380 msgr2=0x55725b13a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 -- 172.21.15.44:0/1981425974 shutdown_connections 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 -- 172.21.15.44:0/1981425974 wait complete. 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.664+0000 7fcb79ece740 1 -- start start 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 Processor -- start 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b141d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b130730 0x55725b142270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 0x55725b13e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b13ebd0 con 0x55725b13afd0 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b13ed40 con 0x55725b12fe10 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725b13ef00 con 0x55725b130730 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb79ece740 10 client.0 _ll_register_callbacks cb 0x7fcb5c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 0x55725b13e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 0x55725b13e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40306/0 (socket says 172.21.15.44:40306) 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 -- 172.21.15.44:0/2053521796 learned_addr learned my addr 172.21.15.44:0/2053521796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b130730 0x55725b142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 -- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b130730 msgr2=0x55725b142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b130730 0x55725b142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 -- 172.21.15.44:0/2053521796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 msgr2=0x55725b141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.665+0000 7fcb7574b640 1 -- 172.21.15.44:0/2053521796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55725b04e840 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb7574b640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 0x55725b13e4d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fcb68013e10 tx=0x7fcb68004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b4427a0e216745 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb4ffff640 1 -- 172.21.15.44:0/2053521796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68024070 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb4ffff640 1 -- 172.21.15.44:0/2053521796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fcb6802a660 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb64015070 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb4ffff640 1 -- 172.21.15.44:0/2053521796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb6802a820 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 10 client.18831 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55725b059ea0 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb4ffff640 1 -- 172.21.15.44:0/2053521796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb64015070 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb4ffff640 1 -- 172.21.15.44:0/2053521796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb6801da80 con 0x55725b13afd0 2024-09-13T17:37:05.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 10 client.18831 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 10 client.18831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 20 client.18831 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.666+0000 7fcb79ece740 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b1a24a0 0x55725b1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb79ece740 4 client.18831 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55725adebe30 con 0x55725b1a24a0 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b1a24a0 0x55725b1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb74f4a640 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b1a24a0 0x55725b1a48c0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7d22b34eeea4c2 server_cookie=2cf02f67e5b09d75 in_seq=0 out_seq=0 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb4ffff640 10 client.18831 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb4ffff640 1 -- 172.21.15.44:0/2053521796 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55725adebe30 con 0x55725b1a24a0 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb4ffff640 10 client.18831 handle_command_reply: tid=0 2024-09-13T17:37:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.667+0000 7fcb79ece740 1 client.18831 shutdown 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 msgr2=0x55725b13e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 0x55725b13e4d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fcb68013e10 tx=0x7fcb68004ab0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 shutdown_connections 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55725b1a24a0 0x55725b1a48c0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55725b130730 0x55725b142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55725b13afd0 0x55725b13e4d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb4ffff640 0 client.18831 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 --2- 172.21.15.44:0/2053521796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55725b12fe10 0x55725b141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 >> 172.21.15.44:0/2053521796 conn(0x55725b12c380 msgr2=0x55725b1518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 shutdown_connections 2024-09-13T17:37:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 1 -- 172.21.15.44:0/2053521796 wait complete. 2024-09-13T17:37:05.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb79ece740 20 client.18831 trim_cache size 0 max 16384 2024-09-13T17:37:05.677 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:37:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:37:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:37:05.677 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb600a1200 msgr2=0x7fcb600a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb600a1200 0x7fcb600a36c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcb68005fd0 tx=0x7fcb68004a40 comp rx=0 tx=0).stop 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 msgr2=0x7fcb700ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb700ff0d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0057c0 tx=0x7fcb5c032be0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 shutdown_connections 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb600a1200 0x7fcb600a36c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7010a310 0x7fcb700ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb70101860 0x7fcb700ff0d0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.668+0000 7fcb771ae640 1 --2- 172.21.15.44:0/1689432198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100f40 0x7fcb700feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.669+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 >> 172.21.15.44:0/1689432198 conn(0x7fcb70075c60 msgr2=0x7fcb700fb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.669+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 shutdown_connections 2024-09-13T17:37:05.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.669+0000 7fcb771ae640 1 -- 172.21.15.44:0/1689432198 wait complete. 2024-09-13T17:37:05.681 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:37:05.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:37:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:05 smithi070 ceph-mon[34502]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T17:37:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:05 smithi070 ceph-mon[34502]: pgmap v323: 129 pgs: 129 active+clean; 4.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-13T17:37:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:05 smithi100 ceph-mon[35055]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T17:37:05.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:05 smithi100 ceph-mon[35055]: pgmap v323: 129 pgs: 129 active+clean; 4.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 -- 172.21.15.44:0/264203380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f144810a3d0 msgr2=0x7f144810c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 --2- 172.21.15.44:0/264203380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f144810a3d0 0x7f144810c850 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f1440009ca0 tx=0x7f144002a950 comp rx=0 tx=0).stop 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 -- 172.21.15.44:0/264203380 shutdown_connections 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 --2- 172.21.15.44:0/264203380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14481018a0 0x7f1448109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 --2- 172.21.15.44:0/264203380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f144810a3d0 0x7f144810c850 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 --2- 172.21.15.44:0/264203380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1448100f80 0x7f1448101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 -- 172.21.15.44:0/264203380 >> 172.21.15.44:0/264203380 conn(0x7f14480fa970 msgr2=0x7f14480fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 -- 172.21.15.44:0/264203380 shutdown_connections 2024-09-13T17:37:05.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.784+0000 7f144da2b640 1 -- 172.21.15.44:0/264203380 wait complete. 2024-09-13T17:37:05.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 Processor -- start 2024-09-13T17:37:05.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 -- start start 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 0x7f1448196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14481018a0 0x7f1448196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144810a3d0 0x7f144819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144819b150 con 0x7f1448100f80 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144819b2c0 con 0x7f14481018a0 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f144da2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144819b480 con 0x7f144810a3d0 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f1446ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 0x7f1448196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f1446ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 0x7f1448196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40332/0 (socket says 172.21.15.44:40332) 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.786+0000 7f1446ffd640 1 -- 172.21.15.44:0/1103513640 learned_addr learned my addr 172.21.15.44:0/1103513640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f14477fe640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144810a3d0 0x7f144819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f14467fc640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14481018a0 0x7f1448196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f1446ffd640 1 -- 172.21.15.44:0/1103513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144810a3d0 msgr2=0x7f144819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f1446ffd640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144810a3d0 0x7f144819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f1446ffd640 1 -- 172.21.15.44:0/1103513640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14481018a0 msgr2=0x7f1448196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f1446ffd640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14481018a0 0x7f1448196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f1446ffd640 1 -- 172.21.15.44:0/1103513640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009900 con 0x7f1448100f80 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f14467fc640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14481018a0 0x7f1448196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f1446ffd640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 0x7f1448196220 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f143800d950 tx=0x7f143800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438009c30 con 0x7f1448100f80 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144819b760 con 0x7f1448100f80 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1438009d90 con 0x7f1448100f80 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800f620 con 0x7f1448100f80 2024-09-13T17:37:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.787+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14481a2fe0 con 0x7f1448100f80 2024-09-13T17:37:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.788+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144819b9d0 con 0x7f1448100f80 2024-09-13T17:37:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.789+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f14380109f0 con 0x7f1448100f80 2024-09-13T17:37:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.790+0000 7f144ca29640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f14300a1200 0x7f14300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.790+0000 7f14467fc640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f14300a1200 0x7f14300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:05.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.790+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f14380ccdc0 con 0x7f1448100f80 2024-09-13T17:37:05.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.791+0000 7f14467fc640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f14300a1200 0x7f14300a36c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f14481977c0 tx=0x7f143c005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:05.797+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1438095e40 con 0x7f1448100f80 2024-09-13T17:37:06.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.005+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14481054c0 con 0x7f1448100f80 2024-09-13T17:37:06.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.006+0000 7f144ca29640 1 -- 172.21.15.44:0/1103513640 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f1438020090 con 0x7f1448100f80 2024-09-13T17:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:37:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:37:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f14300a1200 msgr2=0x7f14300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f14300a1200 0x7f14300a36c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f14481977c0 tx=0x7f143c005f90 comp rx=0 tx=0).stop 2024-09-13T17:37:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 msgr2=0x7f1448196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 0x7f1448196220 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f143800d950 tx=0x7f143800de10 comp rx=0 tx=0).stop 2024-09-13T17:37:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 shutdown_connections 2024-09-13T17:37:06.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f14300a1200 0x7f14300a36c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144810a3d0 0x7f144819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14481018a0 0x7f1448196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 --2- 172.21.15.44:0/1103513640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1448100f80 0x7f1448196220 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 >> 172.21.15.44:0/1103513640 conn(0x7f14480fa970 msgr2=0x7f14480fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 shutdown_connections 2024-09-13T17:37:06.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.008+0000 7f144da2b640 1 -- 172.21.15.44:0/1103513640 wait complete. 2024-09-13T17:37:06.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:37:06.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 -- 172.21.15.44:0/2665030926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc10a310 msgr2=0x7f5bfc10c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 --2- 172.21.15.44:0/2665030926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc10a310 0x7f5bfc10c790 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009ca0 tx=0x7f5bf402a950 comp rx=0 tx=0).stop 2024-09-13T17:37:06.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 -- 172.21.15.44:0/2665030926 shutdown_connections 2024-09-13T17:37:06.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 --2- 172.21.15.44:0/2665030926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc10a310 0x7f5bfc10c790 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 --2- 172.21.15.44:0/2665030926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc101860 0x7f5bfc109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 --2- 172.21.15.44:0/2665030926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bfc100f40 0x7f5bfc101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 -- 172.21.15.44:0/2665030926 >> 172.21.15.44:0/2665030926 conn(0x7f5bfc075c60 msgr2=0x7f5bfc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.126+0000 7f5c03a27640 1 -- 172.21.15.44:0/2665030926 shutdown_connections 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 -- 172.21.15.44:0/2665030926 wait complete. 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 Processor -- start 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 -- start start 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bfc100f40 0x7f5bfc0feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 0x7f5bfc0ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc10a310 0x7f5bfc0ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a4410 con 0x7f5bfc101860 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a4580 con 0x7f5bfc10a310 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c03a27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a4740 con 0x7f5bfc100f40 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c00fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 0x7f5bfc0ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c00fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 0x7f5bfc0ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40348/0 (socket says 172.21.15.44:40348) 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.127+0000 7f5c00fc2640 1 -- 172.21.15.44:0/4148822952 learned_addr learned my addr 172.21.15.44:0/4148822952 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c017c3640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bfc100f40 0x7f5bfc0feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc10a310 0x7f5bfc0ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c00fc2640 1 -- 172.21.15.44:0/4148822952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bfc100f40 msgr2=0x7f5bfc0feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bfc100f40 0x7f5bfc0feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c00fc2640 1 -- 172.21.15.44:0/4148822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc10a310 msgr2=0x7f5bfc0ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc10a310 0x7f5bfc0ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c00fc2640 1 -- 172.21.15.44:0/4148822952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4009900 con 0x7f5bfc101860 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc10a310 0x7f5bfc0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 0x7f5bfc0ff0d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c920 tx=0x7f5bec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5bf27fc640 1 -- 172.21.15.44:0/4148822952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec007c70 con 0x7f5bfc101860 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5bf27fc640 1 -- 172.21.15.44:0/4148822952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bec007dd0 con 0x7f5bfc101860 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc1a4a20 con 0x7f5bfc101860 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5bf27fc640 1 -- 172.21.15.44:0/4148822952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec018730 con 0x7f5bfc101860 2024-09-13T17:37:06.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.128+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc1a4f30 con 0x7f5bfc101860 2024-09-13T17:37:06.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5bf27fc640 1 -- 172.21.15.44:0/4148822952 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5bec00f450 con 0x7f5bfc101860 2024-09-13T17:37:06.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 Processor -- start 2024-09-13T17:37:06.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 -- start start 2024-09-13T17:37:06.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d61b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d6c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 0x55ddc8d6ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8ccbcc0 con 0x55ddc8d6c950 2024-09-13T17:37:06.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8cc9450 con 0x55ddc8d61790 2024-09-13T17:37:06.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.131+0000 7f5c06823740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8cc4d40 con 0x55ddc8d620b0 2024-09-13T17:37:06.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 0x55ddc8d6ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5bf27fc640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5be80a1200 0x7f5be80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 0x55ddc8d6ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40360/0 (socket says 172.21.15.44:40360) 2024-09-13T17:37:06.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3157947760 learned_addr learned my addr 172.21.15.44:0/3157947760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d61b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c017c3640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d6c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5be80a1200 0x7f5be80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5bf27fc640 1 -- 172.21.15.44:0/4148822952 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f5bec0cd2b0 con 0x7f5bfc101860 2024-09-13T17:37:06.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3157947760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 msgr2=0x55ddc8d6c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d6c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3157947760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 msgr2=0x55ddc8d61b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d61b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3157947760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc8c8c620 con 0x55ddc8d6c950 2024-09-13T17:37:06.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d61b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:06.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c017c3640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d6c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:06.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 0x55ddc8d6ed40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0a1ee0 tx=0x7f5bec00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba8a1366941411e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5be80a1200 0x7f5be80a36c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009430 tx=0x7f5bf40095c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5be77fe640 1 -- 172.21.15.44:0/3157947760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec099ad0 con 0x55ddc8d6c950 2024-09-13T17:37:06.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5be77fe640 1 -- 172.21.15.44:0/3157947760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bec0998f0 con 0x55ddc8d6c950 2024-09-13T17:37:06.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.132+0000 7f5be77fe640 1 -- 172.21.15.44:0/3157947760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec09ddc0 con 0x55ddc8d6c950 2024-09-13T17:37:06.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 -- 172.21.15.44:0/3157947760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 msgr2=0x55ddc8d6ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 0x55ddc8d6ed40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0a1ee0 tx=0x7f5bec00fce0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 -- 172.21.15.44:0/3157947760 shutdown_connections 2024-09-13T17:37:06.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d6c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d6c950 0x55ddc8d6ed40 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 --2- 172.21.15.44:0/3157947760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d61b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 -- 172.21.15.44:0/3157947760 >> 172.21.15.44:0/3157947760 conn(0x55ddc8d5dd00 msgr2=0x55ddc8d5e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 -- 172.21.15.44:0/3157947760 shutdown_connections 2024-09-13T17:37:06.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 -- 172.21.15.44:0/3157947760 wait complete. 2024-09-13T17:37:06.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.133+0000 7f5c06823740 1 -- start start 2024-09-13T17:37:06.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 Processor -- start 2024-09-13T17:37:06.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d6bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d620b0 0x55ddc8d69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d69780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8c83180 con 0x55ddc8d61790 2024-09-13T17:37:06.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc899d430 con 0x55ddc8d620b0 2024-09-13T17:37:06.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8c485e0 con 0x55ddc8d6c950 2024-09-13T17:37:06.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c06823740 10 client.0 _ll_register_callbacks cb 0x55ddc8cb4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:06.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d6bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d6bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40362/0 (socket says 172.21.15.44:40362) 2024-09-13T17:37:06.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 -- 172.21.15.44:0/817431516 learned_addr learned my addr 172.21.15.44:0/817431516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d69780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d620b0 0x55ddc8d69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 -- 172.21.15.44:0/817431516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 msgr2=0x55ddc8d69780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d69780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 -- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d620b0 msgr2=0x55ddc8d69240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d620b0 0x55ddc8d69240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c017c3640 1 -- 172.21.15.44:0/817431516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc1a4f30 con 0x55ddc8d61790 2024-09-13T17:37:06.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d69780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.134+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d620b0 0x55ddc8d69240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c017c3640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d6bf20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80126b0 tx=0x7f5bf8012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e503dd192f9c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5be4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:06.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5be4ff9640 1 -- 172.21.15.44:0/817431516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8017070 con 0x55ddc8d61790 2024-09-13T17:37:06.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5be4ff9640 1 -- 172.21.15.44:0/817431516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f5bf8004260 con 0x55ddc8d61790 2024-09-13T17:37:06.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddc8ceeb80 con 0x55ddc8d61790 2024-09-13T17:37:06.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5be4ff9640 1 -- 172.21.15.44:0/817431516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bf8004420 con 0x55ddc8d61790 2024-09-13T17:37:06.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 10 client.18849 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ddc8cc33c0 con 0x55ddc8d61790 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5be4ff9640 1 -- 172.21.15.44:0/817431516 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ddc8ceeb80 con 0x55ddc8d61790 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5be4ff9640 1 -- 172.21.15.44:0/817431516 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bf8003bb0 con 0x55ddc8d61790 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 10 client.18849 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 10 client.18849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 20 client.18849 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8d7c260 0x55ddc8d7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 4 client.18849 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.135+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddc8cb71f0 con 0x55ddc8d7c260 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.136+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8d7c260 0x55ddc8d7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.136+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8d7c260 0x55ddc8d7e680 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40ece2867c659793 server_cookie=e951d2ffd0a37dd1 in_seq=0 out_seq=0 2024-09-13T17:37:06.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.136+0000 7f5be4ff9640 10 client.18849 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5be4ff9640 1 -- 172.21.15.44:0/817431516 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ddc8cb71f0 con 0x55ddc8d7c260 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5be4ff9640 10 client.18849 handle_command_reply: tid=0 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 client.18849 shutdown 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 msgr2=0x55ddc8d6bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d6bf20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80126b0 tx=0x7f5bf8012b70 comp rx=0 tx=0).stop 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 shutdown_connections 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8d7c260 0x55ddc8d7e680 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d69780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d620b0 0x55ddc8d69240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5be4ff9640 0 client.18849 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:06.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 --2- 172.21.15.44:0/817431516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d6bf20 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 >> 172.21.15.44:0/817431516 conn(0x55ddc8d5dd00 msgr2=0x55ddc8d63a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 shutdown_connections 2024-09-13T17:37:06.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 1 -- 172.21.15.44:0/817431516 wait complete. 2024-09-13T17:37:06.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.137+0000 7f5c06823740 20 client.18849 trim_cache size 0 max 16384 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.148+0000 7f5c06823740 1 Processor -- start 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.148+0000 7f5c06823740 1 -- start start 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c06823740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c06823740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 0x55ddc8d784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c06823740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d64200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c06823740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8d69af0 con 0x55ddc8d620b0 2024-09-13T17:37:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c06823740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8d61580 con 0x55ddc8d61790 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c06823740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8d67770 con 0x55ddc8d6c950 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 0x55ddc8d784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 0x55ddc8d784f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40364/0 (socket says 172.21.15.44:40364) 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3142844988 learned_addr learned my addr 172.21.15.44:0/3142844988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c017c3640 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3142844988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 msgr2=0x55ddc8d64200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d64200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3142844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 msgr2=0x55ddc8d77fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d77fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 -- 172.21.15.44:0/3142844988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc8c65770 con 0x55ddc8d620b0 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c017c3640 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 0x55ddc8d784f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5bec007fd0 tx=0x7f5bec01f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0965e31ddad40b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5be7fff640 1 -- 172.21.15.44:0/3142844988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec063070 con 0x55ddc8d620b0 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5be7fff640 1 -- 172.21.15.44:0/3142844988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bec09a690 con 0x55ddc8d620b0 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.149+0000 7f5be7fff640 1 -- 172.21.15.44:0/3142844988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec09a020 con 0x55ddc8d620b0 2024-09-13T17:37:06.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 -- 172.21.15.44:0/3142844988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 msgr2=0x55ddc8d784f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 0x55ddc8d784f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5bec007fd0 tx=0x7f5bec01f300 comp rx=0 tx=0).stop 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 -- 172.21.15.44:0/3142844988 shutdown_connections 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d6c950 0x55ddc8d64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d620b0 0x55ddc8d784f0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 --2- 172.21.15.44:0/3142844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d61790 0x55ddc8d77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 -- 172.21.15.44:0/3142844988 >> 172.21.15.44:0/3142844988 conn(0x55ddc8d5dd00 msgr2=0x55ddc8d83020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 -- 172.21.15.44:0/3142844988 shutdown_connections 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 -- 172.21.15.44:0/3142844988 wait complete. 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 -- start start 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.150+0000 7f5c06823740 1 Processor -- start 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d73620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d6c950 0x55ddc8d71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8d724e0 con 0x55ddc8d61790 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8d72650 con 0x55ddc8d6c950 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc8d727c0 con 0x55ddc8d620b0 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c06823740 10 client.0 _ll_register_callbacks cb 0x7f5bec099d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d73620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d73620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40368/0 (socket says 172.21.15.44:40368) 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 -- 172.21.15.44:0/929228007 learned_addr learned my addr 172.21.15.44:0/929228007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c01fc4640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d6c950 0x55ddc8d71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 -- 172.21.15.44:0/929228007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 msgr2=0x55ddc8d71a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d71a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 -- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d6c950 msgr2=0x55ddc8d71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d6c950 0x55ddc8d71fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.151+0000 7f5c017c3640 1 -- 172.21.15.44:0/929228007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddc8c8c620 con 0x55ddc8d61790 2024-09-13T17:37:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c017c3640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d73620 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8000c00 tx=0x7f5bf8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ce76c68a8bfba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5be4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5be4ff9640 1 -- 172.21.15.44:0/929228007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8017040 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5be4ff9640 1 -- 172.21.15.44:0/929228007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f5bf8011ce0 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bec0998f0 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5be4ff9640 1 -- 172.21.15.44:0/929228007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bf8011ea0 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 10 client.18855 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ddc8c8ae80 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5be4ff9640 1 -- 172.21.15.44:0/929228007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bec0998f0 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5be4ff9640 1 -- 172.21.15.44:0/929228007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bf800f9b0 con 0x55ddc8d61790 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 10 client.18855 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 10 client.18855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 20 client.18855 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:06.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8dd3b40 0x55ddc8dd5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 4 client.18855 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.152+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddc89b84e0 con 0x55ddc8dd3b40 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8dd3b40 0x55ddc8dd5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5be4ff9640 10 client.18855 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5c00fc2640 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8dd3b40 0x55ddc8dd5f60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b41da6e9bcfec396 server_cookie=2902678e15b04669 in_seq=0 out_seq=0 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5be4ff9640 1 -- 172.21.15.44:0/929228007 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddc89b84e0 con 0x55ddc8dd3b40 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5be4ff9640 10 client.18855 handle_command_reply: tid=0 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5c06823740 1 client.18855 shutdown 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 msgr2=0x55ddc8d73620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.153+0000 7f5c06823740 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d73620 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8000c00 tx=0x7f5bf8009640 comp rx=0 tx=0).stop 2024-09-13T17:37:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 shutdown_connections 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x55ddc8dd3b40 0x55ddc8dd5f60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc8d620b0 0x55ddc8d71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc8d6c950 0x55ddc8d71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5be4ff9640 0 client.18855 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 --2- 172.21.15.44:0/929228007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc8d61790 0x55ddc8d73620 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 >> 172.21.15.44:0/929228007 conn(0x55ddc8d5dd00 msgr2=0x55ddc8d77fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 shutdown_connections 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 1 -- 172.21.15.44:0/929228007 wait complete. 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c06823740 20 client.18855 trim_cache size 0 max 16384 2024-09-13T17:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:37:06.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5be80a1200 msgr2=0x7f5be80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5be80a1200 0x7f5be80a36c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009430 tx=0x7f5bf40095c0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 msgr2=0x7f5bfc0ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 0x7f5bfc0ff0d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c920 tx=0x7f5bec00cde0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 shutdown_connections 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5be80a1200 0x7f5be80a36c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bfc10a310 0x7f5bfc0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bfc101860 0x7f5bfc0ff0d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 --2- 172.21.15.44:0/4148822952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bfc100f40 0x7f5bfc0feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 >> 172.21.15.44:0/4148822952 conn(0x7f5bfc075c60 msgr2=0x7f5bfc0fb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 shutdown_connections 2024-09-13T17:37:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.154+0000 7f5c03a27640 1 -- 172.21.15.44:0/4148822952 wait complete. 2024-09-13T17:37:06.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 -- 172.21.15.44:0/4156994083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b0105090 msgr2=0x7fd8b010cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 --2- 172.21.15.44:0/4156994083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b0105090 0x7fd8b010cbe0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009cd0 tx=0x7fd8a802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 -- 172.21.15.44:0/4156994083 shutdown_connections 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 --2- 172.21.15.44:0/4156994083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b0105090 0x7fd8b010cbe0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 --2- 172.21.15.44:0/4156994083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b01008f0 0x7fd8b0104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 --2- 172.21.15.44:0/4156994083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b00fff40 0x7fd8b0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 -- 172.21.15.44:0/4156994083 >> 172.21.15.44:0/4156994083 conn(0x7fd8b00fb9f0 msgr2=0x7fd8b00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 -- 172.21.15.44:0/4156994083 shutdown_connections 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.280+0000 7fd8b5545640 1 -- 172.21.15.44:0/4156994083 wait complete. 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 Processor -- start 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 -- start start 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 0x7fd8b0196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b0105090 0x7fd8b01966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b019aaa0 0x7fd8b019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b019b490 con 0x7fd8b00fff40 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b019b600 con 0x7fd8b0105090 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8b5545640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b019b770 con 0x7fd8b019aaa0 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 0x7fd8b0196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 0x7fd8b0196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40390/0 (socket says 172.21.15.44:40390) 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2392257851 learned_addr learned my addr 172.21.15.44:0/2392257851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b019aaa0 0x7fd8b019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b0105090 0x7fd8b01966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2392257851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b019aaa0 msgr2=0x7fd8b019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b019aaa0 0x7fd8b019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2392257851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b0105090 msgr2=0x7fd8b01966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b0105090 0x7fd8b01966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2392257851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8009900 con 0x7fd8b00fff40 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.281+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b0105090 0x7fd8b01966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.282+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 0x7fd8b0196170 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000d950 tx=0x7fd8a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.282+0000 7fd8a7fff640 1 -- 172.21.15.44:0/2392257851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0009a30 con 0x7fd8b00fff40 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.282+0000 7fd8a7fff640 1 -- 172.21.15.44:0/2392257851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8a00044b0 con 0x7fd8b00fff40 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.282+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b01a2d50 con 0x7fd8b00fff40 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.282+0000 7fd8a7fff640 1 -- 172.21.15.44:0/2392257851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00047a0 con 0x7fd8b00fff40 2024-09-13T17:37:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.282+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b01a31b0 con 0x7fd8b00fff40 2024-09-13T17:37:06.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.284+0000 7fd8b8265740 1 Processor -- start 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.284+0000 7fd8a7fff640 1 -- 172.21.15.44:0/2392257851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd8a0004020 con 0x7fd8b00fff40 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.284+0000 7fd8b8265740 1 -- start start 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52fe790 0x55ddf52feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf5309410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf5309950 0x55ddf530bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8b8265740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5268cc0 con 0x55ddf52ff0b0 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8b8265740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5266450 con 0x55ddf52fe790 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8b8265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5261d40 con 0x55ddf5309950 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf5309410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf5309410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40404/0 (socket says 172.21.15.44:40404) 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2851531626 learned_addr learned my addr 172.21.15.44:0/2851531626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52fe790 0x55ddf52feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf5309950 0x55ddf530bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8a7fff640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd89c0a1200 0x7fd89c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2851531626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf5309950 msgr2=0x55ddf530bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf5309950 0x55ddf530bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2851531626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 msgr2=0x55ddf5309410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf5309410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2851531626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf5229620 con 0x55ddf52fe790 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.285+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf5309410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.286+0000 7fd8a7fff640 1 -- 172.21.15.44:0/2392257851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fd8a00ccaa0 con 0x7fd8b00fff40 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.286+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd89c0a1200 0x7fd89c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.286+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52fe790 0x55ddf52feb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005cc0 tx=0x7fd8a8031430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52bf27dbbca87512 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.286+0000 7fd8a57fa640 1 -- 172.21.15.44:0/2851531626 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8031970 con 0x55ddf52fe790 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.286+0000 7fd8a57fa640 1 -- 172.21.15.44:0/2851531626 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8a8040070 con 0x55ddf52fe790 2024-09-13T17:37:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.286+0000 7fd8a57fa640 1 -- 172.21.15.44:0/2851531626 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004400 con 0x55ddf52fe790 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- 172.21.15.44:0/2851531626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52fe790 msgr2=0x55ddf52feb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd89c0a1200 0x7fd89c0a36c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ddf52ff6d0 tx=0x7fd8a804b000 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52fe790 0x55ddf52feb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005cc0 tx=0x7fd8a8031430 comp rx=0 tx=0).stop 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- 172.21.15.44:0/2851531626 shutdown_connections 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf5309950 0x55ddf530bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf5309410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2851531626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52fe790 0x55ddf52feb70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- 172.21.15.44:0/2851531626 >> 172.21.15.44:0/2851531626 conn(0x55ddf52fad00 msgr2=0x55ddf52fb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- 172.21.15.44:0/2851531626 shutdown_connections 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- 172.21.15.44:0/2851531626 wait complete. 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- start start 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 Processor -- start 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf5308f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52ff0b0 0x55ddf5306280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 0x55ddf53067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5220180 con 0x55ddf5309950 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf4f3a430 con 0x55ddf52ff0b0 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf51e55e0 con 0x55ddf52fe790 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.287+0000 7fd8b8265740 10 client.0 _ll_register_callbacks cb 0x55ddf5251aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 0x55ddf53067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 0x55ddf53067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40420/0 (socket says 172.21.15.44:40420) 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2031075271 learned_addr learned my addr 172.21.15.44:0/2031075271 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf5308f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52ff0b0 0x55ddf5306280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 msgr2=0x55ddf5308f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf5308f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2031075271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52ff0b0 msgr2=0x55ddf5306280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52ff0b0 0x55ddf5306280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 -- 172.21.15.44:0/2031075271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b010d420 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52ff0b0 0x55ddf5306280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 0x55ddf53067c0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd8a802aed0 tx=0x7fd8a802b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d2db7e29463cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8967fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8967fc640 1 -- 172.21.15.44:0/2031075271 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80043c0 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddf528bb80 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8967fc640 1 -- 172.21.15.44:0/2031075271 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fd8a8042070 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.288+0000 7fd8967fc640 1 -- 172.21.15.44:0/2031075271 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a8003970 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 10 client.18867 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ddf52603c0 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8967fc640 1 -- 172.21.15.44:0/2031075271 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ddf528bb80 con 0x55ddf5309950 2024-09-13T17:37:06.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8967fc640 1 -- 172.21.15.44:0/2031075271 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd8a804f030 con 0x55ddf5309950 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 10 client.18867 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 10 client.18867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 20 client.18867 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf5318fa0 0x55ddf531b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 4 client.18867 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddf52541f0 con 0x55ddf5318fa0 2024-09-13T17:37:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.289+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf5318fa0 0x55ddf531b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8967fc640 10 client.18867 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf5318fa0 0x55ddf531b390 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e1748d75d031e60 server_cookie=ba0c29ef00fc381d in_seq=0 out_seq=0 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8967fc640 1 -- 172.21.15.44:0/2031075271 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ddf52541f0 con 0x55ddf5318fa0 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8967fc640 10 client.18867 handle_command_reply: tid=0 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 client.18867 shutdown 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 msgr2=0x55ddf53067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 0x55ddf53067c0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd8a802aed0 tx=0x7fd8a802b020 comp rx=0 tx=0).stop 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 shutdown_connections 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf5318fa0 0x55ddf531b390 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf5309950 0x55ddf53067c0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf52ff0b0 0x55ddf5306280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8967fc640 0 client.18867 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.290+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2031075271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf5308f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.291+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 >> 172.21.15.44:0/2031075271 conn(0x55ddf52fad00 msgr2=0x55ddf5300a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.291+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 shutdown_connections 2024-09-13T17:37:06.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.291+0000 7fd8b8265740 1 -- 172.21.15.44:0/2031075271 wait complete. 2024-09-13T17:37:06.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.291+0000 7fd8b8265740 20 client.18867 trim_cache size 0 max 16384 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 Processor -- start 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 -- start start 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 0x55ddf52fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52ff0b0 0x55ddf5310610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5310b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf52541f0 con 0x55ddf52fe790 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5318dd0 con 0x55ddf5309950 2024-09-13T17:37:06.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8b8265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5309540 con 0x55ddf52ff0b0 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 0x55ddf52fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8aeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 0x55ddf52fa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40428/0 (socket says 172.21.15.44:40428) 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2126580483 learned_addr learned my addr 172.21.15.44:0/2126580483 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.302+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52ff0b0 0x55ddf5310610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5310b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2126580483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52ff0b0 msgr2=0x55ddf5310610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52ff0b0 0x55ddf5310610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2126580483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 msgr2=0x55ddf5310b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5310b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8aeffd640 1 -- 172.21.15.44:0/2126580483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf5202770 con 0x55ddf52fe790 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 0x55ddf52fa6a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00c9ed0 tx=0x7fd8a0084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef266da3cb44e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8a5ffb640 1 -- 172.21.15.44:0/2126580483 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00d2070 con 0x55ddf52fe790 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8a5ffb640 1 -- 172.21.15.44:0/2126580483 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8a00995b0 con 0x55ddf52fe790 2024-09-13T17:37:06.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.303+0000 7fd8a5ffb640 1 -- 172.21.15.44:0/2126580483 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0099ee0 con 0x55ddf52fe790 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- 172.21.15.44:0/2126580483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 msgr2=0x55ddf52fa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 0x55ddf52fa6a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00c9ed0 tx=0x7fd8a0084040 comp rx=0 tx=0).stop 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- 172.21.15.44:0/2126580483 shutdown_connections 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52ff0b0 0x55ddf5310610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5310b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- 172.21.15.44:0/2126580483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52fe790 0x55ddf52fa6a0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- 172.21.15.44:0/2126580483 >> 172.21.15.44:0/2126580483 conn(0x55ddf52fad00 msgr2=0x55ddf531fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- 172.21.15.44:0/2126580483 shutdown_connections 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- 172.21.15.44:0/2126580483 wait complete. 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- start start 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 Processor -- start 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf530c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf530cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5372870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5372db0 con 0x55ddf52ff0b0 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf5372f20 con 0x55ddf5309950 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf53730e0 con 0x55ddf52fe790 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8b8265740 10 client.0 _ll_register_callbacks cb 0x7fd8a8009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:06.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.304+0000 7fd8ae7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf530cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf530cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40444/0 (socket says 172.21.15.44:40444) 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 -- 172.21.15.44:0/1115990853 learned_addr learned my addr 172.21.15.44:0/1115990853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf530c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5372870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 -- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 msgr2=0x55ddf530c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf530c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 -- 172.21.15.44:0/1115990853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 msgr2=0x55ddf5372870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5372870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 -- 172.21.15.44:0/1115990853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a0021070 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8af7fe640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5372870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8ae7fc640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf530cd20 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0197710 tx=0x7fd8980127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21f1ba888288fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8967fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8967fc640 1 -- 172.21.15.44:0/1115990853 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898017070 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8a8040070 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8967fc640 1 -- 172.21.15.44:0/1115990853 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fd898004020 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8967fc640 1 -- 172.21.15.44:0/1115990853 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8980041e0 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8b8265740 10 client.18873 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ddf5227e80 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.305+0000 7fd8967fc640 1 -- 172.21.15.44:0/1115990853 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8a8040070 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8967fc640 1 -- 172.21.15.44:0/1115990853 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd8980035f0 con 0x55ddf52ff0b0 2024-09-13T17:37:06.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8b8265740 10 client.18873 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:06.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8b8265740 10 client.18873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:37:06.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8b8265740 20 client.18873 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:06.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8b8265740 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf53749f0 0x55ddf5376e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8b8265740 4 client.18873 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:37:06.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddf4f554e0 con 0x55ddf53749f0 2024-09-13T17:37:06.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf53749f0 0x55ddf5376e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8aeffd640 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf53749f0 0x55ddf5376e10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91e4abebb01595c4 server_cookie=9c7eb11a114333e in_seq=0 out_seq=0 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.306+0000 7fd8967fc640 10 client.18873 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8967fc640 1 -- 172.21.15.44:0/1115990853 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddf4f554e0 con 0x55ddf53749f0 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8967fc640 10 client.18873 handle_command_reply: tid=0 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 client.18873 shutdown 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 msgr2=0x55ddf530cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf530cd20 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0197710 tx=0x7fd8980127c0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 shutdown_connections 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x55ddf53749f0 0x55ddf5376e10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf5309950 0x55ddf5372870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddf52ff0b0 0x55ddf530cd20 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8967fc640 0 client.18873 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:06.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 --2- 172.21.15.44:0/1115990853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddf52fe790 0x55ddf530c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 >> 172.21.15.44:0/1115990853 conn(0x55ddf52fad00 msgr2=0x55ddf5304ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 shutdown_connections 2024-09-13T17:37:06.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 1 -- 172.21.15.44:0/1115990853 wait complete. 2024-09-13T17:37:06.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b8265740 20 client.18873 trim_cache size 0 max 16384 2024-09-13T17:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd89c0a1200 msgr2=0x7fd89c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd89c0a1200 0x7fd89c0a36c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ddf52ff6d0 tx=0x7fd8a804b000 comp rx=0 tx=0).stop 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 msgr2=0x7fd8b0196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 0x7fd8b0196170 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000d950 tx=0x7fd8a000de10 comp rx=0 tx=0).stop 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 shutdown_connections 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd89c0a1200 0x7fd89c0a36c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b019aaa0 0x7fd8b019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b0105090 0x7fd8b01966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.307+0000 7fd8b5545640 1 --2- 172.21.15.44:0/2392257851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8b00fff40 0x7fd8b0196170 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.308+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 >> 172.21.15.44:0/2392257851 conn(0x7fd8b00fb9f0 msgr2=0x7fd8b00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.308+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 shutdown_connections 2024-09-13T17:37:06.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.308+0000 7fd8b5545640 1 -- 172.21.15.44:0/2392257851 wait complete. 2024-09-13T17:37:06.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 -- 172.21.15.44:0/2147424812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b810a310 msgr2=0x7fc2b810c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 --2- 172.21.15.44:0/2147424812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b810a310 0x7fc2b810c790 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009ca0 tx=0x7fc2b002a950 comp rx=0 tx=0).stop 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 -- 172.21.15.44:0/2147424812 shutdown_connections 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 --2- 172.21.15.44:0/2147424812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2b8101860 0x7fc2b8109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 --2- 172.21.15.44:0/2147424812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b810a310 0x7fc2b810c790 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 --2- 172.21.15.44:0/2147424812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 0x7fc2b8101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 -- 172.21.15.44:0/2147424812 >> 172.21.15.44:0/2147424812 conn(0x7fc2b8075c60 msgr2=0x7fc2b80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 -- 172.21.15.44:0/2147424812 shutdown_connections 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.424+0000 7fc2be473640 1 -- 172.21.15.44:0/2147424812 wait complete. 2024-09-13T17:37:06.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 Processor -- start 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 -- start start 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 0x7fc2b81961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 0x7fc2b81966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2b810a310 0x7fc2b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b819b0d0 con 0x7fc2b8101860 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b819b240 con 0x7fc2b8100f40 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.425+0000 7fc2be473640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b819b400 con 0x7fc2b810a310 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 0x7fc2b81966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 0x7fc2b81966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40470/0 (socket says 172.21.15.44:40470) 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 -- 172.21.15.44:0/2817314898 learned_addr learned my addr 172.21.15.44:0/2817314898 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2bca10640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2b810a310 0x7fc2b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 0x7fc2b81961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 -- 172.21.15.44:0/2817314898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2b810a310 msgr2=0x7fc2b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2b810a310 0x7fc2b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 -- 172.21.15.44:0/2817314898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 msgr2=0x7fc2b81961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 0x7fc2b81961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 -- 172.21.15.44:0/2817314898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0009900 con 0x7fc2b8101860 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 0x7fc2b81961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 0x7fc2b81966e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00d770 tx=0x7fc2ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b57fa640 1 -- 172.21.15.44:0/2817314898 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac009c30 con 0x7fc2b8101860 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b819b6e0 con 0x7fc2b8101860 2024-09-13T17:37:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b57fa640 1 -- 172.21.15.44:0/2817314898 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2ac00de80 con 0x7fc2b8101860 2024-09-13T17:37:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2b57fa640 1 -- 172.21.15.44:0/2817314898 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004d40 con 0x7fc2b8101860 2024-09-13T17:37:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.426+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81a2f60 con 0x7fc2b8101860 2024-09-13T17:37:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b57fa640 1 -- 172.21.15.44:0/2817314898 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc2ac004ea0 con 0x7fc2b8101860 2024-09-13T17:37:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 Processor -- start 2024-09-13T17:37:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 -- start start 2024-09-13T17:37:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc128da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 0x55ddc129c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc129f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc11f7cc0 con 0x55ddc128df90 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc11f5450 con 0x55ddc129cd60 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2c1266740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc11f0d40 con 0x55ddc128d670 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 0x55ddc129c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 0x55ddc129c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40476/0 (socket says 172.21.15.44:40476) 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b7fff640 1 -- 172.21.15.44:0/861107065 learned_addr learned my addr 172.21.15.44:0/861107065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2bca10640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc128da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc129f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b7fff640 1 -- 172.21.15.44:0/861107065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 msgr2=0x55ddc128da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc128da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.429+0000 7fc2b7fff640 1 -- 172.21.15.44:0/861107065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 msgr2=0x55ddc129f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc129f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2b7fff640 1 -- 172.21.15.44:0/861107065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc11b8620 con 0x55ddc128df90 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2bca10640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc128da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc129f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2b57fa640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc2a00a1200 0x7fc2a00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 0x55ddc129c820 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009c60 tx=0x7fc2a8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3442a24934790b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2bca10640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc2a00a1200 0x7fc2a00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc2b57fa640 1 -- 172.21.15.44:0/2817314898 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fc2ac0ccba0 con 0x7fc2b8101860 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc29e7fc640 1 -- 172.21.15.44:0/861107065 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8004030 con 0x55ddc128df90 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc29e7fc640 1 -- 172.21.15.44:0/861107065 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a80035f0 con 0x55ddc128df90 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.430+0000 7fc29e7fc640 1 -- 172.21.15.44:0/861107065 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80126b0 con 0x55ddc128df90 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2bca10640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc2a00a1200 0x7fc2a00a36c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0006010 tx=0x7fc2b0009890 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 -- 172.21.15.44:0/861107065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 msgr2=0x55ddc129c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 0x55ddc129c820 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009c60 tx=0x7fc2a8010740 comp rx=0 tx=0).stop 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 -- 172.21.15.44:0/861107065 shutdown_connections 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc129f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128df90 0x55ddc129c820 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 --2- 172.21.15.44:0/861107065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc128da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 -- 172.21.15.44:0/861107065 >> 172.21.15.44:0/861107065 conn(0x55ddc1289d00 msgr2=0x55ddc128a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 -- 172.21.15.44:0/861107065 shutdown_connections 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 -- 172.21.15.44:0/861107065 wait complete. 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.431+0000 7fc2c1266740 1 -- start start 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 Processor -- start 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128d670 0x55ddc129c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc1299c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc11af180 con 0x55ddc129cd60 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc0ec9430 con 0x55ddc128d670 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc11745e0 con 0x55ddc128df90 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2c1266740 10 client.0 _ll_register_callbacks cb 0x55ddc11e0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc1299c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc1299c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40478/0 (socket says 172.21.15.44:40478) 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128d670 0x55ddc129c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 -- 172.21.15.44:0/3158680513 learned_addr learned my addr 172.21.15.44:0/3158680513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 -- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 msgr2=0x55ddc12996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 -- 172.21.15.44:0/3158680513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128d670 msgr2=0x55ddc129c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128d670 0x55ddc129c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.432+0000 7fc2bca10640 1 -- 172.21.15.44:0/3158680513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a8009890 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2bca10640 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc1299c10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005f50 tx=0x7fc2b00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5274cced9c5f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2977fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2977fe640 1 -- 172.21.15.44:0/3158680513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0030850 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2977fe640 1 -- 172.21.15.44:0/3158680513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fc2b00309b0 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddc121ab80 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2977fe640 1 -- 172.21.15.44:0/3158680513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b0003fb0 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 10 client.18885 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ddc11ef3c0 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2977fe640 1 -- 172.21.15.44:0/3158680513 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ddc121ab80 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2977fe640 1 -- 172.21.15.44:0/3158680513 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2b0044070 con 0x55ddc129cd60 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 10 client.18885 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 10 client.18885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 20 client.18885 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:06.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc12ac7f0 0x55ddc12aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 4 client.18885 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.433+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddc11e31f0 con 0x55ddc12ac7f0 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.434+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc12ac7f0 0x55ddc12aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.434+0000 7fc2977fe640 10 client.18885 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.434+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc12ac7f0 0x55ddc12aec10 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67da141b1d555393 server_cookie=dd765b9b951ac2c0 in_seq=0 out_seq=0 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2977fe640 1 -- 172.21.15.44:0/3158680513 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ddc11e31f0 con 0x55ddc12ac7f0 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2977fe640 10 client.18885 handle_command_reply: tid=0 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 client.18885 shutdown 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 msgr2=0x55ddc1299c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc1299c10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005f50 tx=0x7fc2b00095c0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 shutdown_connections 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc12ac7f0 0x55ddc12aec10 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2977fe640 0 client.18885 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc1299c10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3158680513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128d670 0x55ddc129c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 >> 172.21.15.44:0/3158680513 conn(0x55ddc1289d00 msgr2=0x55ddc128f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 shutdown_connections 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 1 -- 172.21.15.44:0/3158680513 wait complete. 2024-09-13T17:37:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.435+0000 7fc2c1266740 20 client.18885 trim_cache size 0 max 16384 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 Processor -- start 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 -- start start 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 0x55ddc12896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc12a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc12901a0 con 0x55ddc128d670 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc12ac580 con 0x55ddc129cd60 2024-09-13T17:37:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.446+0000 7fc2c1266740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc12a8d20 con 0x55ddc128df90 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 0x55ddc12896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 0x55ddc12896a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40488/0 (socket says 172.21.15.44:40488) 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 -- 172.21.15.44:0/960570698 learned_addr learned my addr 172.21.15.44:0/960570698 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2bca10640 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc12a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 -- 172.21.15.44:0/960570698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 msgr2=0x55ddc12a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 -- 172.21.15.44:0/960570698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 msgr2=0x55ddc12a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc12a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 -- 172.21.15.44:0/960570698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc1191770 con 0x55ddc128d670 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2bca10640 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc12a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 0x55ddc12896a0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8010c40 tx=0x7fc2a8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ef1a2c5f82b9cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc29effd640 1 -- 172.21.15.44:0/960570698 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8021070 con 0x55ddc128d670 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc29effd640 1 -- 172.21.15.44:0/960570698 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a8022040 con 0x55ddc128d670 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.447+0000 7fc29effd640 1 -- 172.21.15.44:0/960570698 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8029330 con 0x55ddc128d670 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 -- 172.21.15.44:0/960570698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 msgr2=0x55ddc12896a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 0x55ddc12896a0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8010c40 tx=0x7fc2a8010fd0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 -- 172.21.15.44:0/960570698 shutdown_connections 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128df90 0x55ddc12a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc129cd60 0x55ddc12a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 --2- 172.21.15.44:0/960570698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc128d670 0x55ddc12896a0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 -- 172.21.15.44:0/960570698 >> 172.21.15.44:0/960570698 conn(0x55ddc1289d00 msgr2=0x55ddc1297f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 -- 172.21.15.44:0/960570698 shutdown_connections 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 -- 172.21.15.44:0/960570698 wait complete. 2024-09-13T17:37:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 -- start start 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.448+0000 7fc2c1266740 1 Processor -- start 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc12faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128df90 0x55ddc12fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc12a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc12a39b0 con 0x55ddc129cd60 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc12a3b20 con 0x55ddc128df90 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc12a3ce0 con 0x55ddc128d670 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2c1266740 10 client.0 _ll_register_callbacks cb 0x7fc2a80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:06.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc12a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc12a3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40496/0 (socket says 172.21.15.44:40496) 2024-09-13T17:37:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 -- 172.21.15.44:0/3144261178 learned_addr learned my addr 172.21.15.44:0/3144261178 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2b77fe640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128df90 0x55ddc12fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc12faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 -- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 msgr2=0x55ddc12faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc12faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 -- 172.21.15.44:0/3144261178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128df90 msgr2=0x55ddc12fb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128df90 0x55ddc12fb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 -- 172.21.15.44:0/3144261178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddc11b8620 con 0x55ddc129cd60 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2bca10640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc12a3470 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0052680 tx=0x7fc2b0030880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e08b03d00b485de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2977fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.449+0000 7fc2977fe640 1 -- 172.21.15.44:0/3144261178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0047830 con 0x55ddc129cd60 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2977fe640 1 -- 172.21.15.44:0/3144261178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fc2b0030a70 con 0x55ddc129cd60 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2a80035f0 con 0x55ddc129cd60 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2977fe640 1 -- 172.21.15.44:0/3144261178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b0047990 con 0x55ddc129cd60 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 10 client.18891 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ddc11b6e80 con 0x55ddc129cd60 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2977fe640 1 -- 172.21.15.44:0/3144261178 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a80035f0 con 0x55ddc129cd60 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2977fe640 1 -- 172.21.15.44:0/3144261178 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2b0044040 con 0x55ddc129cd60 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 10 client.18891 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 20 client.18891 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc1303c50 0x55ddc1306070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 4 client.18891 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddc0ee44e0 con 0x55ddc1303c50 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.450+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc1303c50 0x55ddc1306070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2b7fff640 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc1303c50 0x55ddc1306070 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf510211c17ca0a3 server_cookie=46516675faf74ed6 in_seq=0 out_seq=0 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2977fe640 10 client.18891 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2977fe640 1 -- 172.21.15.44:0/3144261178 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddc0ee44e0 con 0x55ddc1303c50 2024-09-13T17:37:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2977fe640 10 client.18891 handle_command_reply: tid=0 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 client.18891 shutdown 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 msgr2=0x55ddc12a3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc12a3470 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0052680 tx=0x7fc2b0030880 comp rx=0 tx=0).stop 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 shutdown_connections 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55ddc1303c50 0x55ddc1306070 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddc129cd60 0x55ddc12a3470 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddc128df90 0x55ddc12fb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2977fe640 0 client.18891 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 --2- 172.21.15.44:0/3144261178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddc128d670 0x55ddc12faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 >> 172.21.15.44:0/3144261178 conn(0x55ddc1289d00 msgr2=0x55ddc1297eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 shutdown_connections 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.451+0000 7fc2c1266740 1 -- 172.21.15.44:0/3144261178 wait complete. 2024-09-13T17:37:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2c1266740 20 client.18891 trim_cache size 0 max 16384 2024-09-13T17:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:37:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc2a00a1200 msgr2=0x7fc2a00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc2a00a1200 0x7fc2a00a36c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0006010 tx=0x7fc2b0009890 comp rx=0 tx=0).stop 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 msgr2=0x7fc2b81966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 0x7fc2b81966e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00d770 tx=0x7fc2ac00dc30 comp rx=0 tx=0).stop 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 shutdown_connections 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fc2a00a1200 0x7fc2a00a36c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2b810a310 0x7fc2b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b8101860 0x7fc2b81966e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 --2- 172.21.15.44:0/2817314898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b8100f40 0x7fc2b81961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 >> 172.21.15.44:0/2817314898 conn(0x7fc2b8075c60 msgr2=0x7fc2b80fb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 shutdown_connections 2024-09-13T17:37:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:06.452+0000 7fc2be473640 1 -- 172.21.15.44:0/2817314898 wait complete. 2024-09-13T17:37:06.467 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:06 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1103513640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:37:06.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:06 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1103513640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:37:06.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:06 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1103513640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:37:07.469 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:37:07.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:37:07.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 -- 172.21.15.44:0/1580359702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b781050a0 msgr2=0x7f9b7810cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/1580359702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b781050a0 0x7f9b7810cbf0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009cd0 tx=0x7f9b7002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 -- 172.21.15.44:0/1580359702 shutdown_connections 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/1580359702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b781050a0 0x7f9b7810cbf0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/1580359702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b78100900 0x7f9b78104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/1580359702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b780fff50 0x7f9b78100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 -- 172.21.15.44:0/1580359702 >> 172.21.15.44:0/1580359702 conn(0x7f9b780fb9e0 msgr2=0x7f9b780fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 -- 172.21.15.44:0/1580359702 shutdown_connections 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.584+0000 7f9b7f15e640 1 -- 172.21.15.44:0/1580359702 wait complete. 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 Processor -- start 2024-09-13T17:37:07.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 -- start start 2024-09-13T17:37:07.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 0x7f9b781961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b781050a0 0x7f9b78196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b7819aaf0 0x7f9b7819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7819b4e0 con 0x7f9b780fff50 2024-09-13T17:37:07.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7819b650 con 0x7f9b781050a0 2024-09-13T17:37:07.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7f15e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7819b7c0 con 0x7f9b7819aaf0 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 0x7f9b781961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 0x7f9b781961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40500/0 (socket says 172.21.15.44:40500) 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 -- 172.21.15.44:0/505264657 learned_addr learned my addr 172.21.15.44:0/505264657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b77fff640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b781050a0 0x7f9b78196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b7819aaf0 0x7f9b7819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 -- 172.21.15.44:0/505264657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b7819aaf0 msgr2=0x7f9b7819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b7819aaf0 0x7f9b7819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 -- 172.21.15.44:0/505264657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b781050a0 msgr2=0x7f9b78196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b781050a0 0x7f9b78196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 -- 172.21.15.44:0/505264657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70009900 con 0x7f9b780fff50 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b7819aaf0 0x7f9b7819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:07.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.585+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 0x7f9b781961c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c970 tx=0x7f9b6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.586+0000 7f9b75ffb640 1 -- 172.21.15.44:0/505264657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68007bb0 con 0x7f9b780fff50 2024-09-13T17:37:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.586+0000 7f9b75ffb640 1 -- 172.21.15.44:0/505264657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b68007d10 con 0x7f9b780fff50 2024-09-13T17:37:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.586+0000 7f9b75ffb640 1 -- 172.21.15.44:0/505264657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68004d70 con 0x7f9b780fff50 2024-09-13T17:37:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.586+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b781a2da0 con 0x7f9b780fff50 2024-09-13T17:37:07.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.586+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b781a31a0 con 0x7f9b780fff50 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.588+0000 7f9b75ffb640 1 -- 172.21.15.44:0/505264657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9b68004f50 con 0x7f9b780fff50 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.588+0000 7f9b81e80740 1 Processor -- start 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.588+0000 7f9b81e80740 1 -- start start 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b75ffb640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9b640a1280 0x7f9b640a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dbf2680 0x55e68dbf2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dc01850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 0x55e68dc04180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b81e80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68db5dc60 con 0x55e68dc01d90 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9b640a1280 0x7f9b640a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b75ffb640 1 -- 172.21.15.44:0/505264657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f9b680ccdf0 con 0x7f9b780fff50 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b81e80740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68db5b3f0 con 0x55e68dbf2fa0 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b81e80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68db56ce0 con 0x55e68dbf2680 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7d6fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 0x55e68dc04180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7d6fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 0x55e68dc04180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40502/0 (socket says 172.21.15.44:40502) 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7d6fb640 1 -- 172.21.15.44:0/7536309 learned_addr learned my addr 172.21.15.44:0/7536309 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7cefa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dbf2680 0x55e68dbf2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b77fff640 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dc01850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7cefa640 1 -- 172.21.15.44:0/7536309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 msgr2=0x55e68dc01850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dc01850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7cefa640 1 -- 172.21.15.44:0/7536309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 msgr2=0x55e68dc04180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 0x55e68dc04180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7cefa640 1 -- 172.21.15.44:0/7536309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68db12970 con 0x55e68dbf2680 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 0x55e68dc04180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b77fff640 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dc01850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.589+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9b640a1280 0x7f9b640a3740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9b70031710 tx=0x7f9b70031680 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dbf2680 0x55e68dbf2a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b680ca110 tx=0x7f9b68004780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420ca6cf9105b0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b6effd640 1 -- 172.21.15.44:0/7536309 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6809d900 con 0x55e68dbf2680 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b6effd640 1 -- 172.21.15.44:0/7536309 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b680a1f50 con 0x55e68dbf2680 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b6effd640 1 -- 172.21.15.44:0/7536309 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b680a1cc0 con 0x55e68dbf2680 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b81e80740 1 -- 172.21.15.44:0/7536309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dbf2680 msgr2=0x55e68dbf2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b81e80740 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dbf2680 0x55e68dbf2a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b680ca110 tx=0x7f9b68004780 comp rx=0 tx=0).stop 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b81e80740 1 -- 172.21.15.44:0/7536309 shutdown_connections 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.590+0000 7f9b81e80740 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dc01d90 0x55e68dc04180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dc01850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 --2- 172.21.15.44:0/7536309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dbf2680 0x55e68dbf2a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 -- 172.21.15.44:0/7536309 >> 172.21.15.44:0/7536309 conn(0x55e68dbeed50 msgr2=0x55e68dbef130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 -- 172.21.15.44:0/7536309 shutdown_connections 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 -- 172.21.15.44:0/7536309 wait complete. 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 -- start start 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 Processor -- start 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.591+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b81e80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68db14f80 con 0x55e68dbf2fa0 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b81e80740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68db14c20 con 0x55e68dbf2680 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b81e80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dae1ab0 con 0x55e68dc01d90 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b81e80740 10 client.0 _ll_register_callbacks cb 0x55e68db50350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40504/0 (socket says 172.21.15.44:40504) 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 -- 172.21.15.44:0/1851943266 learned_addr learned my addr 172.21.15.44:0/1851943266 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 -- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 msgr2=0x55e68dbff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 -- 172.21.15.44:0/1851943266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 msgr2=0x55e68dc017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 -- 172.21.15.44:0/1851943266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b781a31a0 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b77fff640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfea40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9b60012700 tx=0x7f9b60012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489d3788f0de79af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b57fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b57fff640 1 -- 172.21.15.44:0/1851943266 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60017070 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b57fff640 1 -- 172.21.15.44:0/1851943266 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f9b60004050 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.592+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e68d84a590 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b57fff640 1 -- 172.21.15.44:0/1851943266 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b60004210 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 10 client.18903 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e68db55360 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b57fff640 1 -- 172.21.15.44:0/1851943266 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e68d84a590 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b57fff640 1 -- 172.21.15.44:0/1851943266 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b60003960 con 0x55e68dbf2fa0 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 10 client.18903 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:07.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 20 client.18903 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc11ba0 0x55e68dc13fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 4 client.18903 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e68d85b860 con 0x55e68dc11ba0 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.593+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc11ba0 0x55e68dc13fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.594+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc11ba0 0x55e68dc13fc0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be78edf4f0bbf863 server_cookie=40c26fe53777d038 in_seq=0 out_seq=0 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.594+0000 7f9b57fff640 10 client.18903 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.594+0000 7f9b57fff640 1 -- 172.21.15.44:0/1851943266 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e68d85b860 con 0x55e68dc11ba0 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.594+0000 7f9b57fff640 10 client.18903 handle_command_reply: tid=0 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.594+0000 7f9b81e80740 1 client.18903 shutdown 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 msgr2=0x55e68dbfea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfea40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9b60012700 tx=0x7f9b60012bc0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 shutdown_connections 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc11ba0 0x55e68dc13fc0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfea40 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b57fff640 0 client.18903 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 --2- 172.21.15.44:0/1851943266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 >> 172.21.15.44:0/1851943266 conn(0x55e68dbeed50 msgr2=0x55e68dbee160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 shutdown_connections 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 1 -- 172.21.15.44:0/1851943266 wait complete. 2024-09-13T17:37:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.595+0000 7f9b81e80740 20 client.18903 trim_cache size 0 max 16384 2024-09-13T17:37:07.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:07 smithi044 ceph-mon[31175]: pgmap v324: 129 pgs: 129 active+clean; 4.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-13T17:37:07.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 Processor -- start 2024-09-13T17:37:07.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 -- start start 2024-09-13T17:37:07.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 0x55e68dbfd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbfe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dbefcb0 con 0x55e68dbf2680 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dbf1ff0 con 0x55e68dbf2fa0 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.609+0000 7f9b81e80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dbfddb0 con 0x55e68dc01d90 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 0x55e68dbfd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 0x55e68dbfd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40508/0 (socket says 172.21.15.44:40508) 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 -- 172.21.15.44:0/4039832456 learned_addr learned my addr 172.21.15.44:0/4039832456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 -- 172.21.15.44:0/4039832456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 msgr2=0x55e68dbfe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbfe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 -- 172.21.15.44:0/4039832456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 msgr2=0x55e68dbfd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 -- 172.21.15.44:0/4039832456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68daf71c0 con 0x55e68dbf2680 2024-09-13T17:37:07.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 0x55e68dbfd330 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9b680058c0 tx=0x7f9b6800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa9bd1130d41cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b6f7fe640 1 -- 172.21.15.44:0/4039832456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800a4b0 con 0x55e68dbf2680 2024-09-13T17:37:07.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b6f7fe640 1 -- 172.21.15.44:0/4039832456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b680961e0 con 0x55e68dbf2680 2024-09-13T17:37:07.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.610+0000 7f9b6f7fe640 1 -- 172.21.15.44:0/4039832456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b680b2460 con 0x55e68dbf2680 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 -- 172.21.15.44:0/4039832456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 msgr2=0x55e68dbfd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 0x55e68dbfd330 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9b680058c0 tx=0x7f9b6800cf60 comp rx=0 tx=0).stop 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 -- 172.21.15.44:0/4039832456 shutdown_connections 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dbfe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2fa0 0x55e68dbfd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 --2- 172.21.15.44:0/4039832456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2680 0x55e68dbfd330 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 -- 172.21.15.44:0/4039832456 >> 172.21.15.44:0/4039832456 conn(0x55e68dbeed50 msgr2=0x55e68dbf9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 -- 172.21.15.44:0/4039832456 shutdown_connections 2024-09-13T17:37:07.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 -- 172.21.15.44:0/4039832456 wait complete. 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 -- start start 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.611+0000 7f9b81e80740 1 Processor -- start 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc08ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dc09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dc06aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dc06fe0 con 0x55e68dbf2fa0 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dc07150 con 0x55e68dbf2680 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68dc07310 con 0x55e68dc01d90 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 10 client.0 _ll_register_callbacks cb 0x7f9b60009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dc09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dc09000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40510/0 (socket says 172.21.15.44:40510) 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 -- 172.21.15.44:0/516674134 learned_addr learned my addr 172.21.15.44:0/516674134 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dc06aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc08ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 -- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 msgr2=0x55e68dc06aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dc06aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 -- 172.21.15.44:0/516674134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 msgr2=0x55e68dc08ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc08ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 -- 172.21.15.44:0/516674134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b680b1040 con 0x55e68dbf2fa0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b7d6fb640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dc06aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b77fff640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dc09000 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x55e68dc000a0 tx=0x7f9b600098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65938f295eeb0d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b57fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b57fff640 1 -- 172.21.15.44:0/516674134 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60017070 con 0x55e68dbf2fa0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b57fff640 1 -- 172.21.15.44:0/516674134 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f9b60003940 con 0x55e68dbf2fa0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.612+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b680a1f50 con 0x55e68dbf2fa0 2024-09-13T17:37:07.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b57fff640 1 -- 172.21.15.44:0/516674134 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b6000f920 con 0x55e68dbf2fa0 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 10 client.18909 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e68db1c990 con 0x55e68dbf2fa0 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b57fff640 1 -- 172.21.15.44:0/516674134 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b60033080 con 0x55e68dbf2fa0 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b57fff640 1 -- 172.21.15.44:0/516674134 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b60010520 con 0x55e68dbf2fa0 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 10 client.18909 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 20 client.18909 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc6a0a0 0x55e68dc6c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 4 client.18909 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e68d8aeeb0 con 0x55e68dc6a0a0 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.613+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc6a0a0 0x55e68dc6c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.614+0000 7f9b57fff640 10 client.18909 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.614+0000 7f9b7cefa640 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc6a0a0 0x55e68dc6c4c0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18151ef644f3711c server_cookie=170fd02ac21886bd in_seq=0 out_seq=0 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b57fff640 1 -- 172.21.15.44:0/516674134 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e68d8aeeb0 con 0x55e68dc6a0a0 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b57fff640 10 client.18909 handle_command_reply: tid=0 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 client.18909 shutdown 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 msgr2=0x55e68dc09000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dc09000 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x55e68dc000a0 tx=0x7f9b600098c0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 shutdown_connections 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e68dc6a0a0 0x55e68dc6c4c0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e68dc01d90 0x55e68dc06aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e68dbf2fa0 0x55e68dc09000 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 --2- 172.21.15.44:0/516674134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e68dbf2680 0x55e68dc08ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 >> 172.21.15.44:0/516674134 conn(0x55e68dbeed50 msgr2=0x55e68dbf9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b57fff640 0 client.18909 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 shutdown_connections 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 1 -- 172.21.15.44:0/516674134 wait complete. 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.615+0000 7f9b81e80740 20 client.18909 trim_cache size 0 max 16384 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "81e9c169-979e-45f3-ba70-1ebbd8d2c917", 2024-09-13T17:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:37:07.624 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:37:07.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9b640a1280 msgr2=0x7f9b640a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9b640a1280 0x7f9b640a3740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9b70031710 tx=0x7f9b70031680 comp rx=0 tx=0).stop 2024-09-13T17:37:07.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 msgr2=0x7f9b781961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 0x7f9b781961c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c970 tx=0x7f9b6800ce30 comp rx=0 tx=0).stop 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 shutdown_connections 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9b640a1280 0x7f9b640a3740 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b7819aaf0 0x7f9b7819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b781050a0 0x7f9b78196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 --2- 172.21.15.44:0/505264657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b780fff50 0x7f9b781961c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 >> 172.21.15.44:0/505264657 conn(0x7f9b780fb9e0 msgr2=0x7f9b780fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 shutdown_connections 2024-09-13T17:37:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.616+0000 7f9b7f15e640 1 -- 172.21.15.44:0/505264657 wait complete. 2024-09-13T17:37:07.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:37:07.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:07 smithi070 ceph-mon[34502]: pgmap v324: 129 pgs: 129 active+clean; 4.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-13T17:37:07.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 -- 172.21.15.44:0/1791195653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80ff830 msgr2=0x7fe6d8103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 --2- 172.21.15.44:0/1791195653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80ff830 0x7fe6d8103870 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009930 tx=0x7fe6cc02a420 comp rx=0 tx=0).stop 2024-09-13T17:37:07.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 -- 172.21.15.44:0/1791195653 shutdown_connections 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 --2- 172.21.15.44:0/1791195653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 0x7fe6d810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 --2- 172.21.15.44:0/1791195653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80ff830 0x7fe6d8103870 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 --2- 172.21.15.44:0/1791195653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80fee80 0x7fe6d80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 -- 172.21.15.44:0/1791195653 >> 172.21.15.44:0/1791195653 conn(0x7fe6d80fa930 msgr2=0x7fe6d80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 -- 172.21.15.44:0/1791195653 shutdown_connections 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 -- 172.21.15.44:0/1791195653 wait complete. 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 Processor -- start 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.732+0000 7fe6dd901640 1 -- start start 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dd901640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 0x7fe6d810c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dd901640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 0x7fe6d810c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dd901640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d810ceb0 0x7fe6d81a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dd901640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81a48d0 con 0x7fe6d80fee80 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dd901640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81a4a40 con 0x7fe6d8103fd0 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dd901640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81a4c00 con 0x7fe6d810ceb0 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 0x7fe6d810c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 0x7fe6d810c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40532/0 (socket says 172.21.15.44:40532) 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 -- 172.21.15.44:0/355039401 learned_addr learned my addr 172.21.15.44:0/355039401 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 0x7fe6d810c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d810ceb0 0x7fe6d81a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 -- 172.21.15.44:0/355039401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d810ceb0 msgr2=0x7fe6d81a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d810ceb0 0x7fe6d81a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 -- 172.21.15.44:0/355039401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 msgr2=0x7fe6d810c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 0x7fe6d810c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 -- 172.21.15.44:0/355039401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc009560 con 0x7fe6d80fee80 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 0x7fe6d810c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 0x7fe6d810c430 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800c920 tx=0x7fe6c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.733+0000 7fe6dc8ff640 1 -- 172.21.15.44:0/355039401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8007c70 con 0x7fe6d80fee80 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.734+0000 7fe6dc8ff640 1 -- 172.21.15.44:0/355039401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6c8007dd0 con 0x7fe6d80fee80 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.734+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d81a4ee0 con 0x7fe6d80fee80 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.734+0000 7fe6dc8ff640 1 -- 172.21.15.44:0/355039401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8018730 con 0x7fe6d80fee80 2024-09-13T17:37:07.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.734+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d81a5300 con 0x7fe6d80fee80 2024-09-13T17:37:07.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 Processor -- start 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6dc8ff640 1 -- 172.21.15.44:0/355039401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe6c800f450 con 0x7fe6d80fee80 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 -- start start 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d55489d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d5548aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d554807f20 con 0x55d55489ce10 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5548056b0 con 0x55d55489d730 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6e0620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d554800fa0 con 0x55d5548a7fd0 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d55489d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d55489d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40538/0 (socket says 172.21.15.44:40538) 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.736+0000 7fe6d77fe640 1 -- 172.21.15.44:0/330099790 learned_addr learned my addr 172.21.15.44:0/330099790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d5548aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d77fe640 1 -- 172.21.15.44:0/330099790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 msgr2=0x55d5548aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d5548aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d77fe640 1 -- 172.21.15.44:0/330099790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 msgr2=0x55d5548a7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d77fe640 1 -- 172.21.15.44:0/330099790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5547bb840 con 0x55d55489ce10 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d5548aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d55489d1f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009e60 tx=0x7fe6d0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4183f435761d001e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6dc8ff640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6c00a1230 0x7fe6c00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6bdffb640 1 -- 172.21.15.44:0/330099790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004d20 con 0x55d55489ce10 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6bdffb640 1 -- 172.21.15.44:0/330099790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6d0004e80 con 0x55d55489ce10 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6c00a1230 0x7fe6c00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6bdffb640 1 -- 172.21.15.44:0/330099790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0012740 con 0x55d55489ce10 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.737+0000 7fe6dc8ff640 1 -- 172.21.15.44:0/355039401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fe6c80cd1e0 con 0x7fe6d80fee80 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 -- 172.21.15.44:0/330099790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 msgr2=0x55d55489d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6d6ffd640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6c00a1230 0x7fe6c00a36f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80ca610 tx=0x7fe6c800fbd0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d55489d1f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009e60 tx=0x7fe6d0010740 comp rx=0 tx=0).stop 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 -- 172.21.15.44:0/330099790 shutdown_connections 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d5548aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 --2- 172.21.15.44:0/330099790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d55489d1f0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 -- 172.21.15.44:0/330099790 >> 172.21.15.44:0/330099790 conn(0x55d554899380 msgr2=0x55d554899760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 -- 172.21.15.44:0/330099790 shutdown_connections 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 -- 172.21.15.44:0/330099790 wait complete. 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 -- start start 2024-09-13T17:37:07.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.738+0000 7fe6e0620740 1 Processor -- start 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489ce10 0x55d5548a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5547a22a0 con 0x55d5548a7fd0 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5544d9430 con 0x55d55489d730 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d554784680 con 0x55d55489ce10 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 10 client.0 _ll_register_callbacks cb 0x55d5547fa610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40540/0 (socket says 172.21.15.44:40540) 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 -- 172.21.15.44:0/3009561321 learned_addr learned my addr 172.21.15.44:0/3009561321 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489ce10 0x55d5548a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 -- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489ce10 msgr2=0x55d5548a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489ce10 0x55d5548a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 -- 172.21.15.44:0/3009561321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 msgr2=0x55d5548a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 -- 172.21.15.44:0/3009561321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d0009ac0 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a5110 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0011010 tx=0x7fe6d0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b0008bb3292e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/3009561321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00129e0 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/3009561321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fe6d0012b40 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.739+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5545057e0 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/3009561321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d0012d00 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 10 client.18921 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d5547ff620 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/3009561321 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d5545057e0 con 0x55d5548a7fd0 2024-09-13T17:37:07.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/3009561321 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6d001a710 con 0x55d5548a7fd0 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 10 client.18921 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 10 client.18921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 20 client.18921 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5548b7be0 0x55d5548b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 4 client.18921 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d554558e30 con 0x55d5548b7be0 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.740+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5548b7be0 0x55d5548b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.741+0000 7fe6b6ffd640 10 client.18921 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.741+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5548b7be0 0x55d5548b9fd0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e85ddb553537a1 server_cookie=1cca22bf4236690a in_seq=0 out_seq=0 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.741+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/3009561321 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d554558e30 con 0x55d5548b7be0 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.741+0000 7fe6b6ffd640 10 client.18921 handle_command_reply: tid=0 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.741+0000 7fe6e0620740 1 client.18921 shutdown 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 msgr2=0x55d5548a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a5110 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0011010 tx=0x7fe6d0005a50 comp rx=0 tx=0).stop 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 shutdown_connections 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d5548b7be0 0x55d5548b9fd0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a5110 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 --2- 172.21.15.44:0/3009561321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489ce10 0x55d5548a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6b6ffd640 0 client.18921 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 >> 172.21.15.44:0/3009561321 conn(0x55d554899380 msgr2=0x55d55489f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 shutdown_connections 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 1 -- 172.21.15.44:0/3009561321 wait complete. 2024-09-13T17:37:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.742+0000 7fe6e0620740 20 client.18921 trim_cache size 0 max 16384 2024-09-13T17:37:07.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 Processor -- start 2024-09-13T17:37:07.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 -- start start 2024-09-13T17:37:07.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489ce10 0x55d55489fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489d730 0x55d5548a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55489dca0 con 0x55d5548a7fd0 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5548bf1a0 con 0x55d55489ce10 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6e0620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d554897af0 con 0x55d55489d730 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40552/0 (socket says 172.21.15.44:40552) 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 -- 172.21.15.44:0/2857608315 learned_addr learned my addr 172.21.15.44:0/2857608315 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489d730 0x55d5548a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489ce10 0x55d55489fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 -- 172.21.15.44:0/2857608315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489d730 msgr2=0x55d5548a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489d730 0x55d5548a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 -- 172.21.15.44:0/2857608315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489ce10 msgr2=0x55d55489fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489ce10 0x55d55489fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.756+0000 7fe6d77fe640 1 -- 172.21.15.44:0/2857608315 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5547a1790 con 0x55d5548a7fd0 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a3440 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005880 tx=0x7fe6d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b89ae0fca03fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6be7fc640 1 -- 172.21.15.44:0/2857608315 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0005bd0 con 0x55d5548a7fd0 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6be7fc640 1 -- 172.21.15.44:0/2857608315 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6d0023070 con 0x55d5548a7fd0 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6be7fc640 1 -- 172.21.15.44:0/2857608315 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d001a590 con 0x55d5548a7fd0 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 -- 172.21.15.44:0/2857608315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 msgr2=0x55d5548a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a3440 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005880 tx=0x7fe6d0000f40 comp rx=0 tx=0).stop 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 -- 172.21.15.44:0/2857608315 shutdown_connections 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d55489d730 0x55d5548a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5548a7fd0 0x55d5548a3440 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 --2- 172.21.15.44:0/2857608315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489ce10 0x55d55489fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 -- 172.21.15.44:0/2857608315 >> 172.21.15.44:0/2857608315 conn(0x55d554899380 msgr2=0x55d554899760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 -- 172.21.15.44:0/2857608315 shutdown_connections 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.757+0000 7fe6e0620740 1 -- 172.21.15.44:0/2857608315 wait complete. 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 -- start start 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 Processor -- start 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d5548ace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d55490d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5548ada80 con 0x55d55489ce10 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55490d5b0 con 0x55d55489d730 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55490d740 con 0x55d5548a7fd0 2024-09-13T17:37:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6e0620740 10 client.0 _ll_register_callbacks cb 0x7fe6d0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d5548ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d5548ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40554/0 (socket says 172.21.15.44:40554) 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 -- 172.21.15.44:0/1112264297 learned_addr learned my addr 172.21.15.44:0/1112264297 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d55490d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 -- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 msgr2=0x55d55490d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d55490d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 -- 172.21.15.44:0/1112264297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 msgr2=0x55d5548ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.758+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6d77fe640 1 -- 172.21.15.44:0/1112264297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5547ff620 con 0x55d55489ce10 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6d77fe640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d5548ace00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0011010 tx=0x7fe6d001a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a4bbd4d7bdebc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:07.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/1112264297 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000cbc0 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/1112264297 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fe6d000cd20 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5544f44e0 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/1112264297 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d001a510 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 10 client.18927 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d5547c6ea0 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/1112264297 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d5544f44e0 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/1112264297 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6d0012870 con 0x55d55489ce10 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 10 client.18927 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 20 client.18927 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d55490ea40 0x55d554910e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 4 client.18927 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.759+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d5549115f0 con 0x55d55490ea40 2024-09-13T17:37:07.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.760+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d55490ea40 0x55d554910e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:07.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.760+0000 7fe6d67fc640 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d55490ea40 0x55d554910e30 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ab05180f0e82c2 server_cookie=5393769b1e52f621 in_seq=0 out_seq=0 2024-09-13T17:37:07.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.760+0000 7fe6b6ffd640 10 client.18927 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6b6ffd640 1 -- 172.21.15.44:0/1112264297 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d5549115f0 con 0x55d55490ea40 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6b6ffd640 10 client.18927 handle_command_reply: tid=0 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 client.18927 shutdown 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 msgr2=0x55d5548ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d5548ace00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0011010 tx=0x7fe6d001a410 comp rx=0 tx=0).stop 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 shutdown_connections 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55d55490ea40 0x55d554910e30 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5548a7fd0 0x55d55490d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6b6ffd640 0 client.18927 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d55489d730 0x55d5548ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 --2- 172.21.15.44:0/1112264297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d55489ce10 0x55d5548ace00 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 >> 172.21.15.44:0/1112264297 conn(0x55d554899380 msgr2=0x55d554899760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 shutdown_connections 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 1 -- 172.21.15.44:0/1112264297 wait complete. 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6e0620740 20 client.18927 trim_cache size 0 max 16384 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:37:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "81e9c169-979e-45f3-ba70-1ebbd8d2c917": { 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "81e9c169-979e-45f3-ba70-1ebbd8d2c917", 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:37:07.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6c00a1230 msgr2=0x7fe6c00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6c00a1230 0x7fe6c00a36f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80ca610 tx=0x7fe6c800fbd0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 msgr2=0x7fe6d810c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:07.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 0x7fe6d810c430 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800c920 tx=0x7fe6c800cde0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 shutdown_connections 2024-09-13T17:37:07.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6c00a1230 0x7fe6c00a36f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d810ceb0 0x7fe6d81a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d8103fd0 0x7fe6d810c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 --2- 172.21.15.44:0/355039401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6d80fee80 0x7fe6d810c430 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.761+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 >> 172.21.15.44:0/355039401 conn(0x7fe6d80fa930 msgr2=0x7fe6d80fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.762+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 shutdown_connections 2024-09-13T17:37:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:07.762+0000 7fe6dd901640 1 -- 172.21.15.44:0/355039401 wait complete. 2024-09-13T17:37:07.774 INFO:tasks.cephfs.filesystem:scrub status for tag:81e9c169-979e-45f3-ba70-1ebbd8d2c917 - {'path': '/', 'tag': '81e9c169-979e-45f3-ba70-1ebbd8d2c917', 'options': 'recursive,force'} 2024-09-13T17:37:07.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:07 smithi100 ceph-mon[35055]: pgmap v324: 129 pgs: 129 active+clean; 4.0 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-13T17:37:08.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:08 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:37:08.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:08 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:08.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:08 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:37:08.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:08 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:08 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:37:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:08 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:08 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:37:08.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:08 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:08 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:37:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:08 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:08 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:37:08.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:08 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:09.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:09 smithi070 ceph-mon[34502]: pgmap v325: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 34 op/s 2024-09-13T17:37:09.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:09 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:37:09.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:09 smithi100 ceph-mon[35055]: pgmap v325: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 34 op/s 2024-09-13T17:37:09.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:09 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:37:09.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:09 smithi044 ceph-mon[31175]: pgmap v325: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 34 op/s 2024-09-13T17:37:09.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:09 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:37:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1504282231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80ff780 msgr2=0x7fcbb81037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1504282231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80ff780 0x7fcbb81037c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009930 tx=0x7fcbac02a420 comp rx=0 tx=0).stop 2024-09-13T17:37:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1504282231 shutdown_connections 2024-09-13T17:37:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1504282231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb8103f20 0x7fcbb810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1504282231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80ff780 0x7fcbb81037c0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1504282231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80fedd0 0x7fcbb80ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1504282231 >> 172.21.15.44:0/1504282231 conn(0x7fcbb80fa860 msgr2=0x7fcbb80fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.506+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1504282231 shutdown_connections 2024-09-13T17:37:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.507+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1504282231 wait complete. 2024-09-13T17:37:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.507+0000 7fcbbf60d640 1 Processor -- start 2024-09-13T17:37:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.507+0000 7fcbbf60d640 1 -- start start 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.507+0000 7fcbbf60d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80fedd0 0x7fcbb8100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.507+0000 7fcbbf60d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 0x7fcbb8100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.507+0000 7fcbbf60d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb81012e0 0x7fcbb81a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbf60d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb81a4950 con 0x7fcbb8103f20 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbf60d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb81a4ac0 con 0x7fcbb81012e0 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbf60d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb81a4c80 con 0x7fcbb80fedd0 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 0x7fcbb8100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 0x7fcbb8100da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35146/0 (socket says 172.21.15.44:35146) 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 -- 172.21.15.44:0/1154876219 learned_addr learned my addr 172.21.15.44:0/1154876219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb81012e0 0x7fcbb81a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80fedd0 0x7fcbb8100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 -- 172.21.15.44:0/1154876219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80fedd0 msgr2=0x7fcbb8100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80fedd0 0x7fcbb8100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 -- 172.21.15.44:0/1154876219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb81012e0 msgr2=0x7fcbb81a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb81012e0 0x7fcbb81a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 -- 172.21.15.44:0/1154876219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac009560 con 0x7fcbb8103f20 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb81012e0 0x7fcbb81a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 0x7fcbb8100da0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fcbac005ab0 tx=0x7fcbac032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbaa7fc640 1 -- 172.21.15.44:0/1154876219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0044d0 con 0x7fcbb8103f20 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbaa7fc640 1 -- 172.21.15.44:0/1154876219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbac032e10 con 0x7fcbb8103f20 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb81a4f00 con 0x7fcbb8103f20 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.508+0000 7fcbaa7fc640 1 -- 172.21.15.44:0/1154876219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac003850 con 0x7fcbb8103f20 2024-09-13T17:37:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.509+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb81a5380 con 0x7fcbb8103f20 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.511+0000 7fcbaa7fc640 1 -- 172.21.15.44:0/1154876219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcbac004850 con 0x7fcbb8103f20 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.511+0000 7fcbc240c740 1 Processor -- start 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.511+0000 7fcbc240c740 1 -- start start 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af01710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5af0bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af0e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbc240c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ae6cc30 con 0x560b5af0c4f0 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbc240c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ae6a3c0 con 0x560b5af01c50 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbc240c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ae65cb0 con 0x560b5af01330 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af0e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af0e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35154/0 (socket says 172.21.15.44:35154) 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3583253470 learned_addr learned my addr 172.21.15.44:0/3583253470 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbaa7fc640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcba40a1280 0x7fcba40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5af0bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af01710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcba40a1280 0x7fcba40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbaa7fc640 1 -- 172.21.15.44:0/1154876219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fcbac0ee070 con 0x7fcbb8103f20 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3583253470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 msgr2=0x560b5af01710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af01710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3583253470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 msgr2=0x560b5af0bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5af0bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3583253470 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b5ae0c910 con 0x560b5af0c4f0 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.512+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5af0bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.513+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af0e8e0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0eb300 tx=0x7fcbac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c6cbb91db95205 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.513+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcba40a1280 0x7fcba40a3740 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009fd0 tx=0x7fcbb0009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.513+0000 7fcba37fe640 1 -- 172.21.15.44:0/3583253470 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0ba890 con 0x560b5af0c4f0 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.513+0000 7fcba37fe640 1 -- 172.21.15.44:0/3583253470 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbac0ba6b0 con 0x560b5af0c4f0 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.513+0000 7fcba37fe640 1 -- 172.21.15.44:0/3583253470 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0c31d0 con 0x560b5af0c4f0 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 -- 172.21.15.44:0/3583253470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 msgr2=0x560b5af0e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af0e8e0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0eb300 tx=0x7fcbac009260 comp rx=0 tx=0).stop 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 -- 172.21.15.44:0/3583253470 shutdown_connections 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af0e8e0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5af0bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3583253470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af01710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 -- 172.21.15.44:0/3583253470 >> 172.21.15.44:0/3583253470 conn(0x560b5aefd8c0 msgr2=0x560b5aefdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 -- 172.21.15.44:0/3583253470 shutdown_connections 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 -- 172.21.15.44:0/3583253470 wait complete. 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 -- start start 2024-09-13T17:37:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.514+0000 7fcbc240c740 1 Processor -- start 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af0bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af09840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ae23e80 con 0x560b5af01c50 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ae23b20 con 0x560b5af0c4f0 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ade9260 con 0x560b5af01330 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbc240c740 10 client.0 _ll_register_callbacks cb 0x560b5ae1edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35156/0 (socket says 172.21.15.44:35156) 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3090239419 learned_addr learned my addr 172.21.15.44:0/3090239419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af09840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af0bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 msgr2=0x560b5af0bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af0bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3090239419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 msgr2=0x560b5af09840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af09840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 -- 172.21.15.44:0/3090239419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb81a5380 con 0x560b5af01c50 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af09840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af0bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af091e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0b7250 tx=0x7fcbac0bebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a38d9ed2bf5f0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.515+0000 7fcba0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcba0ff9640 1 -- 172.21.15.44:0/3090239419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac004e60 con 0x560b5af01c50 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcba0ff9640 1 -- 172.21.15.44:0/3090239419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fcbac0bad50 con 0x560b5af01c50 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b5ae8e680 con 0x560b5af01c50 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcba0ff9640 1 -- 172.21.15.44:0/3090239419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbac0eb150 con 0x560b5af01c50 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 10 client.18939 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b5ae64330 con 0x560b5af01c50 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcba0ff9640 1 -- 172.21.15.44:0/3090239419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560b5ae8e680 con 0x560b5af01c50 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcba0ff9640 1 -- 172.21.15.44:0/3090239419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbac04d070 con 0x560b5af01c50 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 10 client.18939 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 10 client.18939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 20 client.18939 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af1b730 0x560b5af1db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 4 client.18939 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.516+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b5ae595d0 con 0x560b5af1b730 2024-09-13T17:37:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.517+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af1b730 0x560b5af1db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.517+0000 7fcba0ff9640 10 client.18939 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.517+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af1b730 0x560b5af1db20 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb167ba775ca3832 server_cookie=55128db815e37d0c in_seq=0 out_seq=0 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcba0ff9640 1 -- 172.21.15.44:0/3090239419 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560b5ae595d0 con 0x560b5af1b730 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcba0ff9640 10 client.18939 handle_command_reply: tid=0 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 client.18939 shutdown 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 msgr2=0x560b5af091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af091e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0b7250 tx=0x7fcbac0bebf0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 shutdown_connections 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af1b730 0x560b5af1db20 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af09840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af091e0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcba0ff9640 0 client.18939 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 --2- 172.21.15.44:0/3090239419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af0bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 >> 172.21.15.44:0/3090239419 conn(0x560b5aefd8c0 msgr2=0x560b5aefcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 shutdown_connections 2024-09-13T17:37:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 1 -- 172.21.15.44:0/3090239419 wait complete. 2024-09-13T17:37:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.518+0000 7fcbc240c740 20 client.18939 trim_cache size 0 max 16384 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 Processor -- start 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 -- start start 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5aef8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5aef8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af07a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5af18940 con 0x560b5af0c4f0 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5af182a0 con 0x560b5af01c50 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbc240c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5af09e40 con 0x560b5af01330 2024-09-13T17:37:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbbdbaa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af07a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbbdbaa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af07a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35162/0 (socket says 172.21.15.44:35162) 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbbdbaa640 1 -- 172.21.15.44:0/1163138874 learned_addr learned my addr 172.21.15.44:0/1163138874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5aef8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.531+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5aef8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbdbaa640 1 -- 172.21.15.44:0/1163138874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 msgr2=0x560b5aef8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5aef8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbdbaa640 1 -- 172.21.15.44:0/1163138874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 msgr2=0x560b5aef8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5aef8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbdbaa640 1 -- 172.21.15.44:0/1163138874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b5ae06300 con 0x560b5af0c4f0 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5aef8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af07a90 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0012db0 tx=0x7fcbb00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54bf481c615fabc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcba3fff640 1 -- 172.21.15.44:0/1163138874 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000dd00 con 0x560b5af0c4f0 2024-09-13T17:37:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcba3fff640 1 -- 172.21.15.44:0/1163138874 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbb000de60 con 0x560b5af0c4f0 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcba3fff640 1 -- 172.21.15.44:0/1163138874 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00059c0 con 0x560b5af0c4f0 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbc240c740 1 -- 172.21.15.44:0/1163138874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 msgr2=0x560b5af07a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.532+0000 7fcbc240c740 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af07a90 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0012db0 tx=0x7fcbb00099e0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- 172.21.15.44:0/1163138874 shutdown_connections 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af0c4f0 0x560b5af07a90 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af01c50 0x560b5aef8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 --2- 172.21.15.44:0/1163138874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5aef8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- 172.21.15.44:0/1163138874 >> 172.21.15.44:0/1163138874 conn(0x560b5aefd8c0 msgr2=0x560b5af044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- 172.21.15.44:0/1163138874 shutdown_connections 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- 172.21.15.44:0/1163138874 wait complete. 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- start start 2024-09-13T17:37:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 Processor -- start 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af02f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af03440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5af03ae0 con 0x560b5af01c50 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5af72110 con 0x560b5af0c4f0 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5af72250 con 0x560b5af01330 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.533+0000 7fcbc240c740 10 client.0 _ll_register_callbacks cb 0x7fcbac0b7110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af03440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af03440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35168/0 (socket says 172.21.15.44:35168) 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af02f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 -- 172.21.15.44:0/2866202649 learned_addr learned my addr 172.21.15.44:0/2866202649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbdbaa640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 -- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 msgr2=0x560b5af02f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af02f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 -- 172.21.15.44:0/2866202649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 msgr2=0x560b5af71bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af71bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 -- 172.21.15.44:0/2866202649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb001a040 con 0x560b5af01c50 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbbcba8640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af03440 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0da450 tx=0x7fcbac0bb1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6c303731f31ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcba0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcba0ff9640 1 -- 172.21.15.44:0/2866202649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0f0e70 con 0x560b5af01c50 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcba0ff9640 1 -- 172.21.15.44:0/2866202649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fcbac0f1610 con 0x560b5af01c50 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.534+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbac0ba6b0 con 0x560b5af01c50 2024-09-13T17:37:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcba0ff9640 1 -- 172.21.15.44:0/2866202649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbac004e60 con 0x560b5af01c50 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 10 client.18945 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b5ae2b8e0 con 0x560b5af01c50 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcba0ff9640 1 -- 172.21.15.44:0/2866202649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbac0ba6b0 con 0x560b5af01c50 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcba0ff9640 1 -- 172.21.15.44:0/2866202649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbac0ef060 con 0x560b5af01c50 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 10 client.18945 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 10 client.18945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 20 client.18945 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af733a0 0x560b5af75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 4 client.18945 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560b5ab59300 con 0x560b5af733a0 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.535+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af733a0 0x560b5af75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.536+0000 7fcbbd3a9640 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af733a0 0x560b5af75790 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ce422d837db8e1 server_cookie=e431185695a908b5 in_seq=0 out_seq=0 2024-09-13T17:37:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.536+0000 7fcba0ff9640 10 client.18945 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcba0ff9640 1 -- 172.21.15.44:0/2866202649 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560b5ab59300 con 0x560b5af733a0 2024-09-13T17:37:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcba0ff9640 10 client.18945 handle_command_reply: tid=0 2024-09-13T17:37:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 client.18945 shutdown 2024-09-13T17:37:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 msgr2=0x560b5af03440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af03440 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0da450 tx=0x7fcbac0bb1c0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 shutdown_connections 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x560b5af733a0 0x560b5af75790 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560b5af0c4f0 0x560b5af71bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5af01c50 0x560b5af03440 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcba0ff9640 0 client.18945 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 --2- 172.21.15.44:0/2866202649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b5af01330 0x560b5af02f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 >> 172.21.15.44:0/2866202649 conn(0x560b5aefd8c0 msgr2=0x560b5aef8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 shutdown_connections 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 1 -- 172.21.15.44:0/2866202649 wait complete. 2024-09-13T17:37:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.548+0000 7fcbc240c740 20 client.18945 trim_cache size 0 max 16384 2024-09-13T17:37:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:37:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:37:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:37:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcba40a1280 msgr2=0x7fcba40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcba40a1280 0x7fcba40a3740 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009fd0 tx=0x7fcbb0009500 comp rx=0 tx=0).stop 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 msgr2=0x7fcbb8100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 0x7fcbb8100da0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fcbac005ab0 tx=0x7fcbac032c20 comp rx=0 tx=0).stop 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 shutdown_connections 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcba40a1280 0x7fcba40a3740 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb81012e0 0x7fcbb81a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb8103f20 0x7fcbb8100da0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 --2- 172.21.15.44:0/1154876219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80fedd0 0x7fcbb8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 >> 172.21.15.44:0/1154876219 conn(0x7fcbb80fa860 msgr2=0x7fcbb8068790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 shutdown_connections 2024-09-13T17:37:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:10.549+0000 7fcbbf60d640 1 -- 172.21.15.44:0/1154876219 wait complete. 2024-09-13T17:37:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:12 smithi044 ceph-mon[31175]: pgmap v326: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 34 op/s 2024-09-13T17:37:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:12 smithi044 ceph-mon[31175]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T17:37:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:12 smithi070 ceph-mon[34502]: pgmap v326: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 34 op/s 2024-09-13T17:37:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:12 smithi070 ceph-mon[34502]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T17:37:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:12 smithi100 ceph-mon[35055]: pgmap v326: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 34 op/s 2024-09-13T17:37:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:12 smithi100 ceph-mon[35055]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T17:37:14.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:14 smithi044 ceph-mon[31175]: pgmap v328: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-13T17:37:14.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:14 smithi070 ceph-mon[34502]: pgmap v328: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-13T17:37:14.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:14 smithi100 ceph-mon[35055]: pgmap v328: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-13T17:37:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:16 smithi044 ceph-mon[31175]: pgmap v329: 129 pgs: 129 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:37:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:16 smithi070 ceph-mon[34502]: pgmap v329: 129 pgs: 129 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:37:16.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:16 smithi100 ceph-mon[35055]: pgmap v329: 129 pgs: 129 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:37:18.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:18 smithi044 ceph-mon[31175]: pgmap v330: 129 pgs: 129 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:37:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:18 smithi070 ceph-mon[34502]: pgmap v330: 129 pgs: 129 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:37:18.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:18 smithi100 ceph-mon[35055]: pgmap v330: 129 pgs: 129 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-13T17:37:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:20 smithi044 ceph-mon[31175]: pgmap v331: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:37:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:20 smithi070 ceph-mon[34502]: pgmap v331: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:37:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:20 smithi100 ceph-mon[35055]: pgmap v331: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:37:20.796 INFO:tasks.workunit.client.0.smithi044.stdout:499/600: open fd = 3 2024-09-13T17:37:20.797 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.797 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.797 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.797 INFO:tasks.workunit.client.0.smithi044.stdout:500/600: open fd = 3 2024-09-13T17:37:20.797 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:501/600: open fd = 3 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:502/600: open fd = 3 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.798 INFO:tasks.workunit.client.0.smithi044.stdout:503/600: open fd = 3 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:504/600: open fd = 3 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:505/600: open fd = 3 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:506/600: open fd = 3 2024-09-13T17:37:20.799 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:507/600: open fd = 3 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:508/600: open fd = 3 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:509/600: open fd = 3 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.800 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:510/600: open fd = 3 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:511/600: open fd = 3 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:512/600: open fd = 3 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.801 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:513/600: open fd = 3 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:514/600: open fd = 3 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.802 INFO:tasks.workunit.client.0.smithi044.stdout:515/600: open fd = 3 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:516/600: open fd = 3 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:517/600: open fd = 3 2024-09-13T17:37:20.803 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:518/600: open fd = 3 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:519/600: open fd = 3 2024-09-13T17:37:20.804 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:520/600: open fd = 3 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:521/600: open fd = 3 2024-09-13T17:37:20.805 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:522/600: open fd = 3 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:523/600: open fd = 3 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.806 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:524/600: open fd = 3 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:525/600: open fd = 3 2024-09-13T17:37:20.807 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:526/600: open fd = 3 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.808 INFO:tasks.workunit.client.0.smithi044.stdout:527/600: open fd = 3 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:528/600: open fd = 3 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.809 INFO:tasks.workunit.client.0.smithi044.stdout:529/600: open fd = 3 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:530/600: open fd = 3 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.810 INFO:tasks.workunit.client.0.smithi044.stdout:531/600: open fd = 3 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:532/600: open fd = 3 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.811 INFO:tasks.workunit.client.0.smithi044.stdout:533/600: open fd = 3 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:534/600: open fd = 3 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.812 INFO:tasks.workunit.client.0.smithi044.stdout:535/600: open fd = 3 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:536/600: open fd = 3 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:537/600: open fd = 3 2024-09-13T17:37:20.813 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:538/600: open fd = 3 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:539/600: open fd = 3 2024-09-13T17:37:20.814 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:540/600: open fd = 3 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:541/600: open fd = 3 2024-09-13T17:37:20.815 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:542/600: open fd = 3 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:543/600: open fd = 3 2024-09-13T17:37:20.816 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:544/600: open fd = 3 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:545/600: open fd = 3 2024-09-13T17:37:20.817 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:546/600: open fd = 3 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:547/600: open fd = 3 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.818 INFO:tasks.workunit.client.0.smithi044.stdout:548/600: open fd = 3 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:549/600: open fd = 3 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:550/600: open fd = 3 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:551/600: open fd = 3 2024-09-13T17:37:20.819 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:552/600: open fd = 3 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:553/600: open fd = 3 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:554/600: open fd = 3 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.820 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:555/600: open fd = 3 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:556/600: open fd = 3 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:557/600: open fd = 3 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.821 INFO:tasks.workunit.client.0.smithi044.stdout:558/600: open fd = 3 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:559/600: open fd = 3 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:560/600: open fd = 3 2024-09-13T17:37:20.822 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.823 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:37:20.823 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:37:22.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:22 smithi044 ceph-mon[31175]: pgmap v332: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:37:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:22 smithi070 ceph-mon[34502]: pgmap v332: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:37:22.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:22 smithi100 ceph-mon[35055]: pgmap v332: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-13T17:37:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 -- 172.21.15.44:0/4070031399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 msgr2=0x7ff11410b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 --2- 172.21.15.44:0/4070031399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff11410b250 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009ca0 tx=0x7ff10c02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 -- 172.21.15.44:0/4070031399 shutdown_connections 2024-09-13T17:37:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 --2- 172.21.15.44:0/4070031399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1140fef60 0x7ff114102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 --2- 172.21.15.44:0/4070031399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff11410b250 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 --2- 172.21.15.44:0/4070031399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1140fe5b0 0x7ff1140fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 -- 172.21.15.44:0/4070031399 >> 172.21.15.44:0/4070031399 conn(0x7ff114075340 msgr2=0x7ff1140fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 -- 172.21.15.44:0/4070031399 shutdown_connections 2024-09-13T17:37:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.647+0000 7ff11909c640 1 -- 172.21.15.44:0/4070031399 wait complete. 2024-09-13T17:37:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.648+0000 7ff11909c640 1 Processor -- start 2024-09-13T17:37:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.648+0000 7ff11909c640 1 -- start start 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.648+0000 7ff11909c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1140fe5b0 0x7ff1140fa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.648+0000 7ff11909c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff1140facd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11909c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1140fb210 0x7ff1141a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11909c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1141a47f0 con 0x7ff114103700 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11909c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1141a4960 con 0x7ff1140fb210 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11909c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1141a4b20 con 0x7ff1140fe5b0 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff1140facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff1140facd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60104/0 (socket says 172.21.15.44:60104) 2024-09-13T17:37:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 -- 172.21.15.44:0/3210374858 learned_addr learned my addr 172.21.15.44:0/3210374858 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff112d9d640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1140fe5b0 0x7ff1140fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11359e640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1140fb210 0x7ff1141a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 -- 172.21.15.44:0/3210374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1140fe5b0 msgr2=0x7ff1140fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1140fe5b0 0x7ff1140fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 -- 172.21.15.44:0/3210374858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1140fb210 msgr2=0x7ff1141a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1140fb210 0x7ff1141a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 -- 172.21.15.44:0/3210374858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c009900 con 0x7ff114103700 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11259c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff1140facd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d740 tx=0x7ff10800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff0fbfff640 1 -- 172.21.15.44:0/3210374858 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108005220 con 0x7ff114103700 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff0fbfff640 1 -- 172.21.15.44:0/3210374858 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff108009cb0 con 0x7ff114103700 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.649+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1141a4e00 con 0x7ff114103700 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.650+0000 7ff0fbfff640 1 -- 172.21.15.44:0/3210374858 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108004cf0 con 0x7ff114103700 2024-09-13T17:37:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.650+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1141a5230 con 0x7ff114103700 2024-09-13T17:37:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff0fbfff640 1 -- 172.21.15.44:0/3210374858 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff108009820 con 0x7ff114103700 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 Processor -- start 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 -- start start 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcfc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abc5ac30 con 0x5639abcfa4f0 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abc583c0 con 0x5639abcef330 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.652+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abc53cb0 con 0x5639abcefc50 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcfc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcfc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60116/0 (socket says 172.21.15.44:60116) 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 -- 172.21.15.44:0/1109011847 learned_addr learned my addr 172.21.15.44:0/1109011847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11359e640 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff112d9d640 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff0fbfff640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0fc0a1200 0x7ff0fc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 -- 172.21.15.44:0/1109011847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 msgr2=0x5639abcf9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 -- 172.21.15.44:0/1109011847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 msgr2=0x5639abcef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 -- 172.21.15.44:0/1109011847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639abbfa910 con 0x5639abcfa4f0 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11359e640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0fc0a1200 0x7ff0fc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff0fbfff640 1 -- 172.21.15.44:0/3210374858 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff1080cc9c0 con 0x7ff114103700 2024-09-13T17:37:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.653+0000 7ff11259c640 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcfc8e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff1080c9d10 tx=0x7ff1080ad270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5544a0f344347ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff0f97fa640 1 -- 172.21.15.44:0/1109011847 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080992a0 con 0x5639abcfa4f0 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff0f97fa640 1 -- 172.21.15.44:0/1109011847 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1080d26d0 con 0x5639abcfa4f0 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff0f97fa640 1 -- 172.21.15.44:0/1109011847 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108004e50 con 0x5639abcfa4f0 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1109011847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 msgr2=0x5639abcfc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11359e640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0fc0a1200 0x7ff0fc0a36c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009430 tx=0x7ff10c0095c0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcfc8e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff1080c9d10 tx=0x7ff1080ad270 comp rx=0 tx=0).stop 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1109011847 shutdown_connections 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcfc8e0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1109011847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1109011847 >> 172.21.15.44:0/1109011847 conn(0x5639abceb8c0 msgr2=0x5639abcebca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1109011847 shutdown_connections 2024-09-13T17:37:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.654+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1109011847 wait complete. 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 -- start start 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 Processor -- start 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abcf9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcfa4f0 0x5639abcf7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abc11e80 con 0x5639abcef330 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abc11b20 con 0x5639abcfa4f0 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abbd7260 con 0x5639abcefc50 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11bdc4740 10 client.0 _ll_register_callbacks cb 0x5639abc0cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff112d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abcf9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff112d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abcf9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60128/0 (socket says 172.21.15.44:60128) 2024-09-13T17:37:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff112d9d640 1 -- 172.21.15.44:0/2549434974 learned_addr learned my addr 172.21.15.44:0/2549434974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.655+0000 7ff11259c640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff112d9d640 1 -- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 msgr2=0x5639abcf7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff112d9d640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff112d9d640 1 -- 172.21.15.44:0/2549434974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcfa4f0 msgr2=0x5639abcf7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff112d9d640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcfa4f0 0x5639abcf7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff112d9d640 1 -- 172.21.15.44:0/2549434974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1141a5230 con 0x5639abcef330 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff11259c640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff112d9d640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abcf9e90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff104010a50 tx=0x7ff104010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311cd45372fcd050 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff0f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/2549434974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104015070 con 0x5639abcef330 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/2549434974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff10400fbd0 con 0x5639abcef330 2024-09-13T17:37:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639abc475d0 con 0x5639abcef330 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/2549434974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff10400fd90 con 0x5639abcef330 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff11bdc4740 10 client.18957 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639abc52330 con 0x5639abcef330 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/2549434974 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5639abc475d0 con 0x5639abcef330 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.656+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/2549434974 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff104018370 con 0x5639abcef330 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11bdc4740 10 client.18957 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11bdc4740 10 client.18957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11bdc4740 20 client.18957 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd09f70 0x5639abd0c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11bdc4740 4 client.18957 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639ab947300 con 0x5639abd09f70 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11259c640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd09f70 0x5639abd0c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff0f6ff5640 10 client.18957 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.657+0000 7ff11259c640 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd09f70 0x5639abd0c390 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33db61484036467 server_cookie=c02bba91ed1fb82a in_seq=0 out_seq=0 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/2549434974 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5639ab947300 con 0x5639abd09f70 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff0f6ff5640 10 client.18957 handle_command_reply: tid=0 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 client.18957 shutdown 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 msgr2=0x5639abcf9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abcf9e90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff104010a50 tx=0x7ff104010f10 comp rx=0 tx=0).stop 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 shutdown_connections 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd09f70 0x5639abd0c390 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcfa4f0 0x5639abcf7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/2549434974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abcf9e90 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff0f6ff5640 0 client.18957 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 >> 172.21.15.44:0/2549434974 conn(0x5639abceb8c0 msgr2=0x5639abceab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 shutdown_connections 2024-09-13T17:37:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 1 -- 172.21.15.44:0/2549434974 wait complete. 2024-09-13T17:37:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.658+0000 7ff11bdc4740 20 client.18957 trim_cache size 0 max 16384 2024-09-13T17:37:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 Processor -- start 2024-09-13T17:37:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 -- start start 2024-09-13T17:37:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcf99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcf5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abcf64b0 con 0x5639abcfa4f0 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abd08780 con 0x5639abcef330 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abcf8220 con 0x5639abcefc50 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcf5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcf5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60132/0 (socket says 172.21.15.44:60132) 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11359e640 1 -- 172.21.15.44:0/1883970853 learned_addr learned my addr 172.21.15.44:0/1883970853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff11259c640 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.671+0000 7ff112d9d640 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcf99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11359e640 1 -- 172.21.15.44:0/1883970853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 msgr2=0x5639abcf5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11359e640 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11359e640 1 -- 172.21.15.44:0/1883970853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 msgr2=0x5639abcf99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11359e640 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcf99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11359e640 1 -- 172.21.15.44:0/1883970853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639abbf4300 con 0x5639abcfa4f0 2024-09-13T17:37:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff112d9d640 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcf99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11359e640 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcf5f70 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff10c035000 tx=0x7ff10c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9c6ed70ff37498 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff0f9ffb640 1 -- 172.21.15.44:0/1883970853 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0318e0 con 0x5639abcfa4f0 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff0f9ffb640 1 -- 172.21.15.44:0/1883970853 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff10c042070 con 0x5639abcfa4f0 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff0f9ffb640 1 -- 172.21.15.44:0/1883970853 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0045b0 con 0x5639abcfa4f0 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1883970853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 msgr2=0x5639abcf5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcf5f70 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff10c035000 tx=0x7ff10c042000 comp rx=0 tx=0).stop 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1883970853 shutdown_connections 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcefc50 0x5639abcf5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcfa4f0 0x5639abcf5f70 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1883970853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcef330 0x5639abcf99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.672+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1883970853 >> 172.21.15.44:0/1883970853 conn(0x5639abceb8c0 msgr2=0x5639abce6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1883970853 shutdown_connections 2024-09-13T17:37:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1883970853 wait complete. 2024-09-13T17:37:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 -- start start 2024-09-13T17:37:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 Processor -- start 2024-09-13T17:37:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abd01420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcefc50 0x5639abcfd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcfa4f0 0x5639abcfdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abcfe270 con 0x5639abcef330 2024-09-13T17:37:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abcfe3e0 con 0x5639abcefc50 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abcfe5a0 con 0x5639abcfa4f0 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11bdc4740 10 client.0 _ll_register_callbacks cb 0x7ff1080d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff112d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abd01420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff112d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abd01420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60136/0 (socket says 172.21.15.44:60136) 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff112d9d640 1 -- 172.21.15.44:0/1880993682 learned_addr learned my addr 172.21.15.44:0/1880993682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11259c640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcefc50 0x5639abcfd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.673+0000 7ff11359e640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcfa4f0 0x5639abcfdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff112d9d640 1 -- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcfa4f0 msgr2=0x5639abcfdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff112d9d640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcfa4f0 0x5639abcfdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff112d9d640 1 -- 172.21.15.44:0/1880993682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcefc50 msgr2=0x5639abcfd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff112d9d640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcefc50 0x5639abcfd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff112d9d640 1 -- 172.21.15.44:0/1880993682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff114068020 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11259c640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcefc50 0x5639abcfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff112d9d640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abd01420 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff10400adf0 tx=0x7ff10400a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ff64662434f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff0f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/1880993682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104015040 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/1880993682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff104000e40 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639abc7c680 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/1880993682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1040189c0 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 10 client.18963 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639abc198e0 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/1880993682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5639abc7c680 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/1880993682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff104021070 con 0x5639abcef330 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 10 client.18963 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 10 client.18963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 20 client.18963 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd619b0 0x5639abd63dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 4 client.18963 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:37:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.674+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639ab958600 con 0x5639abd619b0 2024-09-13T17:37:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.675+0000 7ff11259c640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd619b0 0x5639abd63dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.675+0000 7ff0f6ff5640 10 client.18963 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.675+0000 7ff11259c640 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd619b0 0x5639abd63dd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411b62f5558f4333 server_cookie=29385588be0e4089 in_seq=0 out_seq=0 2024-09-13T17:37:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.689+0000 7ff0f6ff5640 1 -- 172.21.15.44:0/1880993682 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639ab958600 con 0x5639abd619b0 2024-09-13T17:37:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.689+0000 7ff0f6ff5640 10 client.18963 handle_command_reply: tid=0 2024-09-13T17:37:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 client.18963 shutdown 2024-09-13T17:37:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 msgr2=0x5639abd01420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abd01420 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff10400adf0 tx=0x7ff10400a910 comp rx=0 tx=0).stop 2024-09-13T17:37:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 shutdown_connections 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x5639abd619b0 0x5639abd63dd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639abcfa4f0 0x5639abcfdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639abcefc50 0x5639abcfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 --2- 172.21.15.44:0/1880993682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639abcef330 0x5639abd01420 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff0f6ff5640 0 client.18963 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 >> 172.21.15.44:0/1880993682 conn(0x5639abceb8c0 msgr2=0x5639abce6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 shutdown_connections 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 1 -- 172.21.15.44:0/1880993682 wait complete. 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11bdc4740 20 client.18963 trim_cache size 0 max 16384 2024-09-13T17:37:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:37:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:37:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:37:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:37:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0fc0a1200 msgr2=0x7ff0fc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11909c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0fc0a1200 0x7ff0fc0a36c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009430 tx=0x7ff10c0095c0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 msgr2=0x7ff1140facd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.690+0000 7ff11909c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff1140facd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d740 tx=0x7ff10800db40 comp rx=0 tx=0).stop 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 shutdown_connections 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff0fc0a1200 0x7ff0fc0a36c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1140fb210 0x7ff1141a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff114103700 0x7ff1140facd0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 --2- 172.21.15.44:0/3210374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1140fe5b0 0x7ff1140fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 >> 172.21.15.44:0/3210374858 conn(0x7ff114075340 msgr2=0x7ff114101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 shutdown_connections 2024-09-13T17:37:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:23.691+0000 7ff11909c640 1 -- 172.21.15.44:0/3210374858 wait complete. 2024-09-13T17:37:24.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:24 smithi044 ceph-mon[31175]: pgmap v333: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-13T17:37:24.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:24 smithi070 ceph-mon[34502]: pgmap v333: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-13T17:37:24.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:24 smithi100 ceph-mon[35055]: pgmap v333: 129 pgs: 129 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-13T17:37:26.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:26 smithi044 ceph-mon[31175]: pgmap v334: 129 pgs: 129 active+clean; 4.8 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:37:26.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:26 smithi070 ceph-mon[34502]: pgmap v334: 129 pgs: 129 active+clean; 4.8 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:37:26.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:26 smithi100 ceph-mon[35055]: pgmap v334: 129 pgs: 129 active+clean; 4.8 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:37:27.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:27 smithi070 ceph-mon[34502]: pgmap v335: 129 pgs: 129 active+clean; 4.8 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:37:27.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:27 smithi100 ceph-mon[35055]: pgmap v335: 129 pgs: 129 active+clean; 4.8 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:37:27.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:27 smithi044 ceph-mon[31175]: pgmap v335: 129 pgs: 129 active+clean; 4.8 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-13T17:37:29.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:29 smithi044 ceph-mon[31175]: pgmap v336: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:37:29.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:29 smithi070 ceph-mon[34502]: pgmap v336: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:37:30.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:29 smithi100 ceph-mon[35055]: pgmap v336: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:37:31.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:31 smithi044 ceph-mon[31175]: pgmap v337: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:31.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:31 smithi070 ceph-mon[34502]: pgmap v337: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:31 smithi100 ceph-mon[35055]: pgmap v337: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:33.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:33 smithi070 ceph-mon[34502]: pgmap v338: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:34.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:33 smithi100 ceph-mon[35055]: pgmap v338: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:34.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:33 smithi044 ceph-mon[31175]: pgmap v338: 129 pgs: 129 active+clean; 5.0 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:35.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:35 smithi044 ceph-mon[31175]: pgmap v339: 129 pgs: 129 active+clean; 5.2 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:37:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:35 smithi070 ceph-mon[34502]: pgmap v339: 129 pgs: 129 active+clean; 5.2 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:37:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:35 smithi100 ceph-mon[35055]: pgmap v339: 129 pgs: 129 active+clean; 5.2 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-13T17:37:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 -- 172.21.15.44:0/97243097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa034105040 msgr2=0x7fa03410cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 --2- 172.21.15.44:0/97243097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa034105040 0x7fa03410cb90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009cd0 tx=0x7fa02c02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 -- 172.21.15.44:0/97243097 shutdown_connections 2024-09-13T17:37:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 --2- 172.21.15.44:0/97243097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa034105040 0x7fa03410cb90 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 --2- 172.21.15.44:0/97243097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0341008a0 0x7fa0341048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 --2- 172.21.15.44:0/97243097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0340ffef0 0x7fa0341002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 -- 172.21.15.44:0/97243097 >> 172.21.15.44:0/97243097 conn(0x7fa0340fb960 msgr2=0x7fa0340fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 -- 172.21.15.44:0/97243097 shutdown_connections 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.790+0000 7fa038db5640 1 -- 172.21.15.44:0/97243097 wait complete. 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 Processor -- start 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 -- start start 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0340ffef0 0x7fa0341961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa034105040 0x7fa034196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 0x7fa03419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419b4e0 con 0x7fa03419aaf0 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419b650 con 0x7fa034105040 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa038db5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419b7c0 con 0x7fa0340ffef0 2024-09-13T17:37:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa032d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 0x7fa03419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa032d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 0x7fa03419afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37796/0 (socket says 172.21.15.44:37796) 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa032d9d640 1 -- 172.21.15.44:0/1519187681 learned_addr learned my addr 172.21.15.44:0/1519187681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.791+0000 7fa03259c640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0340ffef0 0x7fa0341961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa031d9b640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa034105040 0x7fa034196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa032d9d640 1 -- 172.21.15.44:0/1519187681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0340ffef0 msgr2=0x7fa0341961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa032d9d640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0340ffef0 0x7fa0341961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa032d9d640 1 -- 172.21.15.44:0/1519187681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa034105040 msgr2=0x7fa034196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa032d9d640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa034105040 0x7fa034196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa032d9d640 1 -- 172.21.15.44:0/1519187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c009900 con 0x7fa03419aaf0 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa031d9b640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa034105040 0x7fa034196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa032d9d640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 0x7fa03419afa0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa02c004af0 tx=0x7fa02c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa0237fe640 1 -- 172.21.15.44:0/1519187681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c003970 con 0x7fa03419aaf0 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0341a2da0 con 0x7fa03419aaf0 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa0237fe640 1 -- 172.21.15.44:0/1519187681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa02c004040 con 0x7fa03419aaf0 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa0237fe640 1 -- 172.21.15.44:0/1519187681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c045650 con 0x7fa03419aaf0 2024-09-13T17:37:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.792+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0341a3220 con 0x7fa03419aaf0 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa0237fe640 1 -- 172.21.15.44:0/1519187681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa02c00a040 con 0x7fa03419aaf0 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 Processor -- start 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 -- start start 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661ea8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e13c30 con 0x558661eb34f0 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e113c0 con 0x558661ea8330 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa03bad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e0ccb0 con 0x558661ea8c50 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37798/0 (socket says 172.21.15.44:37798) 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.795+0000 7fa031d9b640 1 -- 172.21.15.44:0/2479941188 learned_addr learned my addr 172.21.15.44:0/2479941188 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661ea8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa03259c640 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa0237fe640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa01c0a1200 0x7fa01c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 -- 172.21.15.44:0/2479941188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 msgr2=0x558661eb2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa032d9d640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa01c0a1200 0x7fa01c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 -- 172.21.15.44:0/2479941188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 msgr2=0x558661ea8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661ea8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 -- 172.21.15.44:0/2479941188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558661db3910 con 0x558661eb34f0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa0237fe640 1 -- 172.21.15.44:0/1519187681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa02c0ee470 con 0x7fa03419aaf0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa031d9b640 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb58e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa028005fd0 tx=0x7fa02800e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266a8b732028d2f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa020ff9640 1 -- 172.21.15.44:0/2479941188 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800b6a0 con 0x558661eb34f0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa032d9d640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa01c0a1200 0x7fa01c0a36c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0eb9a0 tx=0x7fa02c0a6040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa020ff9640 1 -- 172.21.15.44:0/2479941188 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa028019070 con 0x558661eb34f0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.796+0000 7fa020ff9640 1 -- 172.21.15.44:0/2479941188 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028011a40 con 0x558661eb34f0 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 -- 172.21.15.44:0/2479941188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 msgr2=0x558661eb58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb58e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa028005fd0 tx=0x7fa02800e3d0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 -- 172.21.15.44:0/2479941188 shutdown_connections 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb58e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 --2- 172.21.15.44:0/2479941188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661ea8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 -- 172.21.15.44:0/2479941188 >> 172.21.15.44:0/2479941188 conn(0x558661ea48c0 msgr2=0x558661ea4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 -- 172.21.15.44:0/2479941188 shutdown_connections 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.797+0000 7fa03bad6740 1 -- 172.21.15.44:0/2479941188 wait complete. 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 -- start start 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 Processor -- start 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 0x558661eb2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8c50 0x558661eb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661eb34f0 0x558661eb0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661dcae80 con 0x558661ea8330 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661dcab20 con 0x558661ea8c50 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661d90260 con 0x558661eb34f0 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03bad6740 10 client.0 _ll_register_callbacks cb 0x558661dc5df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 0x558661eb2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 0x558661eb2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37810/0 (socket says 172.21.15.44:37810) 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 -- 172.21.15.44:0/202370730 learned_addr learned my addr 172.21.15.44:0/202370730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa03259c640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8c50 0x558661eb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661eb34f0 0x558661eb0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 -- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661eb34f0 msgr2=0x558661eb0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661eb34f0 0x558661eb0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 -- 172.21.15.44:0/202370730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8c50 msgr2=0x558661eb00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8c50 0x558661eb00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.798+0000 7fa031d9b640 1 -- 172.21.15.44:0/202370730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa028013520 con 0x558661ea8330 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa031d9b640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 0x558661eb2e30 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa028005890 tx=0x7fa0280116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1f17c80028f7c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa019ffb640 1 -- 172.21.15.44:0/202370730 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028011e30 con 0x558661ea8330 2024-09-13T17:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa019ffb640 1 -- 172.21.15.44:0/202370730 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa028005030 con 0x558661ea8330 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558661e35680 con 0x558661ea8330 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa019ffb640 1 -- 172.21.15.44:0/202370730 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0280051f0 con 0x558661ea8330 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 10 client.18975 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558661e0b330 con 0x558661ea8330 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa019ffb640 1 -- 172.21.15.44:0/202370730 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558661e35680 con 0x558661ea8330 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa019ffb640 1 -- 172.21.15.44:0/202370730 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa02800b6a0 con 0x558661ea8330 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 10 client.18975 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 10 client.18975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 20 client.18975 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661ec2dc0 0x558661ec51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 4 client.18975 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.799+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558661e005d0 con 0x558661ec2dc0 2024-09-13T17:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.800+0000 7fa03259c640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661ec2dc0 0x558661ec51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.800+0000 7fa03259c640 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661ec2dc0 0x558661ec51e0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1decd36256a0e252 server_cookie=dea8c9e1bc1c7636 in_seq=0 out_seq=0 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.800+0000 7fa019ffb640 10 client.18975 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa019ffb640 1 -- 172.21.15.44:0/202370730 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558661e005d0 con 0x558661ec2dc0 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa019ffb640 10 client.18975 handle_command_reply: tid=0 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 client.18975 shutdown 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 msgr2=0x558661eb2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 0x558661eb2e30 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa028005890 tx=0x7fa0280116c0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 shutdown_connections 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661ec2dc0 0x558661ec51e0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661eb34f0 0x558661eb0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8c50 0x558661eb00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa019ffb640 0 client.18975 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 --2- 172.21.15.44:0/202370730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661ea8330 0x558661eb2e30 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 >> 172.21.15.44:0/202370730 conn(0x558661ea48c0 msgr2=0x558661ea3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 shutdown_connections 2024-09-13T17:37:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 1 -- 172.21.15.44:0/202370730 wait complete. 2024-09-13T17:37:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.801+0000 7fa03bad6740 20 client.18975 trim_cache size 0 max 16384 2024-09-13T17:37:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 Processor -- start 2024-09-13T17:37:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 -- start start 2024-09-13T17:37:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661ea5550 con 0x558661eb34f0 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661eb5fa0 con 0x558661ea8330 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa03bad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661eae5e0 con 0x558661ea8c50 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.814+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37826/0 (socket says 172.21.15.44:37826) 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 -- 172.21.15.44:0/286223997 learned_addr learned my addr 172.21.15.44:0/286223997 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa03259c640 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 -- 172.21.15.44:0/286223997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 msgr2=0x558661eba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 -- 172.21.15.44:0/286223997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 msgr2=0x558661eb2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 -- 172.21.15.44:0/286223997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558661dad300 con 0x558661eb34f0 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa03259c640 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa031d9b640 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eba730 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa028011790 tx=0x7fa02800ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb87ce3f13cf9ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa0217fa640 1 -- 172.21.15.44:0/286223997 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800b870 con 0x558661eb34f0 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa0217fa640 1 -- 172.21.15.44:0/286223997 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa028005bf0 con 0x558661eb34f0 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.815+0000 7fa0217fa640 1 -- 172.21.15.44:0/286223997 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028005020 con 0x558661eb34f0 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- 172.21.15.44:0/286223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 msgr2=0x558661eba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eba730 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa028011790 tx=0x7fa02800ebe0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- 172.21.15.44:0/286223997 shutdown_connections 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eba730 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- 172.21.15.44:0/286223997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- 172.21.15.44:0/286223997 >> 172.21.15.44:0/286223997 conn(0x558661ea48c0 msgr2=0x558661eab370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- 172.21.15.44:0/286223997 shutdown_connections 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- 172.21.15.44:0/286223997 wait complete. 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- start start 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 Processor -- start 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661ebab20 con 0x558661eb34f0 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f19290 con 0x558661ea8330 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.816+0000 7fa03bad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f193d0 con 0x558661ea8c50 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa03bad6740 10 client.0 _ll_register_callbacks cb 0x7fa02800e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:60126/0 (socket says 172.21.15.44:60126) 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 -- 172.21.15.44:0/4165051612 learned_addr learned my addr 172.21.15.44:0/4165051612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa03259c640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 -- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 msgr2=0x558661eb9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 -- 172.21.15.44:0/4165051612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 msgr2=0x558661eb96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 -- 172.21.15.44:0/4165051612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558661e0b330 con 0x558661ea8330 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa03259c640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa031d9b640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb8c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa02800bab0 tx=0x7fa028005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=816dc8c9177d3568 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.817+0000 7fa019ffb640 1 -- 172.21.15.44:0/4165051612 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800ed10 con 0x558661ea8330 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa028019070 con 0x558661ea8330 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa019ffb640 1 -- 172.21.15.44:0/4165051612 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa028000e40 con 0x558661ea8330 2024-09-13T17:37:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa019ffb640 1 -- 172.21.15.44:0/4165051612 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa028005030 con 0x558661ea8330 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 10 client.27998 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558661dd28e0 con 0x558661ea8330 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa019ffb640 1 -- 172.21.15.44:0/4165051612 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa028019070 con 0x558661ea8330 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa019ffb640 1 -- 172.21.15.44:0/4165051612 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa028004680 con 0x558661ea8330 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 10 client.27998 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 10 client.27998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 20 client.27998 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661f1a600 0x558661f1c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 4 client.27998 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558661b00300 con 0x558661f1a600 2024-09-13T17:37:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.818+0000 7fa03259c640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661f1a600 0x558661f1c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.819+0000 7fa019ffb640 10 client.27998 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.819+0000 7fa03259c640 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661f1a600 0x558661f1c9f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916aac119ee51243 server_cookie=c119723456cef597 in_seq=0 out_seq=0 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa019ffb640 1 -- 172.21.15.44:0/4165051612 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558661b00300 con 0x558661f1a600 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa019ffb640 10 client.27998 handle_command_reply: tid=0 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 client.27998 shutdown 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 msgr2=0x558661eb8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb8c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa02800bab0 tx=0x7fa028005900 comp rx=0 tx=0).stop 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 shutdown_connections 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558661f1a600 0x558661f1c9f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558661ea8c50 0x558661eb9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558661eb34f0 0x558661eb96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa019ffb640 0 client.27998 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 --2- 172.21.15.44:0/4165051612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558661ea8330 0x558661eb8c20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 >> 172.21.15.44:0/4165051612 conn(0x558661ea48c0 msgr2=0x558661e9fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 shutdown_connections 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.832+0000 7fa03bad6740 1 -- 172.21.15.44:0/4165051612 wait complete. 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa03bad6740 20 client.27998 trim_cache size 0 max 16384 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:37:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:37:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa01c0a1200 msgr2=0x7fa01c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa01c0a1200 0x7fa01c0a36c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0eb9a0 tx=0x7fa02c0a6040 comp rx=0 tx=0).stop 2024-09-13T17:37:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 msgr2=0x7fa03419afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 0x7fa03419afa0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa02c004af0 tx=0x7fa02c030ec0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 shutdown_connections 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa01c0a1200 0x7fa01c0a36c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa03419aaf0 0x7fa03419afa0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa034105040 0x7fa034196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 --2- 172.21.15.44:0/1519187681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0340ffef0 0x7fa0341961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 >> 172.21.15.44:0/1519187681 conn(0x7fa0340fb960 msgr2=0x7fa0340fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 shutdown_connections 2024-09-13T17:37:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:36.833+0000 7fa038db5640 1 -- 172.21.15.44:0/1519187681 wait complete. 2024-09-13T17:37:37.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 -- 172.21.15.44:0/2251543865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c00fee80 msgr2=0x7f10c00ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 --2- 172.21.15.44:0/2251543865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c00fee80 0x7f10c00ff260 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009930 tx=0x7f10b002a420 comp rx=0 tx=0).stop 2024-09-13T17:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 -- 172.21.15.44:0/2251543865 shutdown_connections 2024-09-13T17:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 --2- 172.21.15.44:0/2251543865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c00ff830 0x7f10c0103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 --2- 172.21.15.44:0/2251543865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c0103fd0 0x7f10c010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 --2- 172.21.15.44:0/2251543865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c00fee80 0x7f10c00ff260 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.894+0000 7f10c5733640 1 -- 172.21.15.44:0/2251543865 >> 172.21.15.44:0/2251543865 conn(0x7f10c00fa930 msgr2=0x7f10c00fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 -- 172.21.15.44:0/2251543865 shutdown_connections 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 -- 172.21.15.44:0/2251543865 wait complete. 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 Processor -- start 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 -- start start 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00fee80 0x7f10c01970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c0103fd0 0x7f10c0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 0x7f10c01a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c01a2800 con 0x7f10c0197b60 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c01a2970 con 0x7f10c00fee80 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.895+0000 7f10c5733640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c01a2b30 con 0x7f10c0103fd0 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 0x7f10c01a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 0x7f10c01a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37850/0 (socket says 172.21.15.44:37850) 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 -- 172.21.15.44:0/17026277 learned_addr learned my addr 172.21.15.44:0/17026277 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10beffd640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00fee80 0x7f10c01970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10b7fff640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c0103fd0 0x7f10c0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 -- 172.21.15.44:0/17026277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c0103fd0 msgr2=0x7f10c0197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c0103fd0 0x7f10c0197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 -- 172.21.15.44:0/17026277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00fee80 msgr2=0x7f10c01970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00fee80 0x7f10c01970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 -- 172.21.15.44:0/17026277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009560 con 0x7f10c0197b60 2024-09-13T17:37:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10b7fff640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c0103fd0 0x7f10c0197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bf7fe640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 0x7f10c01a2160 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f10b800eb10 tx=0x7f10b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bcff9640 1 -- 172.21.15.44:0/17026277 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8004b20 con 0x7f10c0197b60 2024-09-13T17:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bcff9640 1 -- 172.21.15.44:0/17026277 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10b8004c80 con 0x7f10c0197b60 2024-09-13T17:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c01a2e10 con 0x7f10c0197b60 2024-09-13T17:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10bcff9640 1 -- 172.21.15.44:0/17026277 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8005a10 con 0x7f10c0197b60 2024-09-13T17:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.896+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c01a32f0 con 0x7f10c0197b60 2024-09-13T17:37:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10bcff9640 1 -- 172.21.15.44:0/17026277 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f10b8005020 con 0x7f10c0197b60 2024-09-13T17:37:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 Processor -- start 2024-09-13T17:37:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 -- start start 2024-09-13T17:37:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412220 0x55a590412600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a59041cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a59041d3e0 0x55a59041f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a590378f20 con 0x55a590412b40 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5903766b0 con 0x55a59041d3e0 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.899+0000 7f10c845b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a590371fa0 con 0x55a590412220 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a59041cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a59041cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37858/0 (socket says 172.21.15.44:37858) 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 -- 172.21.15.44:0/1189945610 learned_addr learned my addr 172.21.15.44:0/1189945610 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10bcff9640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10ac0a1280 0x7f10ac0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412220 0x55a590412600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a59041d3e0 0x55a59041f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10bf7fe640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10ac0a1280 0x7f10ac0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 -- 172.21.15.44:0/1189945610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412220 msgr2=0x55a590412600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412220 0x55a590412600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 -- 172.21.15.44:0/1189945610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a59041d3e0 msgr2=0x55a59041f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a59041d3e0 0x55a59041f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 -- 172.21.15.44:0/1189945610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a59032c840 con 0x55a590412b40 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412220 0x55a590412600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10bcff9640 1 -- 172.21.15.44:0/17026277 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f10b8014070 con 0x7f10c0197b60 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10beffd640 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a59041cea0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f10b002aa10 tx=0x7f10b0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfdb53c55181e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.900+0000 7f10bf7fe640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10ac0a1280 0x7f10ac0a3740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10b809a2f0 tx=0x7f10b8069620 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10b57fa640 1 -- 172.21.15.44:0/1189945610 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00044f0 con 0x55a590412b40 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10b57fa640 1 -- 172.21.15.44:0/1189945610 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10b0003a80 con 0x55a590412b40 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10b57fa640 1 -- 172.21.15.44:0/1189945610 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0046a50 con 0x55a590412b40 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 -- 172.21.15.44:0/1189945610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 msgr2=0x55a59041cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a59041cea0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f10b002aa10 tx=0x7f10b0004040 comp rx=0 tx=0).stop 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 -- 172.21.15.44:0/1189945610 shutdown_connections 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a59041d3e0 0x55a59041f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a59041cea0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 --2- 172.21.15.44:0/1189945610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412220 0x55a590412600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 -- 172.21.15.44:0/1189945610 >> 172.21.15.44:0/1189945610 conn(0x55a59040e790 msgr2=0x55a59040eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 -- 172.21.15.44:0/1189945610 shutdown_connections 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.901+0000 7f10c845b740 1 -- 172.21.15.44:0/1189945610 wait complete. 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 -- start start 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 Processor -- start 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412b40 0x55a59041a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 0x55a59041a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5903132a0 con 0x55a59041d3e0 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59004a430 con 0x55a590412220 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5902f5680 con 0x55a590412b40 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10c845b740 10 client.0 _ll_register_callbacks cb 0x55a59036b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 0x55a59041a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 0x55a59041a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37870/0 (socket says 172.21.15.44:37870) 2024-09-13T17:37:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10b7fff640 1 -- 172.21.15.44:0/1132592996 learned_addr learned my addr 172.21.15.44:0/1132592996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.902+0000 7f10beffd640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412b40 0x55a59041a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10b7fff640 1 -- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412b40 msgr2=0x55a59041a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412b40 0x55a59041a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10b7fff640 1 -- 172.21.15.44:0/1132592996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 msgr2=0x55a59041ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10b7fff640 1 -- 172.21.15.44:0/1132592996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c010c360 con 0x55a59041d3e0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 0x55a59041a850 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f10c0069730 tx=0x7f10a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691ae9462727f44a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10a67fc640 1 -- 172.21.15.44:0/1132592996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800fe60 con 0x55a59041d3e0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10a67fc640 1 -- 172.21.15.44:0/1132592996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f10a8011440 con 0x55a59041d3e0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5900654e0 con 0x55a59041d3e0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10a67fc640 1 -- 172.21.15.44:0/1132592996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a8011600 con 0x55a59041d3e0 2024-09-13T17:37:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10c845b740 10 client.18993 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a590370620 con 0x55a59041d3e0 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10a67fc640 1 -- 172.21.15.44:0/1132592996 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a5900654e0 con 0x55a59041d3e0 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.903+0000 7f10a67fc640 1 -- 172.21.15.44:0/1132592996 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10a8020600 con 0x55a59041d3e0 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10c845b740 10 client.18993 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10c845b740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10c845b740 20 client.18993 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10c845b740 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a59042d230 0x55a59042f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10c845b740 4 client.18993 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5900767e0 con 0x55a59042d230 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10beffd640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a59042d230 0x55a59042f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10a67fc640 10 client.18993 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.904+0000 7f10beffd640 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a59042d230 0x55a59042f650 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e17579fa498a1f5 server_cookie=dff74b18c8ecf673 in_seq=0 out_seq=0 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10a67fc640 1 -- 172.21.15.44:0/1132592996 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a5900767e0 con 0x55a59042d230 2024-09-13T17:37:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10a67fc640 10 client.18993 handle_command_reply: tid=0 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 client.18993 shutdown 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 msgr2=0x55a59041a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 0x55a59041a850 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f10c0069730 tx=0x7f10a80046c0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 shutdown_connections 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a59042d230 0x55a59042f650 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a590412b40 0x55a59041a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a59041d3e0 0x55a59041a850 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 --2- 172.21.15.44:0/1132592996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10a67fc640 0 client.18993 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 >> 172.21.15.44:0/1132592996 conn(0x55a59040e790 msgr2=0x55a590409510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 shutdown_connections 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 1 -- 172.21.15.44:0/1132592996 wait complete. 2024-09-13T17:37:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.905+0000 7f10c845b740 20 client.18993 trim_cache size 0 max 16384 2024-09-13T17:37:37.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.919+0000 7f10c845b740 1 Processor -- start 2024-09-13T17:37:37.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 -- start start 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a590433fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590415360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59041b050 con 0x55a590412b40 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a590429090 con 0x55a590412220 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10c845b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5904131e0 con 0x55a59041d3e0 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a590433fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a590433fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37876/0 (socket says 172.21.15.44:37876) 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 -- 172.21.15.44:0/566118893 learned_addr learned my addr 172.21.15.44:0/566118893 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:37.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10b7fff640 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590415360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10b7fff640 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 -- 172.21.15.44:0/566118893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 msgr2=0x55a590415360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590415360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 -- 172.21.15.44:0/566118893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 msgr2=0x55a59041f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.920+0000 7f10beffd640 1 -- 172.21.15.44:0/566118893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a590312790 con 0x55a590412b40 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10beffd640 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a590433fb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f10b0005790 tx=0x7f10b0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355045357c9f1bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10b5ffb640 1 -- 172.21.15.44:0/566118893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0003df0 con 0x55a590412b40 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10b5ffb640 1 -- 172.21.15.44:0/566118893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10b004b040 con 0x55a590412b40 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10b5ffb640 1 -- 172.21.15.44:0/566118893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0046cf0 con 0x55a590412b40 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10c845b740 1 -- 172.21.15.44:0/566118893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 msgr2=0x55a590433fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10c845b740 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a590433fb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f10b0005790 tx=0x7f10b0005720 comp rx=0 tx=0).stop 2024-09-13T17:37:37.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10c845b740 1 -- 172.21.15.44:0/566118893 shutdown_connections 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10c845b740 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590415360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10c845b740 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412b40 0x55a590433fb0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.921+0000 7f10c845b740 1 --2- 172.21.15.44:0/566118893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412220 0x55a59041f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- 172.21.15.44:0/566118893 >> 172.21.15.44:0/566118893 conn(0x55a59040e790 msgr2=0x55a5904133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- 172.21.15.44:0/566118893 shutdown_connections 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- 172.21.15.44:0/566118893 wait complete. 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- start start 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 Processor -- start 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 0x55a5904242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412b40 0x55a590420710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590420c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a590421190 con 0x55a590412220 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a590421300 con 0x55a590412b40 2024-09-13T17:37:37.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5904214c0 con 0x55a59041d3e0 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10c845b740 10 client.0 _ll_register_callbacks cb 0x7f10a8009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 0x55a5904242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 0x55a5904242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37878/0 (socket says 172.21.15.44:37878) 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.922+0000 7f10b7fff640 1 -- 172.21.15.44:0/1649125558 learned_addr learned my addr 172.21.15.44:0/1649125558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10beffd640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412b40 0x55a590420710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10b7fff640 1 -- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 msgr2=0x55a590420c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590420c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10b7fff640 1 -- 172.21.15.44:0/1649125558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412b40 msgr2=0x55a590420710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412b40 0x55a590420710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10b7fff640 1 -- 172.21.15.44:0/1649125558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a59032c840 con 0x55a590412220 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10beffd640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412b40 0x55a590420710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10b7fff640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 0x55a5904242f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f10a8020a10 tx=0x7f10a8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed05f2e28f3d465 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10a67fc640 1 -- 172.21.15.44:0/1649125558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8009530 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10a67fc640 1 -- 172.21.15.44:0/1649125558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f10a8000e40 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10b0003a80 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10a67fc640 1 -- 172.21.15.44:0/1649125558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a8005c50 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10c845b740 10 client.18999 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a590337ea0 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.923+0000 7f10a67fc640 1 -- 172.21.15.44:0/1649125558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10b0003a80 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10a67fc640 1 -- 172.21.15.44:0/1649125558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10a8004250 con 0x55a590412220 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10c845b740 10 client.18999 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10c845b740 10 client.18999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10c845b740 20 client.18999 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10c845b740 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a590484a30 0x55a590486e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10c845b740 4 client.18999 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a5900c9e30 con 0x55a590484a30 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10beffd640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a590484a30 0x55a590486e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10a67fc640 10 client.18999 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.924+0000 7f10beffd640 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a590484a30 0x55a590486e50 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c97dd57fa66a5d server_cookie=dac3aebc5efe68e0 in_seq=0 out_seq=0 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10a67fc640 1 -- 172.21.15.44:0/1649125558 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a5900c9e30 con 0x55a590484a30 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10a67fc640 10 client.18999 handle_command_reply: tid=0 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 client.18999 shutdown 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 msgr2=0x55a5904242f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 0x55a5904242f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f10a8020a10 tx=0x7f10a8009ea0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 shutdown_connections 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55a590484a30 0x55a590486e50 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a59041d3e0 0x55a590420c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a590412b40 0x55a590420710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10a67fc640 0 client.18999 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 --2- 172.21.15.44:0/1649125558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a590412220 0x55a5904242f0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 >> 172.21.15.44:0/1649125558 conn(0x55a59040e790 msgr2=0x55a590415360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 shutdown_connections 2024-09-13T17:37:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 1 -- 172.21.15.44:0/1649125558 wait complete. 2024-09-13T17:37:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c845b740 20 client.18999 trim_cache size 0 max 16384 2024-09-13T17:37:37.933 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:37:37.933 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:37:37.933 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:37:37.933 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10ac0a1280 msgr2=0x7f10ac0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10ac0a1280 0x7f10ac0a3740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10b809a2f0 tx=0x7f10b8069620 comp rx=0 tx=0).stop 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 msgr2=0x7f10c01a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 0x7f10c01a2160 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f10b800eb10 tx=0x7f10b800c2d0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 shutdown_connections 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f10ac0a1280 0x7f10ac0a3740 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c0103fd0 0x7f10c0197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c0197b60 0x7f10c01a2160 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 --2- 172.21.15.44:0/17026277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00fee80 0x7f10c01970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.925+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 >> 172.21.15.44:0/17026277 conn(0x7f10c00fa930 msgr2=0x7f10c00fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:37.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.926+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 shutdown_connections 2024-09-13T17:37:37.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:37.926+0000 7f10c5733640 1 -- 172.21.15.44:0/17026277 wait complete. 2024-09-13T17:37:37.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:37 smithi044 ceph-mon[31175]: pgmap v340: 129 pgs: 129 active+clean; 5.2 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:37.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:37:37.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:37:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:37 smithi070 ceph-mon[34502]: pgmap v340: 129 pgs: 129 active+clean; 5.2 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:38.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:37 smithi100 ceph-mon[35055]: pgmap v340: 129 pgs: 129 active+clean; 5.2 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 -- 172.21.15.44:0/1403925420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb700fff90 msgr2=0x7fcb70100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1403925420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb700fff90 0x7fcb70100370 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009930 tx=0x7fcb6002a420 comp rx=0 tx=0).stop 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 -- 172.21.15.44:0/1403925420 shutdown_connections 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1403925420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb70100940 0x7fcb70104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1403925420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb701050e0 0x7fcb7010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1403925420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb700fff90 0x7fcb70100370 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 -- 172.21.15.44:0/1403925420 >> 172.21.15.44:0/1403925420 conn(0x7fcb700fba20 msgr2=0x7fcb700fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 -- 172.21.15.44:0/1403925420 shutdown_connections 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.040+0000 7fcb768f7640 1 -- 172.21.15.44:0/1403925420 wait complete. 2024-09-13T17:37:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 Processor -- start 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 -- start start 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb700fff90 0x7fcb70196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb701050e0 0x7fcb701967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 0x7fcb7019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019b580 con 0x7fcb7019ab90 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019b6f0 con 0x7fcb700fff90 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb768f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019b860 con 0x7fcb701050e0 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 0x7fcb7019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 0x7fcb7019b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37886/0 (socket says 172.21.15.44:37886) 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 -- 172.21.15.44:0/1682049667 learned_addr learned my addr 172.21.15.44:0/1682049667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb700fff90 0x7fcb70196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb6f7fe640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb701050e0 0x7fcb701967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 -- 172.21.15.44:0/1682049667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb701050e0 msgr2=0x7fcb701967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb701050e0 0x7fcb701967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 -- 172.21.15.44:0/1682049667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb700fff90 msgr2=0x7fcb70196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb700fff90 0x7fcb70196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb74e94640 1 -- 172.21.15.44:0/1682049667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009560 con 0x7fcb7019ab90 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.041+0000 7fcb6f7fe640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb701050e0 0x7fcb701967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.042+0000 7fcb74e94640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 0x7fcb7019b040 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800eb10 tx=0x7fcb6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.042+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68004b20 con 0x7fcb7019ab90 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.042+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb68004c80 con 0x7fcb7019ab90 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.042+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb701a2e40 con 0x7fcb7019ab90 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.042+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68005a10 con 0x7fcb7019ab90 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.042+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb701a32c0 con 0x7fcb7019ab90 2024-09-13T17:37:38.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.043+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70101460 con 0x7fcb7019ab90 2024-09-13T17:37:38.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.044+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb68004fd0 con 0x7fcb7019ab90 2024-09-13T17:37:38.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.045+0000 7fcb6d7fa640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb580a1280 0x7fcb580a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.045+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fcb68014070 con 0x7fcb7019ab90 2024-09-13T17:37:38.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.045+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb580a1280 0x7fcb580a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.045+0000 7fcb6ffff640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb580a1280 0x7fcb580a3740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb600057c0 tx=0x7fcb60005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.048+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcb68096860 con 0x7fcb7019ab90 2024-09-13T17:37:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.257+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb70100370 con 0x7fcb7019ab90 2024-09-13T17:37:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.258+0000 7fcb6d7fa640 1 -- 172.21.15.44:0/1682049667 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7fcb68099e80 con 0x7fcb7019ab90 2024-09-13T17:37:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:37:38.259 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:37:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:37:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb580a1280 msgr2=0x7fcb580a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb580a1280 0x7fcb580a3740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb600057c0 tx=0x7fcb60005a20 comp rx=0 tx=0).stop 2024-09-13T17:37:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 msgr2=0x7fcb7019b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 0x7fcb7019b040 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800eb10 tx=0x7fcb6800c2d0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 shutdown_connections 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fcb580a1280 0x7fcb580a3740 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb701050e0 0x7fcb701967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb7019ab90 0x7fcb7019b040 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 --2- 172.21.15.44:0/1682049667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb700fff90 0x7fcb70196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 >> 172.21.15.44:0/1682049667 conn(0x7fcb700fba20 msgr2=0x7fcb700fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 shutdown_connections 2024-09-13T17:37:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.260+0000 7fcb768f7640 1 -- 172.21.15.44:0/1682049667 wait complete. 2024-09-13T17:37:38.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:37:38.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 -- 172.21.15.44:0/3496353344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360410a350 msgr2=0x7f360410c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 --2- 172.21.15.44:0/3496353344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360410a350 0x7f360410c7a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009ca0 tx=0x7f35fc02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 -- 172.21.15.44:0/3496353344 shutdown_connections 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 --2- 172.21.15.44:0/3496353344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360410a350 0x7f360410c7a0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 --2- 172.21.15.44:0/3496353344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3604101800 0x7f3604109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 --2- 172.21.15.44:0/3496353344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3604100e50 0x7f3604101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 -- 172.21.15.44:0/3496353344 >> 172.21.15.44:0/3496353344 conn(0x7f36040fa930 msgr2=0x7f36040fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 -- 172.21.15.44:0/3496353344 shutdown_connections 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.376+0000 7f360bf53640 1 -- 172.21.15.44:0/3496353344 wait complete. 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 Processor -- start 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 -- start start 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3604100e50 0x7f3604196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 0x7f36041966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f360410a350 0x7f360419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360419b050 con 0x7f3604101800 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360419b1c0 con 0x7f3604100e50 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360bf53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360419b380 con 0x7f360410a350 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 0x7f36041966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 0x7f36041966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37904/0 (socket says 172.21.15.44:37904) 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 -- 172.21.15.44:0/3723706331 learned_addr learned my addr 172.21.15.44:0/3723706331 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f360a4f0640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f360410a350 0x7f360419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f3609cef640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3604100e50 0x7f3604196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 -- 172.21.15.44:0/3723706331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f360410a350 msgr2=0x7f360419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f360410a350 0x7f360419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 -- 172.21.15.44:0/3723706331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3604100e50 msgr2=0x7f3604196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3604100e50 0x7f3604196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 -- 172.21.15.44:0/3723706331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc009900 con 0x7f3604101800 2024-09-13T17:37:38.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.377+0000 7f36094ee640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 0x7f36041966b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f35f400d950 tx=0x7f35f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.378+0000 7f35faffd640 1 -- 172.21.15.44:0/3723706331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4009a30 con 0x7f3604101800 2024-09-13T17:37:38.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.378+0000 7f35faffd640 1 -- 172.21.15.44:0/3723706331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35f40044b0 con 0x7f3604101800 2024-09-13T17:37:38.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.378+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360419b660 con 0x7f3604101800 2024-09-13T17:37:38.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.378+0000 7f35faffd640 1 -- 172.21.15.44:0/3723706331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40047a0 con 0x7f3604101800 2024-09-13T17:37:38.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.378+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36041a2ee0 con 0x7f3604101800 2024-09-13T17:37:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.380+0000 7f35faffd640 1 -- 172.21.15.44:0/3723706331 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f35f4004020 con 0x7f3604101800 2024-09-13T17:37:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.380+0000 7f360ec74740 1 Processor -- start 2024-09-13T17:37:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 -- start start 2024-09-13T17:37:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d900a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 0x560d8d90f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d90fd60 0x560d8d912150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f35faffd640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d86acc0 con 0x560d8d900f90 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d868450 con 0x560d8d900670 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360ec74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d863d40 con 0x560d8d90fd60 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 0x560d8d90f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 0x560d8d90f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37908/0 (socket says 172.21.15.44:37908) 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 -- 172.21.15.44:0/338508227 learned_addr learned my addr 172.21.15.44:0/338508227 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f36094ee640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360a4f0640 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d90fd60 0x560d8d912150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f360a4f0640 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d900a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f35faffd640 1 -- 172.21.15.44:0/3723706331 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f35f40cca50 con 0x7f3604101800 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 -- 172.21.15.44:0/338508227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d90fd60 msgr2=0x560d8d912150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d90fd60 0x560d8d912150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 -- 172.21.15.44:0/338508227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 msgr2=0x560d8d900a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d900a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 -- 172.21.15.44:0/338508227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d8d82b620 con 0x560d8d900f90 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f36094ee640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35f4095c90 tx=0x7f35f4095430 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.381+0000 7f3609cef640 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 0x560d8d90f820 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3600009cc0 tx=0x7f36000117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb8436c97c6b0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f35effff640 1 -- 172.21.15.44:0/338508227 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600009760 con 0x560d8d900f90 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f35effff640 1 -- 172.21.15.44:0/338508227 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f360001b900 con 0x560d8d900f90 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f35effff640 1 -- 172.21.15.44:0/338508227 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000129b0 con 0x560d8d900f90 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 -- 172.21.15.44:0/338508227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 msgr2=0x560d8d90f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 0x560d8d90f820 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3600009cc0 tx=0x7f36000117b0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 -- 172.21.15.44:0/338508227 shutdown_connections 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d90fd60 0x560d8d912150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d900f90 0x560d8d90f820 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 --2- 172.21.15.44:0/338508227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d900a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.382+0000 7f360ec74740 1 -- 172.21.15.44:0/338508227 >> 172.21.15.44:0/338508227 conn(0x560d8d8fcd00 msgr2=0x560d8d8fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 -- 172.21.15.44:0/338508227 shutdown_connections 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 -- 172.21.15.44:0/338508227 wait complete. 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 -- start start 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 Processor -- start 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d90f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d90c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d822180 con 0x560d8d90fd60 2024-09-13T17:37:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d53c430 con 0x560d8d900670 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d7e75e0 con 0x560d8d900f90 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360ec74740 10 client.0 _ll_register_callbacks cb 0x560d8d853aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.383+0000 7f360a4f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f3609cef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d90c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37922/0 (socket says 172.21.15.44:37922) 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 -- 172.21.15.44:0/2898093481 learned_addr learned my addr 172.21.15.44:0/2898093481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d90f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 -- 172.21.15.44:0/2898093481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 msgr2=0x560d8d90c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d90c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 -- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 msgr2=0x560d8d90f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d90f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 -- 172.21.15.44:0/2898093481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36000098c0 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90cd90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f35fc02ade0 tx=0x7f35fc031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe98b499bcc4580 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f35ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f35ed7fa640 1 -- 172.21.15.44:0/2898093481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0305c0 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f35ed7fa640 1 -- 172.21.15.44:0/2898093481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f35fc030720 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d8d88db80 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f35ed7fa640 1 -- 172.21.15.44:0/2898093481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35fc0308e0 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360ec74740 10 client.19017 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.384+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560d8d8623c0 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f35ed7fa640 1 -- 172.21.15.44:0/2898093481 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560d8d88db80 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f35ed7fa640 1 -- 172.21.15.44:0/2898093481 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35fc02fce0 con 0x560d8d90fd60 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f360ec74740 10 client.19017 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f360ec74740 10 client.19017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:37:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f360ec74740 20 client.19017 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f360ec74740 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d91f780 0x560d8d921b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f360ec74740 4 client.19017 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d8d8561f0 con 0x560d8d91f780 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.385+0000 7f3609cef640 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d91f780 0x560d8d921b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f35ed7fa640 10 client.19017 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f3609cef640 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d91f780 0x560d8d921b70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de56301fdf8dbdcd server_cookie=e6d382abf50588a0 in_seq=0 out_seq=0 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f35ed7fa640 1 -- 172.21.15.44:0/2898093481 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560d8d8561f0 con 0x560d8d91f780 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f35ed7fa640 10 client.19017 handle_command_reply: tid=0 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 client.19017 shutdown 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 msgr2=0x560d8d90cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90cd90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f35fc02ade0 tx=0x7f35fc031000 comp rx=0 tx=0).stop 2024-09-13T17:37:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 shutdown_connections 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d91f780 0x560d8d921b70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d90c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90cd90 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 --2- 172.21.15.44:0/2898093481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d90f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f35ed7fa640 0 client.19017 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 >> 172.21.15.44:0/2898093481 conn(0x560d8d8fcd00 msgr2=0x560d8d902a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.386+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 shutdown_connections 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.387+0000 7f360ec74740 1 -- 172.21.15.44:0/2898093481 wait complete. 2024-09-13T17:37:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.387+0000 7f360ec74740 20 client.19017 trim_cache size 0 max 16384 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 Processor -- start 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 -- start start 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d8fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d8fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d90f9d0 con 0x560d8d90fd60 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d912480 con 0x560d8d900670 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360ec74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d926d10 con 0x560d8d900f90 2024-09-13T17:37:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360a4f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360a4f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37932/0 (socket says 172.21.15.44:37932) 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360a4f0640 1 -- 172.21.15.44:0/2525109221 learned_addr learned my addr 172.21.15.44:0/2525109221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.398+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d8fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f3609cef640 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d8fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360a4f0640 1 -- 172.21.15.44:0/2525109221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 msgr2=0x560d8d8fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d8fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360a4f0640 1 -- 172.21.15.44:0/2525109221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 msgr2=0x560d8d8fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d8fd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360a4f0640 1 -- 172.21.15.44:0/2525109221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d8d804770 con 0x560d8d90fd60 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360a4f0640 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90b0e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009bf0 tx=0x7f35fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5c6de99efa0694 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f35f8ff9640 1 -- 172.21.15.44:0/2525109221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc005bd0 con 0x560d8d90fd60 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f35f8ff9640 1 -- 172.21.15.44:0/2525109221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35fc044070 con 0x560d8d90fd60 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f35f8ff9640 1 -- 172.21.15.44:0/2525109221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc030730 con 0x560d8d90fd60 2024-09-13T17:37:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360ec74740 1 -- 172.21.15.44:0/2525109221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 msgr2=0x560d8d90b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.399+0000 7f360ec74740 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90b0e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009bf0 tx=0x7f35fc02a8c0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- 172.21.15.44:0/2525109221 shutdown_connections 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900f90 0x560d8d8fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d90b0e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 --2- 172.21.15.44:0/2525109221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900670 0x560d8d8fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- 172.21.15.44:0/2525109221 >> 172.21.15.44:0/2525109221 conn(0x560d8d8fcd00 msgr2=0x560d8d91b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- 172.21.15.44:0/2525109221 shutdown_connections 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- 172.21.15.44:0/2525109221 wait complete. 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- start start 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 Processor -- start 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900670 0x560d8d916a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900f90 0x560d8d916f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d913250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d913950 con 0x560d8d90fd60 2024-09-13T17:37:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d913ac0 con 0x560d8d900f90 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8d913c80 con 0x560d8d900670 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.400+0000 7f360ec74740 10 client.0 _ll_register_callbacks cb 0x7f36000096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d913250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d913250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37944/0 (socket says 172.21.15.44:37944) 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 -- 172.21.15.44:0/4150827883 learned_addr learned my addr 172.21.15.44:0/4150827883 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f3609cef640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900f90 0x560d8d916f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900670 0x560d8d916a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 -- 172.21.15.44:0/4150827883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900670 msgr2=0x560d8d916a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900670 0x560d8d916a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 -- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900f90 msgr2=0x560d8d916f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900f90 0x560d8d916f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 -- 172.21.15.44:0/4150827883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d8d8623c0 con 0x560d8d90fd60 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360a4f0640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d913250 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f35fc04f410 tx=0x7f35fc064040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892e491456c8b409 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f35ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f35ed7fa640 1 -- 172.21.15.44:0/4150827883 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc067070 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f35ed7fa640 1 -- 172.21.15.44:0/4150827883 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f35fc031ab0 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f360001b900 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f35ed7fa640 1 -- 172.21.15.44:0/4150827883 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35fc031c70 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360ec74740 10 client.19023 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.401+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560d8d829e80 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f35ed7fa640 1 -- 172.21.15.44:0/4150827883 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f360001b900 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f35ed7fa640 1 -- 172.21.15.44:0/4150827883 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35fc06a020 con 0x560d8d90fd60 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f360ec74740 10 client.19023 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f360ec74740 10 client.19023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f360ec74740 20 client.19023 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f360ec74740 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d976b70 0x560d8d978f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f360ec74740 4 client.19023 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d8d5574e0 con 0x560d8d976b70 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f3609cef640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d976b70 0x560d8d978f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f35ed7fa640 10 client.19023 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.402+0000 7f3609cef640 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d976b70 0x560d8d978f60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35e1e17f29b027ed server_cookie=f065668ce0cd947b in_seq=0 out_seq=0 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f35ed7fa640 1 -- 172.21.15.44:0/4150827883 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d8d5574e0 con 0x560d8d976b70 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f35ed7fa640 10 client.19023 handle_command_reply: tid=0 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 client.19023 shutdown 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 msgr2=0x560d8d913250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d913250 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f35fc04f410 tx=0x7f35fc064040 comp rx=0 tx=0).stop 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 shutdown_connections 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x560d8d976b70 0x560d8d978f60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d8d90fd60 0x560d8d913250 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d8d900f90 0x560d8d916f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 --2- 172.21.15.44:0/4150827883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d8d900670 0x560d8d916a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f35ed7fa640 0 client.19023 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:37:38.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 >> 172.21.15.44:0/4150827883 conn(0x560d8d8fcd00 msgr2=0x560d8d91b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 shutdown_connections 2024-09-13T17:37:38.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 1 -- 172.21.15.44:0/4150827883 wait complete. 2024-09-13T17:37:38.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360ec74740 20 client.19023 trim_cache size 0 max 16384 2024-09-13T17:37:38.411 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 msgr2=0x7f35f00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35f4095c90 tx=0x7f35f4095430 comp rx=0 tx=0).stop 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 msgr2=0x7f36041966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 0x7f36041966b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f35f400d950 tx=0x7f35f400de10 comp rx=0 tx=0).stop 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 shutdown_connections 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f35f00a1200 0x7f35f00a36c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f360410a350 0x7f360419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3604101800 0x7f36041966b0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 --2- 172.21.15.44:0/3723706331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3604100e50 0x7f3604196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.403+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 >> 172.21.15.44:0/3723706331 conn(0x7f36040fa930 msgr2=0x7f36040fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.404+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 shutdown_connections 2024-09-13T17:37:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.404+0000 7f360bf53640 1 -- 172.21.15.44:0/3723706331 wait complete. 2024-09-13T17:37:38.415 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:37:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 -- 172.21.15.44:0/3926881511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c0fff90 msgr2=0x7f960c100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 --2- 172.21.15.44:0/3926881511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c0fff90 0x7f960c100370 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009930 tx=0x7f95fc02a420 comp rx=0 tx=0).stop 2024-09-13T17:37:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 -- 172.21.15.44:0/3926881511 shutdown_connections 2024-09-13T17:37:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 --2- 172.21.15.44:0/3926881511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c1050e0 0x7f960c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 --2- 172.21.15.44:0/3926881511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f960c100940 0x7f960c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 --2- 172.21.15.44:0/3926881511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c0fff90 0x7f960c100370 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 -- 172.21.15.44:0/3926881511 >> 172.21.15.44:0/3926881511 conn(0x7f960c0fba20 msgr2=0x7f960c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 -- 172.21.15.44:0/3926881511 shutdown_connections 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 -- 172.21.15.44:0/3926881511 wait complete. 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.519+0000 7f96112e9640 1 Processor -- start 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 -- start start 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c0fff90 0x7f960c196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 0x7f960c196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f960c19ab70 0x7f960c19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c19b560 con 0x7f960c1050e0 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c19b6d0 con 0x7f960c19ab70 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f96112e9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c19b840 con 0x7f960c0fff90 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 0x7f960c196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 0x7f960c196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37958/0 (socket says 172.21.15.44:37958) 2024-09-13T17:37:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c0fff90 0x7f960c196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 -- 172.21.15.44:0/2306521888 learned_addr learned my addr 172.21.15.44:0/2306521888 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960b7fe640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f960c19ab70 0x7f960c19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 -- 172.21.15.44:0/2306521888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c0fff90 msgr2=0x7f960c196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c0fff90 0x7f960c196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 -- 172.21.15.44:0/2306521888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f960c19ab70 msgr2=0x7f960c19b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f960c19ab70 0x7f960c19b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 -- 172.21.15.44:0/2306521888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc009560 con 0x7f960c1050e0 2024-09-13T17:37:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.520+0000 7f960a7fc640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 0x7f960c196780 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f960000c970 tx=0x7f960000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.521+0000 7f95f3fff640 1 -- 172.21.15.44:0/2306521888 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600007bb0 con 0x7f960c1050e0 2024-09-13T17:37:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.521+0000 7f95f3fff640 1 -- 172.21.15.44:0/2306521888 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9600007d10 con 0x7f960c1050e0 2024-09-13T17:37:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.521+0000 7f95f3fff640 1 -- 172.21.15.44:0/2306521888 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600004da0 con 0x7f960c1050e0 2024-09-13T17:37:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.521+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c1a2e20 con 0x7f960c1050e0 2024-09-13T17:37:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.521+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c1a3220 con 0x7f960c1050e0 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f95f3fff640 1 -- 172.21.15.44:0/2306521888 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9600004f00 con 0x7f960c1050e0 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 Processor -- start 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 -- start start 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91dcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d9146cc0 con 0x5614d91dd0b0 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d9144450 con 0x5614d91e7950 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.523+0000 7f9614010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d913fd40 con 0x5614d91dc790 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f95f3fff640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95f40a1200 0x7f95f40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37972/0 (socket says 172.21.15.44:37972) 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 -- 172.21.15.44:0/2033507427 learned_addr learned my addr 172.21.15.44:0/2033507427 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960a7fc640 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960b7fe640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95f40a1200 0x7f95f40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960b7fe640 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91dcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f95f3fff640 1 -- 172.21.15.44:0/2306521888 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f96000ccff0 con 0x7f960c1050e0 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 -- 172.21.15.44:0/2033507427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 msgr2=0x5614d91dcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91dcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 -- 172.21.15.44:0/2033507427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 msgr2=0x5614d91e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 -- 172.21.15.44:0/2033507427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d9107620 con 0x5614d91dd0b0 2024-09-13T17:37:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960b7fe640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95f40a1200 0x7f95f40a36c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f96040098b0 tx=0x7f9604009840 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f960affd640 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6fc1067216eee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f95f17fa640 1 -- 172.21.15.44:0/2033507427 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc004300 con 0x5614d91dd0b0 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.524+0000 7f95f17fa640 1 -- 172.21.15.44:0/2033507427 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95fc031df0 con 0x5614d91dd0b0 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f95f17fa640 1 -- 172.21.15.44:0/2033507427 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc003cd0 con 0x5614d91dd0b0 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 -- 172.21.15.44:0/2033507427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 msgr2=0x5614d91e7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc031670 comp rx=0 tx=0).stop 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 -- 172.21.15.44:0/2033507427 shutdown_connections 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 --2- 172.21.15.44:0/2033507427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91dcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 -- 172.21.15.44:0/2033507427 >> 172.21.15.44:0/2033507427 conn(0x5614d91d8d00 msgr2=0x5614d91d90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 -- 172.21.15.44:0/2033507427 shutdown_connections 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 -- 172.21.15.44:0/2033507427 wait complete. 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.525+0000 7f9614010740 1 -- start start 2024-09-13T17:37:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 Processor -- start 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d90fe180 con 0x5614d91dd0b0 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8e18430 con 0x5614d91e7950 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d90c35e0 con 0x5614d91dc790 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f9614010740 10 client.0 _ll_register_callbacks cb 0x5614d912faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37982/0 (socket says 172.21.15.44:37982) 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 -- 172.21.15.44:0/4173237070 learned_addr learned my addr 172.21.15.44:0/4173237070 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960affd640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960b7fe640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 -- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 msgr2=0x5614d91e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91e6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 -- 172.21.15.44:0/4173237070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 msgr2=0x5614d91e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 -- 172.21.15.44:0/4173237070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c10d470 con 0x5614d91dd0b0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960b7fe640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f960a7fc640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f9600099b40 tx=0x7f960000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa859350c540258 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.526+0000 7f95eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f95eeff5640 1 -- 172.21.15.44:0/4173237070 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960009a2a0 con 0x5614d91dd0b0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f95eeff5640 1 -- 172.21.15.44:0/4173237070 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f96000046c0 con 0x5614d91dd0b0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614d9169b80 con 0x5614d91dd0b0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f95eeff5640 1 -- 172.21.15.44:0/4173237070 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96000b1030 con 0x5614d91dd0b0 2024-09-13T17:37:38.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 10 client.19035 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5614d913e3c0 con 0x5614d91dd0b0 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f95eeff5640 1 -- 172.21.15.44:0/4173237070 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5614d9169b80 con 0x5614d91dd0b0 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f95eeff5640 1 -- 172.21.15.44:0/4173237070 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9600020070 con 0x5614d91dd0b0 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 10 client.19035 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 10 client.19035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 20 client.19035 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d91f72a0 0x5614d91f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 4 client.19035 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614d91321f0 con 0x5614d91f72a0 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.527+0000 7f960affd640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d91f72a0 0x5614d91f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.528+0000 7f960affd640 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d91f72a0 0x5614d91f9690 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2712d6436f172564 server_cookie=ada85391556e5be1 in_seq=0 out_seq=0 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.528+0000 7f95eeff5640 10 client.19035 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:38.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.528+0000 7f95eeff5640 1 -- 172.21.15.44:0/4173237070 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5614d91321f0 con 0x5614d91f72a0 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.528+0000 7f95eeff5640 10 client.19035 handle_command_reply: tid=0 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.528+0000 7f9614010740 1 client.19035 shutdown 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 msgr2=0x5614d91e7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f9600099b40 tx=0x7f960000df90 comp rx=0 tx=0).stop 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 shutdown_connections 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d91f72a0 0x5614d91f9690 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91e7410 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 --2- 172.21.15.44:0/4173237070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f95eeff5640 0 client.19035 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 >> 172.21.15.44:0/4173237070 conn(0x5614d91d8d00 msgr2=0x5614d91d7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 shutdown_connections 2024-09-13T17:37:38.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 1 -- 172.21.15.44:0/4173237070 wait complete. 2024-09-13T17:37:38.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.529+0000 7f9614010740 20 client.19035 trim_cache size 0 max 16384 2024-09-13T17:37:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 Processor -- start 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 -- start start 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 0x5614d91ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dd0b0 0x5614d91eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91f70d0 con 0x5614d91dc790 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91f5b70 con 0x5614d91e7950 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f9614010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91e5330 con 0x5614d91dd0b0 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 0x5614d91ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 0x5614d91ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37986/0 (socket says 172.21.15.44:37986) 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 -- 172.21.15.44:0/3450556263 learned_addr learned my addr 172.21.15.44:0/3450556263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960a7fc640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dd0b0 0x5614d91eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 -- 172.21.15.44:0/3450556263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dd0b0 msgr2=0x5614d91eeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dd0b0 0x5614d91eeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 -- 172.21.15.44:0/3450556263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 msgr2=0x5614d91e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960b7fe640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 -- 172.21.15.44:0/3450556263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d90e0770 con 0x5614d91dc790 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960a7fc640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dd0b0 0x5614d91eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960b7fe640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.540+0000 7f960affd640 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 0x5614d91ee620 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0039b0 tx=0x7f95fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeb193abfe19c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f95f1ffb640 1 -- 172.21.15.44:0/3450556263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc04d070 con 0x5614d91dc790 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f95f1ffb640 1 -- 172.21.15.44:0/3450556263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95fc0319e0 con 0x5614d91dc790 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f95f1ffb640 1 -- 172.21.15.44:0/3450556263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc051a90 con 0x5614d91dc790 2024-09-13T17:37:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 -- 172.21.15.44:0/3450556263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 msgr2=0x5614d91ee620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 0x5614d91ee620 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0039b0 tx=0x7f95fc005e80 comp rx=0 tx=0).stop 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 -- 172.21.15.44:0/3450556263 shutdown_connections 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dd0b0 0x5614d91eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 --2- 172.21.15.44:0/3450556263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dc790 0x5614d91ee620 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 -- 172.21.15.44:0/3450556263 >> 172.21.15.44:0/3450556263 conn(0x5614d91d8d00 msgr2=0x5614d91fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 -- 172.21.15.44:0/3450556263 shutdown_connections 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 -- 172.21.15.44:0/3450556263 wait complete. 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 -- start start 2024-09-13T17:37:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.541+0000 7f9614010740 1 Processor -- start 2024-09-13T17:37:38.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91ecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91ed2c0 con 0x5614d91dd0b0 2024-09-13T17:37:38.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91ed430 con 0x5614d91e7950 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91ed5f0 con 0x5614d91dc790 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 10 client.0 _ll_register_callbacks cb 0x7f95fc031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91eeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37994/0 (socket says 172.21.15.44:37994) 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 -- 172.21.15.44:0/1169404210 learned_addr learned my addr 172.21.15.44:0/1169404210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960affd640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960b7fe640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91ecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 -- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 msgr2=0x5614d91ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 -- 172.21.15.44:0/1169404210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 msgr2=0x5614d91ecbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91ecbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 -- 172.21.15.44:0/1169404210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614d9107620 con 0x5614d91dd0b0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960b7fe640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91ecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960affd640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f960a7fc640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91eeaf0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9600000c00 tx=0x7f9600095fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9ddfe7ef3b5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f95eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f95eeff5640 1 -- 172.21.15.44:0/1169404210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96000a2020 con 0x5614d91dd0b0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f95eeff5640 1 -- 172.21.15.44:0/1169404210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f96000b2070 con 0x5614d91dd0b0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.542+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95fc031df0 con 0x5614d91dd0b0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f95eeff5640 1 -- 172.21.15.44:0/1169404210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9600099c10 con 0x5614d91dd0b0 2024-09-13T17:37:38.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 10 client.19041 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5614d9105e80 con 0x5614d91dd0b0 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f95eeff5640 1 -- 172.21.15.44:0/1169404210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95fc031df0 con 0x5614d91dd0b0 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f95eeff5640 1 -- 172.21.15.44:0/1169404210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9600020040 con 0x5614d91dd0b0 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 10 client.19041 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 10 client.19041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 20 client.19041 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d924e690 0x5614d9250ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 4 client.19041 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614d8e334e0 con 0x5614d924e690 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.543+0000 7f960affd640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d924e690 0x5614d9250ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f95eeff5640 10 client.19041 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f960affd640 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d924e690 0x5614d9250ab0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5b881399179226c server_cookie=deed3b8bd60f4fa0 in_seq=0 out_seq=0 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f95eeff5640 1 -- 172.21.15.44:0/1169404210 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614d8e334e0 con 0x5614d924e690 2024-09-13T17:37:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f95eeff5640 10 client.19041 handle_command_reply: tid=0 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 client.19041 shutdown 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 msgr2=0x5614d91eeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91eeaf0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9600000c00 tx=0x7f9600095fc0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 shutdown_connections 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x5614d924e690 0x5614d9250ab0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5614d91e7950 0x5614d91ecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614d91dd0b0 0x5614d91eeaf0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f95eeff5640 0 client.19041 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 --2- 172.21.15.44:0/1169404210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614d91dc790 0x5614d91ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 >> 172.21.15.44:0/1169404210 conn(0x5614d91d8d00 msgr2=0x5614d91e29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 shutdown_connections 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 1 -- 172.21.15.44:0/1169404210 wait complete. 2024-09-13T17:37:38.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f9614010740 20 client.19041 trim_cache size 0 max 16384 2024-09-13T17:37:38.556 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:37:38.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95f40a1200 msgr2=0x7f95f40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f96112e9640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95f40a1200 0x7f95f40a36c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f96040098b0 tx=0x7f9604009840 comp rx=0 tx=0).stop 2024-09-13T17:37:38.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 msgr2=0x7f960c196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.544+0000 7f96112e9640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 0x7f960c196780 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f960000c970 tx=0x7f960000ce30 comp rx=0 tx=0).stop 2024-09-13T17:37:38.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 shutdown_connections 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f95f40a1200 0x7f95f40a36c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f960c19ab70 0x7f960c19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1050e0 0x7f960c196780 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 --2- 172.21.15.44:0/2306521888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c0fff90 0x7f960c196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 >> 172.21.15.44:0/2306521888 conn(0x7f960c0fba20 msgr2=0x7f960c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 shutdown_connections 2024-09-13T17:37:38.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.545+0000 7f96112e9640 1 -- 172.21.15.44:0/2306521888 wait complete. 2024-09-13T17:37:38.558 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:37:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 -- 172.21.15.44:0/2558149769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff940100940 msgr2=0x7ff940104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2558149769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff940100940 0x7ff940104980 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff928009930 tx=0x7ff92802a420 comp rx=0 tx=0).stop 2024-09-13T17:37:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 -- 172.21.15.44:0/2558149769 shutdown_connections 2024-09-13T17:37:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2558149769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9401050e0 0x7ff94010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2558149769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff940100940 0x7ff940104980 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2558149769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9400fff90 0x7ff940100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.662+0000 7ff946a6b640 1 -- 172.21.15.44:0/2558149769 >> 172.21.15.44:0/2558149769 conn(0x7ff9400fba20 msgr2=0x7ff9400fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 -- 172.21.15.44:0/2558149769 shutdown_connections 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 -- 172.21.15.44:0/2558149769 wait complete. 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 Processor -- start 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 -- start start 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 0x7ff940196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9401050e0 0x7ff940196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94019ab50 0x7ff94019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019b540 con 0x7ff9400fff90 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019b6b0 con 0x7ff9401050e0 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff946a6b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019b820 con 0x7ff94019ab50 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff944807640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 0x7ff940196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff944807640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 0x7ff940196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38008/0 (socket says 172.21.15.44:38008) 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.663+0000 7ff944807640 1 -- 172.21.15.44:0/2350023561 learned_addr learned my addr 172.21.15.44:0/2350023561 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff93ffff640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9401050e0 0x7ff940196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff945008640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94019ab50 0x7ff94019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff944807640 1 -- 172.21.15.44:0/2350023561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94019ab50 msgr2=0x7ff94019b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff944807640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94019ab50 0x7ff94019b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff944807640 1 -- 172.21.15.44:0/2350023561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9401050e0 msgr2=0x7ff940196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff944807640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9401050e0 0x7ff940196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff944807640 1 -- 172.21.15.44:0/2350023561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff928009560 con 0x7ff9400fff90 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff944807640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 0x7ff940196220 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e9f0 tx=0x7ff93000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff93dffb640 1 -- 172.21.15.44:0/2350023561 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930009720 con 0x7ff9400fff90 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff93dffb640 1 -- 172.21.15.44:0/2350023561 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff93000cd80 con 0x7ff9400fff90 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401a2e00 con 0x7ff9400fff90 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff93dffb640 1 -- 172.21.15.44:0/2350023561 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930010430 con 0x7ff9400fff90 2024-09-13T17:37:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.664+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a31d0 con 0x7ff9400fff90 2024-09-13T17:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.666+0000 7ff93dffb640 1 -- 172.21.15.44:0/2350023561 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff930004fb0 con 0x7ff9400fff90 2024-09-13T17:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 Processor -- start 2024-09-13T17:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 -- start start 2024-09-13T17:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c951cc0 con 0x561b0c9f2950 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c94f450 con 0x561b0c9e80b0 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff93dffb640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff92c0a1200 0x7ff92c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff94986c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c94ad40 con 0x561b0c9e7790 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff945008640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff945008640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9e7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:54316/0 (socket says 172.21.15.44:54316) 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff945008640 1 -- 172.21.15.44:0/2953068840 learned_addr learned my addr 172.21.15.44:0/2953068840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff944807640 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff945008640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff92c0a1200 0x7ff92c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.667+0000 7ff93dffb640 1 -- 172.21.15.44:0/2350023561 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff930014070 con 0x7ff9400fff90 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff93ffff640 1 -- 172.21.15.44:0/2953068840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 msgr2=0x561b0c9e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff93ffff640 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff93ffff640 1 -- 172.21.15.44:0/2953068840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 msgr2=0x561b0c9f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff93ffff640 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff93ffff640 1 -- 172.21.15.44:0/2953068840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b0c912620 con 0x561b0c9f2950 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff945008640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff92c0a1200 0x7ff92c0a36c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff938005c20 tx=0x7ff938009600 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff93ffff640 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f4d40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff92802a940 tx=0x7ff928005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f49cc474edf0b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff936ffd640 1 -- 172.21.15.44:0/2953068840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928004230 con 0x561b0c9f2950 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff936ffd640 1 -- 172.21.15.44:0/2953068840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff928031790 con 0x561b0c9f2950 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.668+0000 7ff936ffd640 1 -- 172.21.15.44:0/2953068840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928003a70 con 0x561b0c9f2950 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 -- 172.21.15.44:0/2953068840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 msgr2=0x561b0c9f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f4d40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff92802a940 tx=0x7ff928005c00 comp rx=0 tx=0).stop 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 -- 172.21.15.44:0/2953068840 shutdown_connections 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f4d40 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 --2- 172.21.15.44:0/2953068840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 -- 172.21.15.44:0/2953068840 >> 172.21.15.44:0/2953068840 conn(0x561b0c9e3d00 msgr2=0x561b0c9e40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 -- 172.21.15.44:0/2953068840 shutdown_connections 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 -- 172.21.15.44:0/2953068840 wait complete. 2024-09-13T17:37:38.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 -- start start 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.669+0000 7ff94986c740 1 Processor -- start 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 0x561b0c9f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9f2950 0x561b0c9ef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c909180 con 0x561b0c9e7790 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c623430 con 0x561b0c9e80b0 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c8ce5e0 con 0x561b0c9f2950 2024-09-13T17:37:38.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 10 client.0 _ll_register_callbacks cb 0x561b0c93aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 0x561b0c9f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 0x561b0c9f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38030/0 (socket says 172.21.15.44:38030) 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 -- 172.21.15.44:0/378939498 learned_addr learned my addr 172.21.15.44:0/378939498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff945008640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9f2950 0x561b0c9ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff93ffff640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 -- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9f2950 msgr2=0x561b0c9ef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9f2950 0x561b0c9ef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 -- 172.21.15.44:0/378939498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 msgr2=0x561b0c9f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 -- 172.21.15.44:0/378939498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94010d470 con 0x561b0c9e7790 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff944807640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 0x561b0c9f1ec0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff9300cd4a0 tx=0x7ff93001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0c46bb23c23960 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff91ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff91ffff640 1 -- 172.21.15.44:0/378939498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93009a840 con 0x561b0c9e7790 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff91ffff640 1 -- 172.21.15.44:0/378939498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff93009a140 con 0x561b0c9e7790 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.670+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b0c93d1f0 con 0x561b0c9e7790 2024-09-13T17:37:38.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff91ffff640 1 -- 172.21.15.44:0/378939498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93009df70 con 0x561b0c9e7790 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 10 client.19053 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b0c9493c0 con 0x561b0c9e7790 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff91ffff640 1 -- 172.21.15.44:0/378939498 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b0c93d1f0 con 0x561b0c9e7790 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff91ffff640 1 -- 172.21.15.44:0/378939498 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff93001f060 con 0x561b0c9e7790 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 10 client.19053 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 20 client.19053 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:38.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca022b0 0x561b0ca046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 4 client.19053 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b0c63e4e0 con 0x561b0ca022b0 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.671+0000 7ff93ffff640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca022b0 0x561b0ca046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff93ffff640 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca022b0 0x561b0ca046a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5a21b0474cd54b server_cookie=4ae85243094163cd in_seq=0 out_seq=0 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff91ffff640 10 client.19053 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff91ffff640 1 -- 172.21.15.44:0/378939498 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561b0c63e4e0 con 0x561b0ca022b0 2024-09-13T17:37:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff91ffff640 10 client.19053 handle_command_reply: tid=0 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff94986c740 1 client.19053 shutdown 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 msgr2=0x561b0c9f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff94986c740 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 0x561b0c9f1ec0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff9300cd4a0 tx=0x7ff93001d000 comp rx=0 tx=0).stop 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 shutdown_connections 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.672+0000 7ff94986c740 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca022b0 0x561b0ca046a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9f2950 0x561b0c9ef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff91ffff640 0 client.19053 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e80b0 0x561b0c9f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 1 --2- 172.21.15.44:0/378939498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e7790 0x561b0c9f1ec0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 >> 172.21.15.44:0/378939498 conn(0x561b0c9e3d00 msgr2=0x561b0c9e2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 shutdown_connections 2024-09-13T17:37:38.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 1 -- 172.21.15.44:0/378939498 wait complete. 2024-09-13T17:37:38.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.673+0000 7ff94986c740 20 client.19053 trim_cache size 0 max 16384 2024-09-13T17:37:38.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 Processor -- start 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 -- start start 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 0x561b0c9f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9f2950 0x561b0c9ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c9e48b0 con 0x561b0c9e80b0 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0ca00b70 con 0x561b0c9f2950 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff94986c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c9e4e90 con 0x561b0c9e7790 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 0x561b0c9f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 0x561b0c9f9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38034/0 (socket says 172.21.15.44:38034) 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 -- 172.21.15.44:0/1981399255 learned_addr learned my addr 172.21.15.44:0/1981399255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff945008640 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9f2950 0x561b0c9ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 -- 172.21.15.44:0/1981399255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 msgr2=0x561b0c9f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 -- 172.21.15.44:0/1981399255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9f2950 msgr2=0x561b0c9ea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9f2950 0x561b0c9ea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 -- 172.21.15.44:0/1981399255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b0c8eb770 con 0x561b0c9e80b0 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.684+0000 7ff93ffff640 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 0x561b0c9f9b60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff928004000 tx=0x7ff928000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9790218e91127d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff9377fe640 1 -- 172.21.15.44:0/1981399255 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92804c040 con 0x561b0c9e80b0 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff9377fe640 1 -- 172.21.15.44:0/1981399255 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff928031df0 con 0x561b0c9e80b0 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff9377fe640 1 -- 172.21.15.44:0/1981399255 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928003d90 con 0x561b0c9e80b0 2024-09-13T17:37:38.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 -- 172.21.15.44:0/1981399255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 msgr2=0x561b0c9f9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 0x561b0c9f9b60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff928004000 tx=0x7ff928000bc0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 -- 172.21.15.44:0/1981399255 shutdown_connections 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9f2950 0x561b0c9ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9e80b0 0x561b0c9f9b60 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 --2- 172.21.15.44:0/1981399255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e7790 0x561b0c9f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 -- 172.21.15.44:0/1981399255 >> 172.21.15.44:0/1981399255 conn(0x561b0c9e3d00 msgr2=0x561b0c9eda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 -- 172.21.15.44:0/1981399255 shutdown_connections 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 -- 172.21.15.44:0/1981399255 wait complete. 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.685+0000 7ff94986c740 1 -- start start 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 Processor -- start 2024-09-13T17:37:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e7790 0x561b0c9f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e80b0 0x561b0c9f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0c9f8e60 con 0x561b0c9f2950 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0ca58760 con 0x561b0c9e7790 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0ca588a0 con 0x561b0c9e80b0 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff94986c740 10 client.0 _ll_register_callbacks cb 0x7ff928009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff945008640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff944807640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e7790 0x561b0c9f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff945008640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38048/0 (socket says 172.21.15.44:38048) 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff945008640 1 -- 172.21.15.44:0/496811219 learned_addr learned my addr 172.21.15.44:0/496811219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff93ffff640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e80b0 0x561b0c9f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff944807640 1 -- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e80b0 msgr2=0x561b0c9f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff944807640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e80b0 0x561b0c9f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff944807640 1 -- 172.21.15.44:0/496811219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 msgr2=0x561b0c9f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff944807640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff944807640 1 -- 172.21.15.44:0/496811219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b0c912620 con 0x561b0c9e7790 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.686+0000 7ff945008640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff944807640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e7790 0x561b0c9f9b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9300055d0 tx=0x7ff9300cd2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998dcf9ff43f4d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff91ffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff91ffff640 1 -- 172.21.15.44:0/496811219 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930005460 con 0x561b0c9e7790 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff91ffff640 1 -- 172.21.15.44:0/496811219 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff93009a840 con 0x561b0c9e7790 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b0c974b80 con 0x561b0c9e7790 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff91ffff640 1 -- 172.21.15.44:0/496811219 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff930099f10 con 0x561b0c9e7790 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 10 client.28064 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b0c910e80 con 0x561b0c9e7790 2024-09-13T17:37:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff91ffff640 1 -- 172.21.15.44:0/496811219 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b0c974b80 con 0x561b0c9e7790 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff91ffff640 1 -- 172.21.15.44:0/496811219 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff93001f040 con 0x561b0c9e7790 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 10 client.28064 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 10 client.28064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 20 client.28064 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca59af0 0x561b0ca5bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 4 client.28064 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.687+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b0c64f7e0 con 0x561b0ca59af0 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff93ffff640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca59af0 0x561b0ca5bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff93ffff640 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca59af0 0x561b0ca5bee0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193691b9365867a3 server_cookie=3e20d37c41b69a84 in_seq=0 out_seq=0 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff91ffff640 10 client.28064 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff91ffff640 1 -- 172.21.15.44:0/496811219 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b0c64f7e0 con 0x561b0ca59af0 2024-09-13T17:37:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff91ffff640 10 client.28064 handle_command_reply: tid=0 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 client.28064 shutdown 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e7790 msgr2=0x561b0c9f9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e7790 0x561b0c9f9b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9300055d0 tx=0x7ff9300cd2d0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 shutdown_connections 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x561b0ca59af0 0x561b0ca5bee0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b0c9e80b0 0x561b0c9f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b0c9f2950 0x561b0c9f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 --2- 172.21.15.44:0/496811219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561b0c9e7790 0x561b0c9f9b10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff91ffff640 0 client.28064 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:38.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.688+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 >> 172.21.15.44:0/496811219 conn(0x561b0c9e3d00 msgr2=0x561b0c9e2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 shutdown_connections 2024-09-13T17:37:38.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff94986c740 1 -- 172.21.15.44:0/496811219 wait complete. 2024-09-13T17:37:38.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff94986c740 20 client.28064 trim_cache size 0 max 16384 2024-09-13T17:37:38.701 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:37:38.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff92c0a1200 msgr2=0x7ff92c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff92c0a1200 0x7ff92c0a36c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff938005c20 tx=0x7ff938009600 comp rx=0 tx=0).stop 2024-09-13T17:37:38.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 msgr2=0x7ff940196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 0x7ff940196220 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e9f0 tx=0x7ff93000edf0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 shutdown_connections 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff92c0a1200 0x7ff92c0a36c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94019ab50 0x7ff94019b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9401050e0 0x7ff940196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 --2- 172.21.15.44:0/2350023561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400fff90 0x7ff940196220 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 >> 172.21.15.44:0/2350023561 conn(0x7ff9400fba20 msgr2=0x7ff9400fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 shutdown_connections 2024-09-13T17:37:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:38.689+0000 7ff946a6b640 1 -- 172.21.15.44:0/2350023561 wait complete. 2024-09-13T17:37:38.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:38 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1682049667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:37:38.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:38 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1682049667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:37:39.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:38 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1682049667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:37:39.704 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T17:37:39.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 -- 172.21.15.44:0/3887973670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c0103f10 msgr2=0x7ff8c010ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/3887973670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c0103f10 0x7ff8c010ba60 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009cd0 tx=0x7ff8b802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 -- 172.21.15.44:0/3887973670 shutdown_connections 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/3887973670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8c00ff770 0x7ff8c01037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/3887973670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c0103f10 0x7ff8c010ba60 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/3887973670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c00fedc0 0x7ff8c00ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 -- 172.21.15.44:0/3887973670 >> 172.21.15.44:0/3887973670 conn(0x7ff8c00fa870 msgr2=0x7ff8c00fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 -- 172.21.15.44:0/3887973670 shutdown_connections 2024-09-13T17:37:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.824+0000 7ff8bf59e640 1 -- 172.21.15.44:0/3887973670 wait complete. 2024-09-13T17:37:39.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.825+0000 7ff8bf59e640 1 Processor -- start 2024-09-13T17:37:39.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.825+0000 7ff8bf59e640 1 -- start start 2024-09-13T17:37:39.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.825+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8c00fedc0 0x7ff8c0196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.825+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c0103f10 0x7ff8c0196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 0x7ff8c019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019b4e0 con 0x7ff8c019aaf0 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019b650 con 0x7ff8c0103f10 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019b7c0 con 0x7ff8c00fedc0 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 0x7ff8c019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 0x7ff8c019afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33516/0 (socket says 172.21.15.44:33516) 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 -- 172.21.15.44:0/2351376814 learned_addr learned my addr 172.21.15.44:0/2351376814 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8be59c640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8c00fedc0 0x7ff8c0196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c0103f10 0x7ff8c0196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 -- 172.21.15.44:0/2351376814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8c00fedc0 msgr2=0x7ff8c0196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8c00fedc0 0x7ff8c0196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 -- 172.21.15.44:0/2351376814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c0103f10 msgr2=0x7ff8c0196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c0103f10 0x7ff8c0196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 -- 172.21.15.44:0/2351376814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009900 con 0x7ff8c019aaf0 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bed9d640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 0x7ff8c019afa0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004af0 tx=0x7ff8b8030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8af7fe640 1 -- 172.21.15.44:0/2351376814 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8031b80 con 0x7ff8c019aaf0 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8af7fe640 1 -- 172.21.15.44:0/2351376814 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b8003e00 con 0x7ff8c019aaf0 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c01a2da0 con 0x7ff8c019aaf0 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.826+0000 7ff8af7fe640 1 -- 172.21.15.44:0/2351376814 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80047e0 con 0x7ff8c019aaf0 2024-09-13T17:37:39.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.827+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c01a3250 con 0x7ff8c019aaf0 2024-09-13T17:37:39.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8af7fe640 1 -- 172.21.15.44:0/2351376814 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff8b800a040 con 0x7ff8c019aaf0 2024-09-13T17:37:39.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 Processor -- start 2024-09-13T17:37:39.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 -- start start 2024-09-13T17:37:39.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb348c60 con 0x55e6fb3dd840 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb3463f0 con 0x55e6fb3de160 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb341ce0 con 0x55e6fb3ecf70 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8bdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8bdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ddc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33528/0 (socket says 172.21.15.44:33528) 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/1645788353 learned_addr learned my addr 172.21.15.44:0/1645788353 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8be59c640 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.830+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8af7fe640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff8a80a1280 0x7ff8a80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/1645788353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 msgr2=0x55e6fb3ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/1645788353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 msgr2=0x55e6fb3eca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3eca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/1645788353 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6fb2fd970 con 0x55e6fb3dd840 2024-09-13T17:37:39.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8be59c640 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:39.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bed9d640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff8a80a1280 0x7ff8a80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8af7fe640 1 -- 172.21.15.44:0/2351376814 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff8b80ee460 con 0x7ff8c019aaf0 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ddc20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009f70 tx=0x7ff8b4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e19620fb085efbf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8acff9640 1 -- 172.21.15.44:0/1645788353 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004050 con 0x55e6fb3dd840 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8acff9640 1 -- 172.21.15.44:0/1645788353 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b4005020 con 0x55e6fb3dd840 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.831+0000 7ff8acff9640 1 -- 172.21.15.44:0/1645788353 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40049a0 con 0x55e6fb3dd840 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/1645788353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 msgr2=0x55e6fb3ddc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8bed9d640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff8a80a1280 0x7ff8a80a3740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80eb990 tx=0x7ff8b80b7690 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ddc20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009f70 tx=0x7ff8b4005850 comp rx=0 tx=0).stop 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/1645788353 shutdown_connections 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/1645788353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ddc20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/1645788353 >> 172.21.15.44:0/1645788353 conn(0x55e6fb3d9f30 msgr2=0x55e6fb3da310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/1645788353 shutdown_connections 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/1645788353 wait complete. 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 -- start start 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.832+0000 7ff8c7dfb740 1 Processor -- start 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 0x55e6fb3e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb2fff80 con 0x55e6fb3de160 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb2ffc20 con 0x55e6fb3dd840 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb2ccab0 con 0x55e6fb3ecf70 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8c7dfb740 10 client.0 _ll_register_callbacks cb 0x55e6fb33b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 0x55e6fb3e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 0x55e6fb3e9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33534/0 (socket says 172.21.15.44:33534) 2024-09-13T17:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 -- 172.21.15.44:0/477143889 learned_addr learned my addr 172.21.15.44:0/477143889 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 -- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 msgr2=0x55e6fb3ea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ea2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 -- 172.21.15.44:0/477143889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3dd840 msgr2=0x55e6fb3ec980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ec980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 -- 172.21.15.44:0/477143889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b4009cc0 con 0x55e6fb3de160 2024-09-13T17:37:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8be59c640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 0x55e6fb3e9c50 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0012700 tx=0x7ff8b0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c30462bfa001bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.833+0000 7ff8a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/477143889 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0017070 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/477143889 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff8b0004020 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6fb035590 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/477143889 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b00041e0 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 10 client.19071 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e6fb340360 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/477143889 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e6fb035590 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/477143889 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8b0003820 con 0x55e6fb3de160 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 10 client.19071 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 10 client.19071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 20 client.19071 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3fce20 0x55e6fb3ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 4 client.19071 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.834+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6fb046860 con 0x55e6fb3fce20 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.835+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3fce20 0x55e6fb3ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.835+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3fce20 0x55e6fb3ff240 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca7cd8846aeb26a server_cookie=6593dcd086be3635 in_seq=0 out_seq=0 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.835+0000 7ff8a5ffb640 10 client.19071 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/477143889 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e6fb046860 con 0x55e6fb3fce20 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8a5ffb640 10 client.19071 handle_command_reply: tid=0 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 client.19071 shutdown 2024-09-13T17:37:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 msgr2=0x55e6fb3e9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 0x55e6fb3e9c50 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0012700 tx=0x7ff8b0012bc0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 shutdown_connections 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3fce20 0x55e6fb3ff240 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3ecf70 0x55e6fb3ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3de160 0x55e6fb3e9c50 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/477143889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3dd840 0x55e6fb3ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 >> 172.21.15.44:0/477143889 conn(0x55e6fb3d9f30 msgr2=0x55e6fb3d9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8a5ffb640 0 client.19071 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 shutdown_connections 2024-09-13T17:37:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/477143889 wait complete. 2024-09-13T17:37:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.836+0000 7ff8c7dfb740 20 client.19071 trim_cache size 0 max 16384 2024-09-13T17:37:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.850+0000 7ff8c7dfb740 1 Processor -- start 2024-09-13T17:37:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 -- start start 2024-09-13T17:37:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3dd840 0x55e6fb3e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3e5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb3f8bb0 con 0x55e6fb3d9f30 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb3dac20 con 0x55e6fb3de160 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb3ec570 con 0x55e6fb3dd840 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3d9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33540/0 (socket says 172.21.15.44:33540) 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/2583872863 learned_addr learned my addr 172.21.15.44:0/2583872863 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/2583872863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3dd840 msgr2=0x55e6fb3e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3dd840 0x55e6fb3e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/2583872863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 msgr2=0x55e6fb3e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 -- 172.21.15.44:0/2583872863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6fb2e21c0 con 0x55e6fb3d9f30 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.851+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3d9950 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40058c0 tx=0x7ff8b4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f713810066c0ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8ad7fa640 1 -- 172.21.15.44:0/2583872863 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4005630 con 0x55e6fb3d9f30 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8ad7fa640 1 -- 172.21.15.44:0/2583872863 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b40132d0 con 0x55e6fb3d9f30 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8ad7fa640 1 -- 172.21.15.44:0/2583872863 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004d40 con 0x55e6fb3d9f30 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/2583872863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 msgr2=0x55e6fb3d9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3d9950 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40058c0 tx=0x7ff8b4009830 comp rx=0 tx=0).stop 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/2583872863 shutdown_connections 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3dd840 0x55e6fb3e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3de160 0x55e6fb3e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/2583872863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3d9950 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/2583872863 >> 172.21.15.44:0/2583872863 conn(0x55e6fb3ecae0 msgr2=0x55e6fb403ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/2583872863 shutdown_connections 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/2583872863 wait complete. 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.852+0000 7ff8c7dfb740 1 -- start start 2024-09-13T17:37:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 Processor -- start 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3f0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3de160 0x55e6fb3f1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb4533c0 con 0x55e6fb3dd840 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb453500 con 0x55e6fb3d9f30 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6fb4536c0 con 0x55e6fb3de160 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8c7dfb740 10 client.0 _ll_register_callbacks cb 0x7ff8b0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33552/0 (socket says 172.21.15.44:33552) 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 -- 172.21.15.44:0/3687465694 learned_addr learned my addr 172.21.15.44:0/3687465694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3de160 0x55e6fb3f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 -- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3de160 msgr2=0x55e6fb3f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3de160 0x55e6fb3f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 -- 172.21.15.44:0/3687465694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3d9f30 msgr2=0x55e6fb3f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 -- 172.21.15.44:0/3687465694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6fb2fd970 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8be59c640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3f0e50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0003c00 tx=0x7ff8b0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a002c1ddaa1e00f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.853+0000 7ff8a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/3687465694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0017070 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/3687465694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff8b0000e40 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8b4005020 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/3687465694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b000f9a0 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 10 client.19077 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e6fb307990 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/3687465694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b4005020 con 0x55e6fb3dd840 2024-09-13T17:37:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/3687465694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8b0031070 con 0x55e6fb3dd840 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 10 client.19077 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 10 client.19077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 20 client.19077 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3ecf70 0x55e6fb456e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 4 client.19077 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e6fb099eb0 con 0x55e6fb3ecf70 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.854+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3ecf70 0x55e6fb456e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.855+0000 7ff8bdd9b640 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3ecf70 0x55e6fb456e70 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da5a285340d6648 server_cookie=8025b37fd5c27058 in_seq=0 out_seq=0 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.855+0000 7ff8a5ffb640 10 client.19077 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8a5ffb640 1 -- 172.21.15.44:0/3687465694 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e6fb099eb0 con 0x55e6fb3ecf70 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8a5ffb640 10 client.19077 handle_command_reply: tid=0 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 client.19077 shutdown 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 msgr2=0x55e6fb3f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3f0e50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0003c00 tx=0x7ff8b0009240 comp rx=0 tx=0).stop 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 shutdown_connections 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55e6fb3ecf70 0x55e6fb456e70 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6fb3de160 0x55e6fb3f1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6fb3dd840 0x55e6fb3f0e50 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8a5ffb640 0 client.19077 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 --2- 172.21.15.44:0/3687465694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6fb3d9f30 0x55e6fb3f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 >> 172.21.15.44:0/3687465694 conn(0x55e6fb3ecae0 msgr2=0x55e6fb3e4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 shutdown_connections 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 1 -- 172.21.15.44:0/3687465694 wait complete. 2024-09-13T17:37:39.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.856+0000 7ff8c7dfb740 20 client.19077 trim_cache size 0 max 16384 2024-09-13T17:37:39.865 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:37:39.865 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-09-13T17:37:39.865 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "f554d1d3-07d6-4820-959b-92aad51282df", 2024-09-13T17:37:39.865 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-09-13T17:37:39.865 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff8a80a1280 msgr2=0x7ff8a80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff8a80a1280 0x7ff8a80a3740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80eb990 tx=0x7ff8b80b7690 comp rx=0 tx=0).stop 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 msgr2=0x7ff8c019afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 0x7ff8c019afa0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004af0 tx=0x7ff8b8030dd0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 shutdown_connections 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff8a80a1280 0x7ff8a80a3740 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c019aaf0 0x7ff8c019afa0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c0103f10 0x7ff8c0196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 --2- 172.21.15.44:0/2351376814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8c00fedc0 0x7ff8c0196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 >> 172.21.15.44:0/2351376814 conn(0x7ff8c00fa870 msgr2=0x7ff8c00fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 shutdown_connections 2024-09-13T17:37:39.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.857+0000 7ff8bf59e640 1 -- 172.21.15.44:0/2351376814 wait complete. 2024-09-13T17:37:39.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:37:39.891 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:39 smithi044 ceph-mon[31175]: pgmap v341: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-09-13T17:37:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:39 smithi070 ceph-mon[34502]: pgmap v341: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-09-13T17:37:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 -- 172.21.15.44:0/2345659030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 msgr2=0x7feca4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 --2- 172.21.15.44:0/2345659030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca4102560 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fec98009930 tx=0x7fec9802a420 comp rx=0 tx=0).stop 2024-09-13T17:37:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 -- 172.21.15.44:0/2345659030 shutdown_connections 2024-09-13T17:37:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 --2- 172.21.15.44:0/2345659030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca41072d0 0x7feca410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 --2- 172.21.15.44:0/2345659030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca4102b30 0x7feca4106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 --2- 172.21.15.44:0/2345659030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca4102560 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 -- 172.21.15.44:0/2345659030 >> 172.21.15.44:0/2345659030 conn(0x7feca40fdbd0 msgr2=0x7feca40ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 -- 172.21.15.44:0/2345659030 shutdown_connections 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 -- 172.21.15.44:0/2345659030 wait complete. 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 Processor -- start 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.974+0000 7fecab096640 1 -- start start 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7fecab096640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7fecab096640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca41072d0 0x7feca41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7fecab096640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca419ccd0 0x7feca419d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7fecab096640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca419d6c0 con 0x7feca4102180 2024-09-13T17:37:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7fecab096640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca419d830 con 0x7feca41072d0 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7fecab096640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca419d9a0 con 0x7feca419ccd0 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33566/0 (socket says 172.21.15.44:33566) 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 -- 172.21.15.44:0/3741570571 learned_addr learned my addr 172.21.15.44:0/3741570571 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca3fff640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca41072d0 0x7feca41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca9633640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca419ccd0 0x7feca419d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 -- 172.21.15.44:0/3741570571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca419ccd0 msgr2=0x7feca419d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca419ccd0 0x7feca419d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 -- 172.21.15.44:0/3741570571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca41072d0 msgr2=0x7feca41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca41072d0 0x7feca41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 -- 172.21.15.44:0/3741570571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009490 con 0x7feca4102180 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca9633640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca419ccd0 0x7feca419d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca3fff640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca41072d0 0x7feca41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca8e32640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca41983a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fec98005ab0 tx=0x7fec980329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.975+0000 7feca1ffb640 1 -- 172.21.15.44:0/3741570571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98032df0 con 0x7feca4102180 2024-09-13T17:37:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.976+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec98009560 con 0x7feca4102180 2024-09-13T17:37:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.976+0000 7feca1ffb640 1 -- 172.21.15.44:0/3741570571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec98005490 con 0x7feca4102180 2024-09-13T17:37:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.976+0000 7feca1ffb640 1 -- 172.21.15.44:0/3741570571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980035f0 con 0x7feca4102180 2024-09-13T17:37:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.976+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca41a5210 con 0x7feca4102180 2024-09-13T17:37:39.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.978+0000 7feca1ffb640 1 -- 172.21.15.44:0/3741570571 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fec980046c0 con 0x7feca4102180 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 Processor -- start 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 -- start start 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3ee10 0x55db20a3f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a49a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a4c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db209a9f20 con 0x55db20a49fd0 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db209a76b0 con 0x55db20a3ee10 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7fecade90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db209a2fa0 con 0x55db20a3f730 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca1ffb640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fec940a1200 0x7fec940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a4c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a4c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33574/0 (socket says 172.21.15.44:33574) 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 -- 172.21.15.44:0/3227606370 learned_addr learned my addr 172.21.15.44:0/3227606370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca9633640 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3ee10 0x55db20a3f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca8e32640 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a49a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca9633640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fec940a1200 0x7fec940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca1ffb640 1 -- 172.21.15.44:0/3741570571 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fec980edcb0 con 0x7feca4102180 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 -- 172.21.15.44:0/3227606370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 msgr2=0x55db20a49a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a49a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 -- 172.21.15.44:0/3227606370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3ee10 msgr2=0x55db20a3f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3ee10 0x55db20a3f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.979+0000 7feca3fff640 1 -- 172.21.15.44:0/3227606370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db2095d840 con 0x55db20a49fd0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.980+0000 7feca3fff640 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a4c3c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fec90009e40 tx=0x7fec900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f6626a2d98c0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.980+0000 7feca9633640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fec940a1200 0x7fec940a36c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0094d0 tx=0x7fec9c009500 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.980+0000 7fec8effd640 1 -- 172.21.15.44:0/3227606370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90012650 con 0x55db20a49fd0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.980+0000 7fec8effd640 1 -- 172.21.15.44:0/3227606370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec90012c40 con 0x55db20a49fd0 2024-09-13T17:37:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.980+0000 7fec8effd640 1 -- 172.21.15.44:0/3227606370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000fc40 con 0x55db20a49fd0 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- 172.21.15.44:0/3227606370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 msgr2=0x55db20a4c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a4c3c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fec90009e40 tx=0x7fec900117b0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- 172.21.15.44:0/3227606370 shutdown_connections 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a49a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a4c3c0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- 172.21.15.44:0/3227606370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3ee10 0x55db20a3f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- 172.21.15.44:0/3227606370 >> 172.21.15.44:0/3227606370 conn(0x55db20a3b380 msgr2=0x55db20a3b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- 172.21.15.44:0/3227606370 shutdown_connections 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- 172.21.15.44:0/3227606370 wait complete. 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- start start 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 Processor -- start 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a49920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a46c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20a47170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db209442a0 con 0x55db20a3ee10 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2067b430 con 0x55db20a49fd0 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20926680 con 0x55db20a3f730 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.981+0000 7fecade90740 10 client.0 _ll_register_callbacks cb 0x55db2099c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a49920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a49920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33576/0 (socket says 172.21.15.44:33576) 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 -- 172.21.15.44:0/3840799837 learned_addr learned my addr 172.21.15.44:0/3840799837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca3fff640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a46c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 -- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 msgr2=0x55db20a46c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a46c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 -- 172.21.15.44:0/3840799837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 msgr2=0x55db20a47170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca9633640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20a47170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20a47170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 -- 172.21.15.44:0/3840799837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec90009a70 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7feca8e32640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a49920 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fec980b7460 tx=0x7fec98037f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1510f049c1dbaaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fec83fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fec83fff640 1 -- 172.21.15.44:0/3840799837 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980eb4b0 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fec83fff640 1 -- 172.21.15.44:0/3840799837 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fec980ba9f0 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db206964e0 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fec83fff640 1 -- 172.21.15.44:0/3840799837 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec980be820 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fecade90740 10 client.19089 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55db209a1620 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.982+0000 7fec83fff640 1 -- 172.21.15.44:0/3840799837 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55db206964e0 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fec83fff640 1 -- 172.21.15.44:0/3840799837 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec980bac50 con 0x55db20a3ee10 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fecade90740 10 client.19089 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fecade90740 10 client.19089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fecade90740 20 client.19089 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fecade90740 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20a59b10 0x55db20a5bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fecade90740 4 client.19089 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db206a77e0 con 0x55db20a59b10 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7feca3fff640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20a59b10 0x55db20a5bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7fec83fff640 10 client.19089 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.983+0000 7feca3fff640 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20a59b10 0x55db20a5bf00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe66d2922782cdf server_cookie=b2d09b056fa364b3 in_seq=0 out_seq=0 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fec83fff640 1 -- 172.21.15.44:0/3840799837 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55db206a77e0 con 0x55db20a59b10 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fec83fff640 10 client.19089 handle_command_reply: tid=0 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 client.19089 shutdown 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 msgr2=0x55db20a49920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a49920 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fec980b7460 tx=0x7fec98037f50 comp rx=0 tx=0).stop 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 shutdown_connections 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20a59b10 0x55db20a5bf00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a46c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fec83fff640 0 client.19089 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:39.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20a47170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 --2- 172.21.15.44:0/3840799837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a49920 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:39.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 >> 172.21.15.44:0/3840799837 conn(0x55db20a3b380 msgr2=0x55db20a3a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:39.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 shutdown_connections 2024-09-13T17:37:39.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 1 -- 172.21.15.44:0/3840799837 wait complete. 2024-09-13T17:37:39.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.984+0000 7fecade90740 20 client.19089 trim_cache size 0 max 16384 2024-09-13T17:37:40.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.998+0000 7fecade90740 1 Processor -- start 2024-09-13T17:37:40.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.998+0000 7fecade90740 1 -- start start 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fecade90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3ee10 0x55db20a3ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fecade90740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3f730 0x55db20a49430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fecade90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a50d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fecade90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20a3fca0 con 0x55db20a49fd0 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fecade90740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20a55c80 con 0x55db20a3f730 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fecade90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20a56410 con 0x55db20a3ee10 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a50d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a50d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33592/0 (socket says 172.21.15.44:33592) 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 -- 172.21.15.44:0/2897947467 learned_addr learned my addr 172.21.15.44:0/2897947467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca3fff640 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3f730 0x55db20a49430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca8e32640 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3ee10 0x55db20a3ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 -- 172.21.15.44:0/2897947467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3ee10 msgr2=0x55db20a3ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3ee10 0x55db20a3ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 -- 172.21.15.44:0/2897947467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3f730 msgr2=0x55db20a49430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3f730 0x55db20a49430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 -- 172.21.15.44:0/2897947467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db20943790 con 0x55db20a49fd0 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7feca9633640 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a50d60 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fec9c005ea0 tx=0x7fec9c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570d099255ac1bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fec8f7fe640 1 -- 172.21.15.44:0/2897947467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c004250 con 0x55db20a49fd0 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fec8f7fe640 1 -- 172.21.15.44:0/2897947467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec9c024070 con 0x55db20a49fd0 2024-09-13T17:37:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:39.999+0000 7fec8f7fe640 1 -- 172.21.15.44:0/2897947467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c02b9d0 con 0x55db20a49fd0 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 -- 172.21.15.44:0/2897947467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 msgr2=0x55db20a50d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a50d60 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fec9c005ea0 tx=0x7fec9c009920 comp rx=0 tx=0).stop 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 -- 172.21.15.44:0/2897947467 shutdown_connections 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a49fd0 0x55db20a50d60 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a3f730 0x55db20a49430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 --2- 172.21.15.44:0/2897947467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3ee10 0x55db20a3ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 -- 172.21.15.44:0/2897947467 >> 172.21.15.44:0/2897947467 conn(0x55db20a3b380 msgr2=0x55db20a41ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 -- 172.21.15.44:0/2897947467 shutdown_connections 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 -- 172.21.15.44:0/2897947467 wait complete. 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 -- start start 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.000+0000 7fecade90740 1 Processor -- start 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20aa8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20aa9490 con 0x55db20a3ee10 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20aa9600 con 0x55db20a49fd0 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20aa97c0 con 0x55db20a3f730 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 10 client.0 _ll_register_callbacks cb 0x7fec9802cfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a4d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33602/0 (socket says 172.21.15.44:33602) 2024-09-13T17:37:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 -- 172.21.15.44:0/1926390618 learned_addr learned my addr 172.21.15.44:0/1926390618 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca3fff640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca9633640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20aa8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 -- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 msgr2=0x55db20a4d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a4d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 -- 172.21.15.44:0/1926390618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 msgr2=0x55db20aa8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20aa8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 -- 172.21.15.44:0/1926390618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0148b0 con 0x55db20a3ee10 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7feca8e32640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a4d0d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c00 tx=0x7fec980049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87329bcb7a5c818c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fec83fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fec83fff640 1 -- 172.21.15.44:0/1926390618 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980eb4b0 con 0x55db20a3ee10 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fec83fff640 1 -- 172.21.15.44:0/1926390618 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fec980ba9f0 con 0x55db20a3ee10 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.001+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec90012c40 con 0x55db20a3ee10 2024-09-13T17:37:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fec83fff640 1 -- 172.21.15.44:0/1926390618 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec980b9ee0 con 0x55db20a3ee10 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 10 client.19095 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55db20968ea0 con 0x55db20a3ee10 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fec83fff640 1 -- 172.21.15.44:0/1926390618 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec90012c40 con 0x55db20a3ee10 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fec83fff640 1 -- 172.21.15.44:0/1926390618 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec980b3050 con 0x55db20a3ee10 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 10 client.19095 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 10 client.19095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 20 client.19095 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20ab0af0 0x55db20ab2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 4 client.19095 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db206fae30 con 0x55db20ab0af0 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.002+0000 7feca3fff640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20ab0af0 0x55db20ab2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fec83fff640 10 client.19095 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7feca3fff640 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20ab0af0 0x55db20ab2ee0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e2b48d29ee5c9f server_cookie=a5e55905d0dc4372 in_seq=0 out_seq=0 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fec83fff640 1 -- 172.21.15.44:0/1926390618 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55db206fae30 con 0x55db20ab0af0 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fec83fff640 10 client.19095 handle_command_reply: tid=0 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 client.19095 shutdown 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 msgr2=0x55db20a4d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a4d0d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c00 tx=0x7fec980049e0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 shutdown_connections 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55db20ab0af0 0x55db20ab2ee0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db20a3f730 0x55db20a4d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fec83fff640 0 client.19095 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db20a49fd0 0x55db20aa8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 --2- 172.21.15.44:0/1926390618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55db20a3ee10 0x55db20a4d0d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 >> 172.21.15.44:0/1926390618 conn(0x55db20a3b380 msgr2=0x55db20a39cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 shutdown_connections 2024-09-13T17:37:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 1 -- 172.21.15.44:0/1926390618 wait complete. 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.003+0000 7fecade90740 20 client.19095 trim_cache size 0 max 16384 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: "f554d1d3-07d6-4820-959b-92aad51282df": { 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "f554d1d3-07d6-4820-959b-92aad51282df", 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-09-13T17:37:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:37:40.012 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:37:40.012 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:37:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fec940a1200 msgr2=0x7fec940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fec940a1200 0x7fec940a36c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0094d0 tx=0x7fec9c009500 comp rx=0 tx=0).stop 2024-09-13T17:37:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 msgr2=0x7feca41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca41983a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fec98005ab0 tx=0x7fec980329f0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 shutdown_connections 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fec940a1200 0x7fec940a36c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca419ccd0 0x7feca419d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca41072d0 0x7feca41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 --2- 172.21.15.44:0/3741570571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca4102180 0x7feca41983a0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 >> 172.21.15.44:0/3741570571 conn(0x7feca40fdbd0 msgr2=0x7feca40ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 shutdown_connections 2024-09-13T17:37:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:37:40.004+0000 7fecab096640 1 -- 172.21.15.44:0/3741570571 wait complete. 2024-09-13T17:37:40.015 INFO:tasks.cephfs.filesystem:scrub status for tag:f554d1d3-07d6-4820-959b-92aad51282df - {'path': '/', 'tag': 'f554d1d3-07d6-4820-959b-92aad51282df', 'options': 'recursive,force'} 2024-09-13T17:37:40.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:39 smithi100 ceph-mon[35055]: pgmap v341: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-09-13T17:37:40.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:40 smithi070 ceph-mon[34502]: scrub queued for path: / 2024-09-13T17:37:40.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:40 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:40.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:40 smithi070 ceph-mon[34502]: scrub summary: active paths [/] 2024-09-13T17:37:40.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:40 smithi070 ceph-mon[34502]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:41.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:40 smithi100 ceph-mon[35055]: scrub queued for path: / 2024-09-13T17:37:41.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:40 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:41.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:40 smithi100 ceph-mon[35055]: scrub summary: active paths [/] 2024-09-13T17:37:41.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:40 smithi100 ceph-mon[35055]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:41.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:40 smithi044 ceph-mon[31175]: scrub queued for path: / 2024-09-13T17:37:41.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:40 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:41.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:40 smithi044 ceph-mon[31175]: scrub summary: active paths [/] 2024-09-13T17:37:41.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:40 smithi044 ceph-mon[31175]: scrub summary: idle+waiting paths [/] 2024-09-13T17:37:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:41 smithi070 ceph-mon[34502]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:37:42.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:41 smithi100 ceph-mon[35055]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:37:42.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:41 smithi044 ceph-mon[31175]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:37:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:43 smithi070 ceph-mon[34502]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:37:44.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:43 smithi100 ceph-mon[35055]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:37:44.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:43 smithi044 ceph-mon[31175]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-13T17:37:44.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:44 smithi070 ceph-mon[34502]: scrub summary: idle 2024-09-13T17:37:45.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:44 smithi100 ceph-mon[35055]: scrub summary: idle 2024-09-13T17:37:45.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:44 smithi044 ceph-mon[31175]: scrub summary: idle 2024-09-13T17:37:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:45 smithi070 ceph-mon[34502]: pgmap v344: 129 pgs: 129 active+clean; 1.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-13T17:37:46.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:45 smithi100 ceph-mon[35055]: pgmap v344: 129 pgs: 129 active+clean; 1.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-13T17:37:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:45 smithi044 ceph-mon[31175]: pgmap v344: 129 pgs: 129 active+clean; 1.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-13T17:37:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:47 smithi070 ceph-mon[34502]: pgmap v345: 129 pgs: 129 active+clean; 1.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:37:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:47 smithi100 ceph-mon[35055]: pgmap v345: 129 pgs: 129 active+clean; 1.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:37:48.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:47 smithi044 ceph-mon[31175]: pgmap v345: 129 pgs: 129 active+clean; 1.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 -- 172.21.15.44:0/2086244056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2864067d70 msgr2=0x7f28640681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 --2- 172.21.15.44:0/2086244056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2864067d70 0x7f28640681d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2854009900 tx=0x7f285402a410 comp rx=0 tx=0).stop 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 -- 172.21.15.44:0/2086244056 shutdown_connections 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 --2- 172.21.15.44:0/2086244056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f286410c660 0x7f286410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 --2- 172.21.15.44:0/2086244056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2864067d70 0x7f28640681d0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 --2- 172.21.15.44:0/2086244056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2864100ca0 0x7f2864067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 -- 172.21.15.44:0/2086244056 >> 172.21.15.44:0/2086244056 conn(0x7f2864074940 msgr2=0x7f2864074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 -- 172.21.15.44:0/2086244056 shutdown_connections 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.932+0000 7f286b733640 1 -- 172.21.15.44:0/2086244056 wait complete. 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 Processor -- start 2024-09-13T17:37:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 -- start start 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2864067d70 0x7f2864198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2864100ca0 0x7f2864198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 0x7f286419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286419d330 con 0x7f286410c660 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286419d4a0 con 0x7f2864067d70 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f286b733640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286419d660 con 0x7f2864100ca0 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f2869cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 0x7f286419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f2869cd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 0x7f286419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54230/0 (socket says 172.21.15.44:54230) 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f2869cd0640 1 -- 172.21.15.44:0/3816664952 learned_addr learned my addr 172.21.15.44:0/3816664952 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f28694cf640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2864067d70 0x7f2864198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f2868cce640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2864100ca0 0x7f2864198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f2869cd0640 1 -- 172.21.15.44:0/3816664952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2864100ca0 msgr2=0x7f2864198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.933+0000 7f2869cd0640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2864100ca0 0x7f2864198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f2869cd0640 1 -- 172.21.15.44:0/3816664952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2864067d70 msgr2=0x7f2864198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f2869cd0640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2864067d70 0x7f2864198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f2869cd0640 1 -- 172.21.15.44:0/3816664952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854009560 con 0x7f286410c660 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f28694cf640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2864067d70 0x7f2864198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f2869cd0640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 0x7f286419cd30 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f285c00e9d0 tx=0x7f285c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f285a7fc640 1 -- 172.21.15.44:0/3816664952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00cd80 con 0x7f286410c660 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f285a7fc640 1 -- 172.21.15.44:0/3816664952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f285c005020 con 0x7f286410c660 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286419d940 con 0x7f286410c660 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f285a7fc640 1 -- 172.21.15.44:0/3816664952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0116d0 con 0x7f286410c660 2024-09-13T17:37:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.934+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28641a51c0 con 0x7f286410c660 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.937+0000 7f286e457740 1 Processor -- start 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.937+0000 7f285a7fc640 1 -- 172.21.15.44:0/3816664952 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f285c005190 con 0x7f286410c660 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 -- start start 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 0x558dface3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3c50 0x558dfacedfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacee4f0 0x558dfacf08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfac4ec30 con 0x558dface3330 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfac4c3c0 con 0x558dface3c50 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f286e457740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfac47cb0 con 0x558dfacee4f0 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 0x558dface3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 0x558dface3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54242/0 (socket says 172.21.15.44:54242) 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 -- 172.21.15.44:0/447593479 learned_addr learned my addr 172.21.15.44:0/447593479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f28694cf640 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3c50 0x558dfacedfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacee4f0 0x558dfacf08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f285a7fc640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28500a1200 0x7f28500a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 -- 172.21.15.44:0/447593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacee4f0 msgr2=0x558dfacf08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacee4f0 0x558dfacf08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 -- 172.21.15.44:0/447593479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3c50 msgr2=0x558dfacedfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3c50 0x558dfacedfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2868cce640 1 -- 172.21.15.44:0/447593479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dfabee910 con 0x558dface3330 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f28694cf640 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3c50 0x558dfacedfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.938+0000 7f2869cd0640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28500a1200 0x7f28500a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.939+0000 7f285a7fc640 1 -- 172.21.15.44:0/3816664952 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f285c0cd2c0 con 0x7f286410c660 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.939+0000 7f2868cce640 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 0x558dface3710 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2854005c40 tx=0x7f2854005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec30e9984db01258 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.939+0000 7f284f7fe640 1 -- 172.21.15.44:0/447593479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854031960 con 0x558dface3330 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.939+0000 7f284f7fe640 1 -- 172.21.15.44:0/447593479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2854045020 con 0x558dface3330 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.939+0000 7f284f7fe640 1 -- 172.21.15.44:0/447593479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854003d80 con 0x558dface3330 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 -- 172.21.15.44:0/447593479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 msgr2=0x558dface3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f2869cd0640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28500a1200 0x7f28500a36c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f285c099f90 tx=0x7f285c06c0e0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 0x558dface3710 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2854005c40 tx=0x7f2854005760 comp rx=0 tx=0).stop 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 -- 172.21.15.44:0/447593479 shutdown_connections 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacee4f0 0x558dfacf08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3c50 0x558dfacedfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 --2- 172.21.15.44:0/447593479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3330 0x558dface3710 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 -- 172.21.15.44:0/447593479 >> 172.21.15.44:0/447593479 conn(0x558dfacdf8c0 msgr2=0x558dfacdfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 -- 172.21.15.44:0/447593479 shutdown_connections 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 -- 172.21.15.44:0/447593479 wait complete. 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 -- start start 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.940+0000 7f286e457740 1 Processor -- start 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 0x558dfaceb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacee4f0 0x558dfaceb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfac05e80 con 0x558dface3c50 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfac05b20 con 0x558dfacee4f0 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfabcb260 con 0x558dface3330 2024-09-13T17:37:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f286e457740 10 client.0 _ll_register_callbacks cb 0x558dfac00df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 0x558dfaceb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 0x558dfaceb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54256/0 (socket says 172.21.15.44:54256) 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 -- 172.21.15.44:0/288115495 learned_addr learned my addr 172.21.15.44:0/288115495 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f2868cce640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacee4f0 0x558dfaceb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f2868cce640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 -- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 msgr2=0x558dfacede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 -- 172.21.15.44:0/288115495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacee4f0 msgr2=0x558dfaceb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacee4f0 0x558dfaceb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 -- 172.21.15.44:0/288115495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286410f6a0 con 0x558dface3c50 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f2868cce640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f28694cf640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 0x558dfaceb140 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f2860012480 tx=0x7f2860012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273be5bdfde14e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f284cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.941+0000 7f284cff9640 1 -- 172.21.15.44:0/288115495 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600109e0 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dfac70680 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f284cff9640 1 -- 172.21.15.44:0/288115495 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f2860010b40 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f284cff9640 1 -- 172.21.15.44:0/288115495 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2860010d00 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 10 client.19107 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558dfac46330 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f284cff9640 1 -- 172.21.15.44:0/288115495 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558dfac70680 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f284cff9640 1 -- 172.21.15.44:0/288115495 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f286000d350 con 0x558dface3c50 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 10 client.19107 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 10 client.19107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 20 client.19107 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dfacfdf00 0x558dfad00320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 4 client.19107 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.942+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dfac3b5d0 con 0x558dfacfdf00 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.943+0000 7f2868cce640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dfacfdf00 0x558dfad00320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.943+0000 7f2868cce640 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dfacfdf00 0x558dfad00320 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d479f492536b1449 server_cookie=9a50c83e71d44766 in_seq=0 out_seq=0 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.943+0000 7f284cff9640 10 client.19107 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f284cff9640 1 -- 172.21.15.44:0/288115495 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558dfac3b5d0 con 0x558dfacfdf00 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f284cff9640 10 client.19107 handle_command_reply: tid=0 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 client.19107 shutdown 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 msgr2=0x558dfaceb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 0x558dfaceb140 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f2860012480 tx=0x7f2860012940 comp rx=0 tx=0).stop 2024-09-13T17:37:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 shutdown_connections 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dfacfdf00 0x558dfad00320 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacee4f0 0x558dfaceb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dface3c50 0x558dfaceb140 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 --2- 172.21.15.44:0/288115495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 >> 172.21.15.44:0/288115495 conn(0x558dfacdf8c0 msgr2=0x558dfacdeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f284cff9640 0 client.19107 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 shutdown_connections 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 1 -- 172.21.15.44:0/288115495 wait complete. 2024-09-13T17:37:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.944+0000 7f286e457740 20 client.19107 trim_cache size 0 max 16384 2024-09-13T17:37:49.955 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:49 smithi070 ceph-mon[34502]: pgmap v346: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:37:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 Processor -- start 2024-09-13T17:37:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 -- start start 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 0x558dfacdf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacdf8c0 0x558dface63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3330 0x558dface6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dface4880 con 0x558dfacedf80 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dface4170 con 0x558dface3330 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f286e457740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfacfc730 con 0x558dfacdf8c0 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 0x558dfacdf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 0x558dfacdf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54264/0 (socket says 172.21.15.44:54264) 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 -- 172.21.15.44:0/1874330745 learned_addr learned my addr 172.21.15.44:0/1874330745 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3330 0x558dface6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f28694cf640 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacdf8c0 0x558dface63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 -- 172.21.15.44:0/1874330745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacdf8c0 msgr2=0x558dface63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacdf8c0 0x558dface63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 -- 172.21.15.44:0/1874330745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3330 msgr2=0x558dface6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3330 0x558dface6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f2868cce640 1 -- 172.21.15.44:0/1874330745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dfabe8300 con 0x558dfacedf80 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.957+0000 7f28694cf640 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacdf8c0 0x558dface63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f2868cce640 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 0x558dfacdf300 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2854003a30 tx=0x7f2854000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a327bdbe81155d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f284ffff640 1 -- 172.21.15.44:0/1874330745 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540037f0 con 0x558dfacedf80 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f284ffff640 1 -- 172.21.15.44:0/1874330745 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2854031400 con 0x558dfacedf80 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f284ffff640 1 -- 172.21.15.44:0/1874330745 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854031c50 con 0x558dfacedf80 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 -- 172.21.15.44:0/1874330745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 msgr2=0x558dfacdf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 0x558dfacdf300 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2854003a30 tx=0x7f2854000bc0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 -- 172.21.15.44:0/1874330745 shutdown_connections 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dfacdf8c0 0x558dface63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dface3330 0x558dface6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 --2- 172.21.15.44:0/1874330745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacedf80 0x558dfacdf300 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 -- 172.21.15.44:0/1874330745 >> 172.21.15.44:0/1874330745 conn(0x558dface0240 msgr2=0x558dfad04c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 -- 172.21.15.44:0/1874330745 shutdown_connections 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.958+0000 7f286e457740 1 -- 172.21.15.44:0/1874330745 wait complete. 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 -- start start 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 Processor -- start 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 0x558dfacf5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacf18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacedf80 0x558dfacf1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfacf2510 con 0x558dfacdf8c0 2024-09-13T17:37:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfacf2680 con 0x558dfacedf80 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dfad54400 con 0x558dface3330 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f286e457740 10 client.0 _ll_register_callbacks cb 0x7f2854009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f2868cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 0x558dfacf5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f2868cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 0x558dfacf5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54276/0 (socket says 172.21.15.44:54276) 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.959+0000 7f2868cce640 1 -- 172.21.15.44:0/352451727 learned_addr learned my addr 172.21.15.44:0/352451727 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f28694cf640 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacf18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f2868cce640 1 -- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 msgr2=0x558dfacf18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f2868cce640 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacf18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f2868cce640 1 -- 172.21.15.44:0/352451727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacedf80 msgr2=0x558dfacf1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f2868cce640 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacedf80 0x558dfacf1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f2868cce640 1 -- 172.21.15.44:0/352451727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dfabee910 con 0x558dfacdf8c0 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f2868cce640 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 0x558dfacf5480 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2854004180 tx=0x7f28540045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d890d4b0b6b77ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:37:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f284cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f284cff9640 1 -- 172.21.15.44:0/352451727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854003d20 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f284cff9640 1 -- 172.21.15.44:0/352451727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f2854000e40 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2854045020 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f284cff9640 1 -- 172.21.15.44:0/352451727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2854003e80 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f286e457740 10 client.19113 fetch_fsmap learned FSMap version 25 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.960+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558dfac0d8e0 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f284cff9640 1 -- 172.21.15.44:0/352451727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2854045020 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f284cff9640 1 -- 172.21.15.44:0/352451727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f285404ba60 con 0x558dfacdf8c0 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f286e457740 10 client.19113 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f286e457740 10 client.19113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f286e457740 20 client.19113 populate_metadata read hostname 'smithi044' 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f286e457740 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dface3c50 0x558dfad574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:37:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f286e457740 4 client.19113 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T17:37:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558dfa93b300 con 0x558dface3c50 2024-09-13T17:37:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f28694cf640 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dface3c50 0x558dfad574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:37:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f28694cf640 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dface3c50 0x558dfad574e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378157563b153ebd server_cookie=f0ce1dfd94f12ceb in_seq=0 out_seq=0 2024-09-13T17:37:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.961+0000 7f284cff9640 10 client.19113 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f284cff9640 1 -- 172.21.15.44:0/352451727 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558dfa93b300 con 0x558dface3c50 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f284cff9640 10 client.19113 handle_command_reply: tid=0 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 client.19113 shutdown 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 msgr2=0x558dfacf5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 0x558dfacf5480 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2854004180 tx=0x7f28540045b0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 shutdown_connections 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x558dface3c50 0x558dfad574e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dface3330 0x558dfacf18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558dfacedf80 0x558dfacf1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f284cff9640 0 client.19113 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 --2- 172.21.15.44:0/352451727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfacdf8c0 0x558dfacf5480 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 >> 172.21.15.44:0/352451727 conn(0x558dface0240 msgr2=0x558dfacf9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 shutdown_connections 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 1 -- 172.21.15.44:0/352451727 wait complete. 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.974+0000 7f286e457740 20 client.19113 trim_cache size 0 max 16384 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-09-13T17:37:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-09-13T17:37:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28500a1200 msgr2=0x7f28500a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28500a1200 0x7f28500a36c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f285c099f90 tx=0x7f285c06c0e0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 msgr2=0x7f286419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:37:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 0x7f286419cd30 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f285c00e9d0 tx=0x7f285c00ee90 comp rx=0 tx=0).stop 2024-09-13T17:37:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 shutdown_connections 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f28500a1200 0x7f28500a36c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2864100ca0 0x7f2864198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410c660 0x7f286419cd30 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 --2- 172.21.15.44:0/3816664952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2864067d70 0x7f2864198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 >> 172.21.15.44:0/3816664952 conn(0x7f2864074940 msgr2=0x7f286410d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 shutdown_connections 2024-09-13T17:37:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-09-13T17:37:49.975+0000 7f286b733640 1 -- 172.21.15.44:0/3816664952 wait complete. 2024-09-13T17:37:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:49 smithi100 ceph-mon[35055]: pgmap v346: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:37:50.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:49 smithi044 ceph-mon[31175]: pgmap v346: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-13T17:37:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:51 smithi070 ceph-mon[34502]: pgmap v347: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:37:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:51 smithi100 ceph-mon[35055]: pgmap v347: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:37:52.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:51 smithi044 ceph-mon[31175]: pgmap v347: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:37:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:53 smithi070 ceph-mon[34502]: pgmap v348: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:37:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:53 smithi100 ceph-mon[35055]: pgmap v348: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:37:54.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:53 smithi044 ceph-mon[31175]: pgmap v348: 129 pgs: 129 active+clean; 1.8 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-13T17:37:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:55 smithi070 ceph-mon[34502]: pgmap v349: 129 pgs: 129 active+clean; 2.0 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:37:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:55 smithi100 ceph-mon[35055]: pgmap v349: 129 pgs: 129 active+clean; 2.0 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:37:56.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:55 smithi044 ceph-mon[31175]: pgmap v349: 129 pgs: 129 active+clean; 2.0 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-13T17:37:57.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:57 smithi070 ceph-mon[34502]: pgmap v350: 129 pgs: 129 active+clean; 2.0 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:58.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:57 smithi100 ceph-mon[35055]: pgmap v350: 129 pgs: 129 active+clean; 2.0 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:58.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:57 smithi044 ceph-mon[31175]: pgmap v350: 129 pgs: 129 active+clean; 2.0 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-13T17:37:58.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:58 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:37:59.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:58 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:37:59.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:58 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:37:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:59 smithi070 ceph-mon[34502]: pgmap v351: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:37:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:59 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:37:59.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:59 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:37:59.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:59 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:37:59.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:59 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:37:59.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:37:59 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:59 smithi100 ceph-mon[35055]: pgmap v351: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:38:00.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:59 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:59 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:59 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:38:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:59 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:38:00.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:37:59 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:00.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:59 smithi044 ceph-mon[31175]: pgmap v351: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-13T17:38:00.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:59 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:00.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:59 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:00.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:59 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:38:00.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:59 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:38:00.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:37:59 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:38:01.145 INFO:tasks.workunit.client.0.smithi044.stdout:561/600: open fd = 3 2024-09-13T17:38:01.145 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.145 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:562/600: open fd = 3 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:563/600: open fd = 3 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:564/600: open fd = 3 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.146 INFO:tasks.workunit.client.0.smithi044.stdout:565/600: open fd = 3 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:566/600: open fd = 3 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:567/600: open fd = 3 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:568/600: open fd = 3 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.147 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:569/600: open fd = 3 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:570/600: open fd = 3 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:571/600: open fd = 3 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:572/600: open fd = 3 2024-09-13T17:38:01.148 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:573/600: open fd = 3 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:574/600: open fd = 3 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:575/600: open fd = 3 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.149 INFO:tasks.workunit.client.0.smithi044.stdout:576/600: open fd = 3 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:577/600: open fd = 3 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:578/600: open fd = 3 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:579/600: open fd = 3 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.150 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:580/600: open fd = 3 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:581/600: open fd = 3 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:582/600: open fd = 3 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:583/600: open fd = 3 2024-09-13T17:38:01.151 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:584/600: open fd = 3 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:585/600: open fd = 3 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:586/600: open fd = 3 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.152 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:587/600: open fd = 3 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:588/600: open fd = 3 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:589/600: open fd = 3 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:590/600: open fd = 3 2024-09-13T17:38:01.153 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:591/600: open fd = 3 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:592/600: open fd = 3 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:593/600: open fd = 3 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.154 INFO:tasks.workunit.client.0.smithi044.stdout:594/600: open fd = 3 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:595/600: open fd = 3 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:596/600: open fd = 3 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.155 INFO:tasks.workunit.client.0.smithi044.stdout:597/600: open fd = 3 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:598/600: open fd = 3 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:599/600: open fd = 3 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:600/600: open fd = 3 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.156 INFO:tasks.workunit.client.0.smithi044.stdout:write ret = 32 2024-09-13T17:38:01.157 INFO:tasks.workunit.client.0.smithi044.stdout:pread ret = 64 2024-09-13T17:38:01.157 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2024-09-13T17:38:01.158 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2024-09-13T17:38:01.158 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T17:38:01.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T17:38:01.202 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-13T17:38:01.202 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T17:38:01.531 DEBUG:teuthology.parallel:result is None 2024-09-13T17:38:01.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T17:38:01.568 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T17:38:01.568 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-13T17:38:01.580 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:38:01.617 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-13T17:38:01.626 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-13T17:38:01.626 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T17:38:01.627 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:38:01.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-13T17:38:01.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-13T17:38:01.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-13T17:38:01.815 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:01 smithi044 ceph-mon[31175]: pgmap v352: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:38:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 -- 172.21.15.44:0/462624384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c103f50 msgr2=0x7f7b6c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 --2- 172.21.15.44:0/462624384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c103f50 0x7f7b6c10baa0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009cd0 tx=0x7f7b6402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 -- 172.21.15.44:0/462624384 shutdown_connections 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 --2- 172.21.15.44:0/462624384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ff7b0 0x7f7b6c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 --2- 172.21.15.44:0/462624384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c103f50 0x7f7b6c10baa0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 --2- 172.21.15.44:0/462624384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 -- 172.21.15.44:0/462624384 >> 172.21.15.44:0/462624384 conn(0x7f7b6c0fa890 msgr2=0x7f7b6c0fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 -- 172.21.15.44:0/462624384 shutdown_connections 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.894+0000 7f7b73287640 1 -- 172.21.15.44:0/462624384 wait complete. 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 Processor -- start 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 -- start start 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c103f50 0x7f7b6c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c19aac0 0x7f7b6c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19b4b0 con 0x7f7b6c0fee00 2024-09-13T17:38:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19b620 con 0x7f7b6c19aac0 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b73287640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19b790 con 0x7f7b6c103f50 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41262/0 (socket says 172.21.15.44:41262) 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 -- 172.21.15.44:0/1942892001 learned_addr learned my addr 172.21.15.44:0/1942892001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71824640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c19aac0 0x7f7b6c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b70822640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c103f50 0x7f7b6c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 -- 172.21.15.44:0/1942892001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c103f50 msgr2=0x7f7b6c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c103f50 0x7f7b6c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 -- 172.21.15.44:0/1942892001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c19aac0 msgr2=0x7f7b6c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c19aac0 0x7f7b6c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.895+0000 7f7b71023640 1 -- 172.21.15.44:0/1942892001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009900 con 0x7f7b6c0fee00 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.896+0000 7f7b71023640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c196190 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800d950 tx=0x7f7b6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.896+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68009c30 con 0x7f7b6c0fee00 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.896+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b68009d90 con 0x7f7b6c0fee00 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.896+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c1a2d70 con 0x7f7b6c0fee00 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.896+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800f620 con 0x7f7b6c0fee00 2024-09-13T17:38:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.896+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c1a31d0 con 0x7f7b6c0fee00 2024-09-13T17:38:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.897+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c1002d0 con 0x7f7b6c0fee00 2024-09-13T17:38:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.898+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7b680109f0 con 0x7f7b6c0fee00 2024-09-13T17:38:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.898+0000 7f7b627fc640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7b580a1200 0x7f7b580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.899+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f7b680ccc50 con 0x7f7b6c0fee00 2024-09-13T17:38:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.899+0000 7f7b70822640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7b580a1200 0x7f7b580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.899+0000 7f7b70822640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7b580a1200 0x7f7b580a36c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005fd0 tx=0x7f7b5c005850 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:01.903+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7b680d2020 con 0x7f7b6c0fee00 2024-09-13T17:38:01.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:01 smithi070 ceph-mon[34502]: pgmap v352: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:38:02.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:01 smithi100 ceph-mon[35055]: pgmap v352: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:38:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.093+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b6c0ff7b0 con 0x7f7b580a1200 2024-09-13T17:38:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.099+0000 7f7b627fc640 1 -- 172.21.15.44:0/1942892001 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f7b6c0ff7b0 con 0x7f7b580a1200 2024-09-13T17:38:02.102 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-13T00:00:00", "created": "2024-09-13T17:26:51", "first": "2024-09-13T17:27:00", "last": "2024-09-13T17:38:00", "last_pruned": "2024-09-13T17:38:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-09-13T17:38:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7b580a1200 msgr2=0x7f7b580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7b580a1200 0x7f7b580a36c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005fd0 tx=0x7f7b5c005850 comp rx=0 tx=0).stop 2024-09-13T17:38:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 msgr2=0x7f7b6c196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c196190 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800d950 tx=0x7f7b6800de10 comp rx=0 tx=0).stop 2024-09-13T17:38:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 shutdown_connections 2024-09-13T17:38:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7b580a1200 0x7f7b580a36c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c103f50 0x7f7b6c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b6c19aac0 0x7f7b6c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 --2- 172.21.15.44:0/1942892001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b6c0fee00 0x7f7b6c196190 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.101+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 >> 172.21.15.44:0/1942892001 conn(0x7f7b6c0fa890 msgr2=0x7f7b6c0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.102+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 shutdown_connections 2024-09-13T17:38:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.102+0000 7f7b73287640 1 -- 172.21.15.44:0/1942892001 wait complete. 2024-09-13T17:38:02.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-13T17:38:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 -- 172.21.15.44:0/1042297808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 msgr2=0x7f2e0c0ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 --2- 172.21.15.44:0/1042297808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c0ff200 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009930 tx=0x7f2dfc02a420 comp rx=0 tx=0).stop 2024-09-13T17:38:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 -- 172.21.15.44:0/1042297808 shutdown_connections 2024-09-13T17:38:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 --2- 172.21.15.44:0/1042297808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0ff7d0 0x7f2e0c103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 --2- 172.21.15.44:0/1042297808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e0c103f70 0x7f2e0c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 --2- 172.21.15.44:0/1042297808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c0ff200 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 -- 172.21.15.44:0/1042297808 >> 172.21.15.44:0/1042297808 conn(0x7f2e0c0fa8f0 msgr2=0x7f2e0c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 -- 172.21.15.44:0/1042297808 shutdown_connections 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 -- 172.21.15.44:0/1042297808 wait complete. 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.257+0000 7f2e11325640 1 Processor -- start 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 -- start start 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c1950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c103f70 0x7f2e0c195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e0c1999f0 0x7f2e0c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c19a3e0 con 0x7f2e0c0fee20 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c19a550 con 0x7f2e0c1999f0 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e11325640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c19a6c0 con 0x7f2e0c103f70 2024-09-13T17:38:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c1950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c1950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41268/0 (socket says 172.21.15.44:41268) 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 -- 172.21.15.44:0/3180345247 learned_addr learned my addr 172.21.15.44:0/3180345247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0b7fe640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e0c1999f0 0x7f2e0c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0a7fc640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c103f70 0x7f2e0c195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 -- 172.21.15.44:0/3180345247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c103f70 msgr2=0x7f2e0c195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c103f70 0x7f2e0c195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 -- 172.21.15.44:0/3180345247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e0c1999f0 msgr2=0x7f2e0c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e0c1999f0 0x7f2e0c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.258+0000 7f2e0affd640 1 -- 172.21.15.44:0/3180345247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc009560 con 0x7f2e0c0fee20 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.259+0000 7f2e0affd640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c1950c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc02a940 tx=0x7f2dfc005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.259+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc004b50 con 0x7f2e0c0fee20 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.259+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dfc004cb0 con 0x7f2e0c0fee20 2024-09-13T17:38:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.259+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c1a1ca0 con 0x7f2e0c0fee20 2024-09-13T17:38:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.259+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc03c890 con 0x7f2e0c0fee20 2024-09-13T17:38:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.259+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1a2120 con 0x7f2e0c0fee20 2024-09-13T17:38:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.260+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c1002f0 con 0x7f2e0c0fee20 2024-09-13T17:38:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.261+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2dfc0035f0 con 0x7f2e0c0fee20 2024-09-13T17:38:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.261+0000 7f2e03fff640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2df80a1280 0x7f2df80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.262+0000 7f2e0a7fc640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2df80a1280 0x7f2df80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.262+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f2dfc0eddf0 con 0x7f2e0c0fee20 2024-09-13T17:38:02.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.262+0000 7f2e0a7fc640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2df80a1280 0x7f2df80a3740 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c196660 tx=0x7f2df4005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:02.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.266+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2dfc0b6f20 con 0x7f2e0c0fee20 2024-09-13T17:38:02.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.457+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 --> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2e0c0ff7d0 con 0x7f2df80a1280 2024-09-13T17:38:02.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.464+0000 7f2e03fff640 1 -- 172.21.15.44:0/3180345247 <== mgr.15468 v2:172.21.15.44:6800/4124099065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2e0c0ff7d0 con 0x7f2df80a1280 2024-09-13T17:38:02.466 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1m 6m3h 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.466+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2df80a1280 msgr2=0x7f2df80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.466+0000 7f2e11325640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2df80a1280 0x7f2df80a3740 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c196660 tx=0x7f2df4005f90 comp rx=0 tx=0).stop 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.466+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 msgr2=0x7f2e0c1950c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.466+0000 7f2e11325640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c1950c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc02a940 tx=0x7f2dfc005530 comp rx=0 tx=0).stop 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 shutdown_connections 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f2df80a1280 0x7f2df80a3740 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c103f70 0x7f2e0c195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e0c1999f0 0x7f2e0c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 --2- 172.21.15.44:0/3180345247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c0fee20 0x7f2e0c1950c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 >> 172.21.15.44:0/3180345247 conn(0x7f2e0c0fa8f0 msgr2=0x7f2e0c0fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 shutdown_connections 2024-09-13T17:38:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:02.467+0000 7f2e11325640 1 -- 172.21.15.44:0/3180345247 wait complete. 2024-09-13T17:38:02.480 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-13T17:38:02.489 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-13T17:38:02.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:02 smithi044 ceph-mon[31175]: from='client.19119 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:38:02.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:02 smithi070 ceph-mon[34502]: from='client.19119 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:38:03.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:02 smithi100 ceph-mon[35055]: from='client.19119 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:38:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:03 smithi070 ceph-mon[34502]: from='client.19125 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:38:03.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:03 smithi070 ceph-mon[34502]: pgmap v353: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:38:04.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:03 smithi100 ceph-mon[35055]: from='client.19125 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:38:04.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:03 smithi100 ceph-mon[35055]: pgmap v353: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:38:04.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:03 smithi044 ceph-mon[31175]: from='client.19125 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T17:38:04.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:03 smithi044 ceph-mon[31175]: pgmap v353: 129 pgs: 129 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-13T17:38:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:05 smithi070 ceph-mon[34502]: pgmap v354: 129 pgs: 129 active+clean; 2.3 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 96 KiB/s wr, 37 op/s 2024-09-13T17:38:05.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:05 smithi070 ceph-mon[34502]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T17:38:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:05 smithi100 ceph-mon[35055]: pgmap v354: 129 pgs: 129 active+clean; 2.3 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 96 KiB/s wr, 37 op/s 2024-09-13T17:38:06.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:05 smithi100 ceph-mon[35055]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T17:38:06.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:05 smithi044 ceph-mon[31175]: pgmap v354: 129 pgs: 129 active+clean; 2.3 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 96 KiB/s wr, 37 op/s 2024-09-13T17:38:06.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:05 smithi044 ceph-mon[31175]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T17:38:07.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:07 smithi070 ceph-mon[34502]: pgmap v356: 129 pgs: 129 active+clean; 2.3 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-09-13T17:38:08.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:07 smithi100 ceph-mon[35055]: pgmap v356: 129 pgs: 129 active+clean; 2.3 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-09-13T17:38:08.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:07 smithi044 ceph-mon[31175]: pgmap v356: 129 pgs: 129 active+clean; 2.3 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-09-13T17:38:09.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:09 smithi070 ceph-mon[34502]: pgmap v357: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-13T17:38:10.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T17:38:10.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:09 smithi100 ceph-mon[35055]: pgmap v357: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-13T17:38:10.046 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:09 smithi044 ceph-mon[31175]: pgmap v357: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-13T17:38:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 -- 172.21.15.44:0/59667395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f300ff380 msgr2=0x7f8f3010c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 --2- 172.21.15.44:0/59667395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f300ff380 0x7f8f3010c0d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f2002a410 comp rx=0 tx=0).stop 2024-09-13T17:38:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 -- 172.21.15.44:0/59667395 shutdown_connections 2024-09-13T17:38:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 --2- 172.21.15.44:0/59667395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f3010c610 0x7f8f3010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 --2- 172.21.15.44:0/59667395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f300ff380 0x7f8f3010c0d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 --2- 172.21.15.44:0/59667395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f300fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 -- 172.21.15.44:0/59667395 >> 172.21.15.44:0/59667395 conn(0x7f8f300fa930 msgr2=0x7f8f300fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 -- 172.21.15.44:0/59667395 shutdown_connections 2024-09-13T17:38:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 -- 172.21.15.44:0/59667395 wait complete. 2024-09-13T17:38:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 Processor -- start 2024-09-13T17:38:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.135+0000 7f8f3824f640 1 -- start start 2024-09-13T17:38:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f3824f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f30198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f3824f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f300ff380 0x7f8f30198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f3824f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3010c610 0x7f8f3019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f3824f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3019d350 con 0x7f8f3010c610 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f3824f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3019d4c0 con 0x7f8f300fea60 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f3824f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3019d680 con 0x7f8f300ff380 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f30198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f30198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:46136/0 (socket says 172.21.15.44:46136) 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 -- 172.21.15.44:0/2731076499 learned_addr learned my addr 172.21.15.44:0/2731076499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f357ea640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f300ff380 0x7f8f30198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f367ec640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3010c610 0x7f8f3019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 -- 172.21.15.44:0/2731076499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f300ff380 msgr2=0x7f8f30198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f300ff380 0x7f8f30198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 -- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3010c610 msgr2=0x7f8f3019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3010c610 0x7f8f3019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 -- 172.21.15.44:0/2731076499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f20009560 con 0x7f8f300fea60 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.136+0000 7f8f35feb640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f30198420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d950 tx=0x7f8f2c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.137+0000 7f8f26ffd640 1 -- 172.21.15.44:0/2731076499 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c009c30 con 0x7f8f300fea60 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.137+0000 7f8f26ffd640 1 -- 172.21.15.44:0/2731076499 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f2c009d90 con 0x7f8f300fea60 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.137+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3019d960 con 0x7f8f300fea60 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.137+0000 7f8f26ffd640 1 -- 172.21.15.44:0/2731076499 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00f620 con 0x7f8f300fea60 2024-09-13T17:38:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.137+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f301a51e0 con 0x7f8f300fea60 2024-09-13T17:38:10.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.139+0000 7f8f26ffd640 1 -- 172.21.15.44:0/2731076499 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f2c0109f0 con 0x7f8f300fea60 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.139+0000 7f8f3b04b740 1 Processor -- start 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 -- start start 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead57e10 0x55cfead581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead62a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeacc2f20 con 0x55cfead62fd0 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeacc06b0 con 0x55cfead58730 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeacbbfa0 con 0x55cfead57e10 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38216/0 (socket says 172.21.15.44:38216) 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 -- 172.21.15.44:0/3003871173 learned_addr learned my addr 172.21.15.44:0/3003871173 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f35feb640 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead62a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f367ec640 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead57e10 0x55cfead581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f26ffd640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8f1c0a1200 0x7f8f1c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 -- 172.21.15.44:0/3003871173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead57e10 msgr2=0x55cfead581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead57e10 0x55cfead581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 -- 172.21.15.44:0/3003871173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 msgr2=0x55cfead62a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead62a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f357ea640 1 -- 172.21.15.44:0/3003871173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfeac76840 con 0x55cfead62fd0 2024-09-13T17:38:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.140+0000 7f8f367ec640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8f1c0a1200 0x7f8f1c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.141+0000 7f8f26ffd640 1 -- 172.21.15.44:0/2731076499 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f8f2c0ccc60 con 0x7f8f300fea60 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.141+0000 7f8f357ea640 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead653c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002a930 tx=0x7f8f20005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea47ddf27f94662 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.141+0000 7f8f1bfff640 1 -- 172.21.15.44:0/3003871173 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20004230 con 0x55cfead62fd0 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.141+0000 7f8f1bfff640 1 -- 172.21.15.44:0/3003871173 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f20031df0 con 0x55cfead62fd0 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.141+0000 7f8f1bfff640 1 -- 172.21.15.44:0/3003871173 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20003be0 con 0x55cfead62fd0 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f367ec640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8f1c0a1200 0x7f8f1c0a36c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8f30068350 tx=0x7f8f28004a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3003871173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 msgr2=0x55cfead653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead653c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002a930 tx=0x7f8f20005c00 comp rx=0 tx=0).stop 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3003871173 shutdown_connections 2024-09-13T17:38:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead653c0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead62a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3003871173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead57e10 0x55cfead581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3003871173 >> 172.21.15.44:0/3003871173 conn(0x55cfead54380 msgr2=0x55cfead54760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3003871173 shutdown_connections 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3003871173 wait complete. 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 -- start start 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.142+0000 7f8f3b04b740 1 Processor -- start 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead57e10 0x55cfead62960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeac5d2a0 con 0x55cfead62fd0 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfea994430 con 0x55cfead57e10 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeac3f680 con 0x55cfead58730 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f3b04b740 10 client.0 _ll_register_callbacks cb 0x55cfeacb5610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38228/0 (socket says 172.21.15.44:38228) 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 -- 172.21.15.44:0/3828878583 learned_addr learned my addr 172.21.15.44:0/3828878583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f35feb640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead57e10 0x55cfead62960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f357ea640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 -- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 msgr2=0x55cfead5fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 -- 172.21.15.44:0/3828878583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead57e10 msgr2=0x55cfead62960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead57e10 0x55cfead62960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 -- 172.21.15.44:0/3828878583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3010f630 con 0x55cfead62fd0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f357ea640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f367ec640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead601d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8f28013010 tx=0x7f8f28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4f98b156265b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f197fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f197fa640 1 -- 172.21.15.44:0/3828878583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2801ed40 con 0x55cfead62fd0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.143+0000 7f8f197fa640 1 -- 172.21.15.44:0/3828878583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f8f28013e40 con 0x55cfead62fd0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfea9af4e0 con 0x55cfead62fd0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f197fa640 1 -- 172.21.15.44:0/3828878583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f2801eea0 con 0x55cfead62fd0 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 10 client.19137 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cfeacba620 con 0x55cfead62fd0 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f197fa640 1 -- 172.21.15.44:0/3828878583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cfea9af4e0 con 0x55cfead62fd0 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f197fa640 1 -- 172.21.15.44:0/3828878583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f2801f5c0 con 0x55cfead62fd0 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 10 client.19137 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 10 client.19137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 20 client.19137 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead72e50 0x55cfead75270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 4 client.19137 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.144+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfea9c07e0 con 0x55cfead72e50 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.145+0000 7f8f35feb640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead72e50 0x55cfead75270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.145+0000 7f8f35feb640 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead72e50 0x55cfead75270 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6084be23d75b78 server_cookie=24fa96e9eba9c539 in_seq=0 out_seq=0 2024-09-13T17:38:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.145+0000 7f8f197fa640 10 client.19137 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.145+0000 7f8f197fa640 1 -- 172.21.15.44:0/3828878583 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cfea9c07e0 con 0x55cfead72e50 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.145+0000 7f8f197fa640 10 client.19137 handle_command_reply: tid=0 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 client.19137 shutdown 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 msgr2=0x55cfead601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead601d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8f28013010 tx=0x7f8f28004ab0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 shutdown_connections 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead72e50 0x55cfead75270 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead62fd0 0x55cfead601d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f197fa640 0 client.19137 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/3828878583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead57e10 0x55cfead62960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 >> 172.21.15.44:0/3828878583 conn(0x55cfead54380 msgr2=0x55cfead5a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 shutdown_connections 2024-09-13T17:38:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 1 -- 172.21.15.44:0/3828878583 wait complete. 2024-09-13T17:38:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.146+0000 7f8f3b04b740 20 client.19137 trim_cache size 0 max 16384 2024-09-13T17:38:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 Processor -- start 2024-09-13T17:38:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 -- start start 2024-09-13T17:38:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead54380 0x55cfead53d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead69d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead6a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfead57b90 con 0x55cfead57e10 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.160+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfead58fb0 con 0x55cfead58730 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfead607d0 con 0x55cfead54380 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead69d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead69d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38242/0 (socket says 172.21.15.44:38242) 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 -- 172.21.15.44:0/2858206218 learned_addr learned my addr 172.21.15.44:0/2858206218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f367ec640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead6a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 -- 172.21.15.44:0/2858206218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead54380 msgr2=0x55cfead53d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f35feb640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead54380 0x55cfead53d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead54380 0x55cfead53d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 -- 172.21.15.44:0/2858206218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 msgr2=0x55cfead6a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead6a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 -- 172.21.15.44:0/2858206218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfeac5c790 con 0x55cfead57e10 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f367ec640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead6a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f35feb640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead54380 0x55cfead53d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f357ea640 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead69d30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002a930 tx=0x7f8f2002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6952b2a643544a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f24ff9640 1 -- 172.21.15.44:0/2858206218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20031960 con 0x55cfead57e10 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f24ff9640 1 -- 172.21.15.44:0/2858206218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f2004d070 con 0x55cfead57e10 2024-09-13T17:38:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.161+0000 7f8f24ff9640 1 -- 172.21.15.44:0/2858206218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20045400 con 0x55cfead57e10 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 -- 172.21.15.44:0/2858206218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 msgr2=0x55cfead69d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead69d30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002a930 tx=0x7f8f2002a410 comp rx=0 tx=0).stop 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 -- 172.21.15.44:0/2858206218 shutdown_connections 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead58730 0x55cfead6a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead69d30 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/2858206218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead54380 0x55cfead53d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 -- 172.21.15.44:0/2858206218 >> 172.21.15.44:0/2858206218 conn(0x55cfead62ac0 msgr2=0x55cfead5e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 -- 172.21.15.44:0/2858206218 shutdown_connections 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 -- 172.21.15.44:0/2858206218 wait complete. 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 -- start start 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.162+0000 7f8f3b04b740 1 Processor -- start 2024-09-13T17:38:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead54380 0x55cfead5c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead5ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfead6a630 con 0x55cfead57e10 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeadc9370 con 0x55cfead54380 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeadc94b0 con 0x55cfead58730 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 10 client.0 _ll_register_callbacks cb 0x7f8f28009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead5ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead5ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38244/0 (socket says 172.21.15.44:38244) 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 -- 172.21.15.44:0/922167110 learned_addr learned my addr 172.21.15.44:0/922167110 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 -- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 msgr2=0x55cfead5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 -- 172.21.15.44:0/922167110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead54380 msgr2=0x55cfead5c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f35feb640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead54380 0x55cfead5c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead54380 0x55cfead5c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 -- 172.21.15.44:0/922167110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfeac76840 con 0x55cfead57e10 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f35feb640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead54380 0x55cfead5c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f357ea640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead5ccf0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f20031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c1cb9dbd372228 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f197fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f197fa640 1 -- 172.21.15.44:0/922167110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20004440 con 0x55cfead57e10 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f197fa640 1 -- 172.21.15.44:0/922167110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f8f2003d450 con 0x55cfead57e10 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.163+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f20031df0 con 0x55cfead57e10 2024-09-13T17:38:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f197fa640 1 -- 172.21.15.44:0/922167110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f2003d610 con 0x55cfead57e10 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 10 client.19143 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cfeac81ea0 con 0x55cfead57e10 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f197fa640 1 -- 172.21.15.44:0/922167110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f20031df0 con 0x55cfead57e10 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f197fa640 1 -- 172.21.15.44:0/922167110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f20043020 con 0x55cfead57e10 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 10 client.19143 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 10 client.19143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 20 client.19143 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead62fd0 0x55cfeadcc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 4 client.19143 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cfeaa13e30 con 0x55cfead62fd0 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.164+0000 7f8f35feb640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead62fd0 0x55cfeadcc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f35feb640 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead62fd0 0x55cfeadcc760 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddd19c4e61afae2 server_cookie=e361fc22de5c44d8 in_seq=0 out_seq=0 2024-09-13T17:38:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f197fa640 10 client.19143 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f197fa640 1 -- 172.21.15.44:0/922167110 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cfeaa13e30 con 0x55cfead62fd0 2024-09-13T17:38:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f197fa640 10 client.19143 handle_command_reply: tid=0 2024-09-13T17:38:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 client.19143 shutdown 2024-09-13T17:38:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 msgr2=0x55cfead5ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead5ccf0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f20031b90 comp rx=0 tx=0).stop 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 shutdown_connections 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55cfead62fd0 0x55cfeadcc760 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfead58730 0x55cfead5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfead57e10 0x55cfead5ccf0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f197fa640 0 client.19143 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 --2- 172.21.15.44:0/922167110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cfead54380 0x55cfead5c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 >> 172.21.15.44:0/922167110 conn(0x55cfead62ac0 msgr2=0x55cfead59870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 shutdown_connections 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 1 -- 172.21.15.44:0/922167110 wait complete. 2024-09-13T17:38:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.165+0000 7f8f3b04b740 20 client.19143 trim_cache size 0 max 16384 2024-09-13T17:38:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:38:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-09-13T17:38:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-09-13T17:38:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:38:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8f1c0a1200 msgr2=0x7f8f1c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8f1c0a1200 0x7f8f1c0a36c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8f30068350 tx=0x7f8f28004a40 comp rx=0 tx=0).stop 2024-09-13T17:38:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 msgr2=0x7f8f30198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f30198420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d950 tx=0x7f8f2c00de10 comp rx=0 tx=0).stop 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 shutdown_connections 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f8f1c0a1200 0x7f8f1c0a36c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f300ff380 0x7f8f30198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3010c610 0x7f8f3019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 --2- 172.21.15.44:0/2731076499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f300fea60 0x7f8f30198420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 >> 172.21.15.44:0/2731076499 conn(0x7f8f300fa930 msgr2=0x7f8f300fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 shutdown_connections 2024-09-13T17:38:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.166+0000 7f8f3824f640 1 -- 172.21.15.44:0/2731076499 wait complete. 2024-09-13T17:38:10.182 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T17:38:10.183 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:38:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 -- 172.21.15.44:0/3819522679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff65410a370 msgr2=0x7ff65410c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/3819522679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff65410a370 0x7ff65410c7c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009ca0 tx=0x7ff64c02a950 comp rx=0 tx=0).stop 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 -- 172.21.15.44:0/3819522679 shutdown_connections 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/3819522679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6541018c0 0x7ff654109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/3819522679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff65410a370 0x7ff65410c7c0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/3819522679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff654100fa0 0x7ff654101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 -- 172.21.15.44:0/3819522679 >> 172.21.15.44:0/3819522679 conn(0x7ff654075c60 msgr2=0x7ff6540fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 -- 172.21.15.44:0/3819522679 shutdown_connections 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.323+0000 7ff65ad3e640 1 -- 172.21.15.44:0/3819522679 wait complete. 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 Processor -- start 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 -- start start 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 0x7ff6540febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6541018c0 0x7ff6540ff130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65410a370 0x7ff6540ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541a4510 con 0x7ff654100fa0 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541a4680 con 0x7ff65410a370 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff65ad3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541a4840 con 0x7ff6541018c0 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 0x7ff6540febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 0x7ff6540febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38258/0 (socket says 172.21.15.44:38258) 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 -- 172.21.15.44:0/2545568982 learned_addr learned my addr 172.21.15.44:0/2545568982 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff653fff640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6541018c0 0x7ff6540ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff6592db640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65410a370 0x7ff6540ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 -- 172.21.15.44:0/2545568982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6541018c0 msgr2=0x7ff6540ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6541018c0 0x7ff6540ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 -- 172.21.15.44:0/2545568982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65410a370 msgr2=0x7ff6540ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65410a370 0x7ff6540ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.324+0000 7ff658ada640 1 -- 172.21.15.44:0/2545568982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c009900 con 0x7ff654100fa0 2024-09-13T17:38:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.325+0000 7ff658ada640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 0x7ff6540febf0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff64400d950 tx=0x7ff64400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.325+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644009a30 con 0x7ff654100fa0 2024-09-13T17:38:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.325+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6440044b0 con 0x7ff654100fa0 2024-09-13T17:38:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.325+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6541a4b20 con 0x7ff654100fa0 2024-09-13T17:38:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.325+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440047a0 con 0x7ff654100fa0 2024-09-13T17:38:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.325+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6541a5080 con 0x7ff654100fa0 2024-09-13T17:38:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.326+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654067dc0 con 0x7ff654100fa0 2024-09-13T17:38:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.327+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff644004020 con 0x7ff654100fa0 2024-09-13T17:38:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.328+0000 7ff651ffb640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff6400a1280 0x7ff6400a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.328+0000 7ff653fff640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff6400a1280 0x7ff6400a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.328+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7ff6440ccb60 con 0x7ff654100fa0 2024-09-13T17:38:10.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.328+0000 7ff653fff640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff6400a1280 0x7ff6400a3740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c00b510 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.332+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff644095b80 con 0x7ff654100fa0 2024-09-13T17:38:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.540+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff654105420 con 0x7ff654100fa0 2024-09-13T17:38:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.541+0000 7ff651ffb640 1 -- 172.21.15.44:0/2545568982 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7ff6440991a0 con 0x7ff654100fa0 2024-09-13T17:38:10.543 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:10.543 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:38:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:38:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff6400a1280 msgr2=0x7ff6400a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff6400a1280 0x7ff6400a3740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c00b510 comp rx=0 tx=0).stop 2024-09-13T17:38:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 msgr2=0x7ff6540febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 0x7ff6540febf0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff64400d950 tx=0x7ff64400de10 comp rx=0 tx=0).stop 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 shutdown_connections 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff6400a1280 0x7ff6400a3740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6541018c0 0x7ff6540ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65410a370 0x7ff6540ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 --2- 172.21.15.44:0/2545568982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff654100fa0 0x7ff6540febf0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 >> 172.21.15.44:0/2545568982 conn(0x7ff654075c60 msgr2=0x7ff6540fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 shutdown_connections 2024-09-13T17:38:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.543+0000 7ff65ad3e640 1 -- 172.21.15.44:0/2545568982 wait complete. 2024-09-13T17:38:10.556 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T17:38:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 -- 172.21.15.44:0/49284825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6664100e50 msgr2=0x7f6664101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 --2- 172.21.15.44:0/49284825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6664100e50 0x7f6664101230 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6654009900 tx=0x7f665402a410 comp rx=0 tx=0).stop 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 -- 172.21.15.44:0/49284825 shutdown_connections 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 --2- 172.21.15.44:0/49284825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f666410a350 0x7f666410c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 --2- 172.21.15.44:0/49284825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6664101800 0x7f6664109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 --2- 172.21.15.44:0/49284825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6664100e50 0x7f6664101230 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 -- 172.21.15.44:0/49284825 >> 172.21.15.44:0/49284825 conn(0x7f66640fa930 msgr2=0x7f66640fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 -- 172.21.15.44:0/49284825 shutdown_connections 2024-09-13T17:38:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.659+0000 7f6669027640 1 -- 172.21.15.44:0/49284825 wait complete. 2024-09-13T17:38:10.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 Processor -- start 2024-09-13T17:38:10.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 -- start start 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6664100e50 0x7f6664196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6664101800 0x7f6664196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 0x7f666419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666419b080 con 0x7f666410a350 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666419b1f0 con 0x7f6664100e50 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6669027640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666419b3b0 con 0x7f6664101800 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f666359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 0x7f666419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f666359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 0x7f666419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38268/0 (socket says 172.21.15.44:38268) 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f666359e640 1 -- 172.21.15.44:0/3210437207 learned_addr learned my addr 172.21.15.44:0/3210437207 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6664100e50 0x7f6664196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.660+0000 7f665bfff640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6664101800 0x7f6664196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f666359e640 1 -- 172.21.15.44:0/3210437207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6664101800 msgr2=0x7f6664196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f666359e640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6664101800 0x7f6664196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f666359e640 1 -- 172.21.15.44:0/3210437207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6664100e50 msgr2=0x7f6664196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f666359e640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6664100e50 0x7f6664196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f666359e640 1 -- 172.21.15.44:0/3210437207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009560 con 0x7f666410a350 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f665bfff640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6664101800 0x7f6664196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f666359e640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 0x7f666419aa80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f665c00eaa0 tx=0x7f665c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f6660d99640 1 -- 172.21.15.44:0/3210437207 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00cdb0 con 0x7f666410a350 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f6660d99640 1 -- 172.21.15.44:0/3210437207 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f665c005020 con 0x7f666410a350 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666419b690 con 0x7f666410a350 2024-09-13T17:38:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f6660d99640 1 -- 172.21.15.44:0/3210437207 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c0116d0 con 0x7f666410a350 2024-09-13T17:38:10.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.661+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66641a2f10 con 0x7f666410a350 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.663+0000 7f6660d99640 1 -- 172.21.15.44:0/3210437207 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f665c005190 con 0x7f666410a350 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.663+0000 7f666be23740 1 Processor -- start 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.663+0000 7f666be23740 1 -- start start 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666be23740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666be23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 0x5615935be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666be23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666be23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561593519cc0 con 0x5615935affb0 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666be23740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561593517450 con 0x5615935af690 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666be23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561593512d40 con 0x5615935bed60 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 0x5615935be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 0x5615935be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38282/0 (socket says 172.21.15.44:38282) 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 -- 172.21.15.44:0/3898613757 learned_addr learned my addr 172.21.15.44:0/3898613757 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f665bfff640 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f665bfff640 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6660d99640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f66500a1200 0x7f66500a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 -- 172.21.15.44:0/3898613757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 msgr2=0x5615935c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 -- 172.21.15.44:0/3898613757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 msgr2=0x5615935afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6662d9d640 1 -- 172.21.15.44:0/3898613757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615934da620 con 0x5615935affb0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f666359e640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f66500a1200 0x7f66500a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.664+0000 7f6660d99640 1 -- 172.21.15.44:0/3210437207 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f665c0cd210 con 0x7f666410a350 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.665+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 0x5615935be820 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f665402a930 tx=0x7f6654031530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f00300a4ea40675 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.665+0000 7f66597fa640 1 -- 172.21.15.44:0/3898613757 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654004230 con 0x5615935affb0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.665+0000 7f666359e640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f66500a1200 0x7f66500a36c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ca640 tx=0x7f665c06c030 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.665+0000 7f66597fa640 1 -- 172.21.15.44:0/3898613757 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6654031760 con 0x5615935affb0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.665+0000 7f66597fa640 1 -- 172.21.15.44:0/3898613757 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654003b00 con 0x5615935affb0 2024-09-13T17:38:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- 172.21.15.44:0/3898613757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 msgr2=0x5615935be820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 0x5615935be820 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f665402a930 tx=0x7f6654031530 comp rx=0 tx=0).stop 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- 172.21.15.44:0/3898613757 shutdown_connections 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935affb0 0x5615935be820 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- 172.21.15.44:0/3898613757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- 172.21.15.44:0/3898613757 >> 172.21.15.44:0/3898613757 conn(0x5615935abd00 msgr2=0x5615935ac0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- 172.21.15.44:0/3898613757 shutdown_connections 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- 172.21.15.44:0/3898613757 wait complete. 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- start start 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 Processor -- start 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935bed60 0x5615935bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934d1180 con 0x5615935af690 2024-09-13T17:38:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615931eb430 con 0x5615935bed60 2024-09-13T17:38:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934965e0 con 0x5615935affb0 2024-09-13T17:38:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.666+0000 7f666be23740 10 client.0 _ll_register_callbacks cb 0x561593502aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935be3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38290/0 (socket says 172.21.15.44:38290) 2024-09-13T17:38:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 -- 172.21.15.44:0/2039148880 learned_addr learned my addr 172.21.15.44:0/2039148880 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f6662d9d640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 -- 172.21.15.44:0/2039148880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 msgr2=0x5615935bb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935bb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 -- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935bed60 msgr2=0x5615935bbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935bed60 0x5615935bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 -- 172.21.15.44:0/2039148880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666410d420 con 0x5615935af690 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f6662d9d640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935bb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f665bfff640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935be3b0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f664c014cb0 tx=0x7f664c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dedcb1675d6894 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f664a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f664a7fc640 1 -- 172.21.15.44:0/2039148880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c014270 con 0x5615935af690 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f664a7fc640 1 -- 172.21.15.44:0/2039148880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f664c004030 con 0x5615935af690 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56159353cb80 con 0x5615935af690 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f664a7fc640 1 -- 172.21.15.44:0/2039148880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664c004230 con 0x5615935af690 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f666be23740 10 client.19161 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.667+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5615935113c0 con 0x5615935af690 2024-09-13T17:38:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f664a7fc640 1 -- 172.21.15.44:0/2039148880 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56159353cb80 con 0x5615935af690 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f664a7fc640 1 -- 172.21.15.44:0/2039148880 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f664c012450 con 0x5615935af690 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f666be23740 10 client.19161 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f666be23740 10 client.19161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f666be23740 20 client.19161 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f666be23740 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5615935ce700 0x5615935d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f666be23740 4 client.19161 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615935051f0 con 0x5615935ce700 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f6662d9d640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5615935ce700 0x5615935d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f664a7fc640 10 client.19161 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.668+0000 7f6662d9d640 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5615935ce700 0x5615935d0b20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f437f1f13903f54 server_cookie=5f3ce2b32f9ccaad in_seq=0 out_seq=0 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f664a7fc640 1 -- 172.21.15.44:0/2039148880 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5615935051f0 con 0x5615935ce700 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f664a7fc640 10 client.19161 handle_command_reply: tid=0 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 client.19161 shutdown 2024-09-13T17:38:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 msgr2=0x5615935be3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935be3b0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f664c014cb0 tx=0x7f664c011810 comp rx=0 tx=0).stop 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 shutdown_connections 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x5615935ce700 0x5615935d0b20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935bb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935bed60 0x5615935bbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f664a7fc640 0 client.19161 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 --2- 172.21.15.44:0/2039148880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935be3b0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 >> 172.21.15.44:0/2039148880 conn(0x5615935abd00 msgr2=0x5615935b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 shutdown_connections 2024-09-13T17:38:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 1 -- 172.21.15.44:0/2039148880 wait complete. 2024-09-13T17:38:10.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.669+0000 7f666be23740 20 client.19161 trim_cache size 0 max 16384 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.680+0000 7f666be23740 1 Processor -- start 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.680+0000 7f666be23740 1 -- start start 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f666be23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f666be23740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935affb0 0x5615935be390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f666be23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f666be23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615935051f0 con 0x5615935af690 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f666be23740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159353cb80 con 0x5615935affb0 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f666be23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615935b0a60 con 0x5615935bed60 2024-09-13T17:38:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38300/0 (socket says 172.21.15.44:38300) 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 -- 172.21.15.44:0/1308417460 learned_addr learned my addr 172.21.15.44:0/1308417460 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 -- 172.21.15.44:0/1308417460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 msgr2=0x5615935b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f6662d9d640 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935affb0 0x5615935be390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 -- 172.21.15.44:0/1308417460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935affb0 msgr2=0x5615935be390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935affb0 0x5615935be390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 -- 172.21.15.44:0/1308417460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615934b3770 con 0x5615935af690 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f6662d9d640 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935affb0 0x5615935be390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f665bfff640 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935bde50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f664c000c00 tx=0x7f664c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a8816b5cf722e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f6659ffb640 1 -- 172.21.15.44:0/1308417460 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c005850 con 0x5615935af690 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f6659ffb640 1 -- 172.21.15.44:0/1308417460 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f664c01a070 con 0x5615935af690 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.681+0000 7f6659ffb640 1 -- 172.21.15.44:0/1308417460 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0035f0 con 0x5615935af690 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 -- 172.21.15.44:0/1308417460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 msgr2=0x5615935bde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935bde50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f664c000c00 tx=0x7f664c005d30 comp rx=0 tx=0).stop 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 -- 172.21.15.44:0/1308417460 shutdown_connections 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935bed60 0x5615935b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935affb0 0x5615935be390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 --2- 172.21.15.44:0/1308417460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935af690 0x5615935bde50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 -- 172.21.15.44:0/1308417460 >> 172.21.15.44:0/1308417460 conn(0x5615935abd00 msgr2=0x5615935bb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 -- 172.21.15.44:0/1308417460 shutdown_connections 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 -- 172.21.15.44:0/1308417460 wait complete. 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.682+0000 7f666be23740 1 -- start start 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 Processor -- start 2024-09-13T17:38:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 0x561593624c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615935c3540 con 0x5615935bed60 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615935c36b0 con 0x5615935af690 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561593625160 con 0x5615935affb0 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f666be23740 10 client.0 _ll_register_callbacks cb 0x7f6654009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 0x561593624c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 0x561593624c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38304/0 (socket says 172.21.15.44:38304) 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 -- 172.21.15.44:0/3448939938 learned_addr learned my addr 172.21.15.44:0/3448939938 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 -- 172.21.15.44:0/3448939938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 msgr2=0x5615935c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935c2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 -- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 msgr2=0x5615935c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 -- 172.21.15.44:0/3448939938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615935113c0 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f665bfff640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 0x561593624c20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f664c005e00 tx=0x7f664c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cfed76209f62ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.683+0000 7f664a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f664a7fc640 1 -- 172.21.15.44:0/3448939938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c01e620 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f664a7fc640 1 -- 172.21.15.44:0/3448939938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f664c000e40 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6654031760 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f664a7fc640 1 -- 172.21.15.44:0/3448939938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664c01e780 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 10 client.19167 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5615934d8e80 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f664a7fc640 1 -- 172.21.15.44:0/3448939938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6654031760 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f664a7fc640 1 -- 172.21.15.44:0/3448939938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f664c02b700 con 0x5615935bed60 2024-09-13T17:38:10.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 10 client.19167 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 10 client.19167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 20 client.19167 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561593626440 0x561593628860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 4 client.19167 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 --> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615932064e0 con 0x561593626440 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.684+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561593626440 0x561593628860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f664a7fc640 10 client.19167 ms_handle_connect on v2:172.21.15.70:6836/4202473697 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f6662d9d640 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561593626440 0x561593628860 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=289501677acb0b58 server_cookie=8aa2f184f4b33932 in_seq=0 out_seq=0 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f664a7fc640 1 -- 172.21.15.44:0/3448939938 <== mds.2 v2:172.21.15.70:6836/4202473697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615932064e0 con 0x561593626440 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f664a7fc640 10 client.19167 handle_command_reply: tid=0 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 client.19167 shutdown 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 msgr2=0x561593624c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 0x561593624c20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f664c005e00 tx=0x7f664c009bb0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 shutdown_connections 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:6836/4202473697,v1:172.21.15.70:6837/4202473697] conn(0x561593626440 0x561593628860 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615935affb0 0x5615935c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615935bed60 0x561593624c20 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f664a7fc640 0 client.19167 ms_handle_reset on v2:172.21.15.70:6836/4202473697 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 --2- 172.21.15.44:0/3448939938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615935af690 0x5615935c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 >> 172.21.15.44:0/3448939938 conn(0x5615935abd00 msgr2=0x5615935b9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 shutdown_connections 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 1 -- 172.21.15.44:0/3448939938 wait complete. 2024-09-13T17:38:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.685+0000 7f666be23740 20 client.19167 trim_cache size 0 max 16384 2024-09-13T17:38:10.698 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f66500a1200 msgr2=0x7f66500a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f66500a1200 0x7f66500a36c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ca640 tx=0x7f665c06c030 comp rx=0 tx=0).stop 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 msgr2=0x7f666419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 0x7f666419aa80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f665c00eaa0 tx=0x7f665c00ef60 comp rx=0 tx=0).stop 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 shutdown_connections 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f66500a1200 0x7f66500a36c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6664101800 0x7f6664196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f666410a350 0x7f666419aa80 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 --2- 172.21.15.44:0/3210437207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6664100e50 0x7f6664196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 >> 172.21.15.44:0/3210437207 conn(0x7f66640fa930 msgr2=0x7f66640fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 shutdown_connections 2024-09-13T17:38:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.686+0000 7f6669027640 1 -- 172.21.15.44:0/3210437207 wait complete. 2024-09-13T17:38:10.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T17:38:10.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 -- 172.21.15.44:0/4084007517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f40ff810 msgr2=0x7fe0f4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/4084007517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f40ff810 0x7fe0f4103850 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009930 tx=0x7fe0e402a420 comp rx=0 tx=0).stop 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 -- 172.21.15.44:0/4084007517 shutdown_connections 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/4084007517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4103fb0 0x7fe0f410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/4084007517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f40ff810 0x7fe0f4103850 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/4084007517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0f40fee60 0x7fe0f40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 -- 172.21.15.44:0/4084007517 >> 172.21.15.44:0/4084007517 conn(0x7fe0f4075c60 msgr2=0x7fe0f40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 -- 172.21.15.44:0/4084007517 shutdown_connections 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.803+0000 7fe0fb85a640 1 -- 172.21.15.44:0/4084007517 wait complete. 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 Processor -- start 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 -- start start 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0f40fee60 0x7fe0f4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4103fb0 0x7fe0f4196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 0x7fe0f419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f419b530 con 0x7fe0f419ab40 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f419b6a0 con 0x7fe0f40fee60 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0fb85a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f419b810 con 0x7fe0f4103fb0 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 0x7fe0f419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 0x7fe0f419aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38330/0 (socket says 172.21.15.44:38330) 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 -- 172.21.15.44:0/849636471 learned_addr learned my addr 172.21.15.44:0/849636471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4103fb0 0x7fe0f4196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0f40fee60 0x7fe0f4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 -- 172.21.15.44:0/849636471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4103fb0 msgr2=0x7fe0f4196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4103fb0 0x7fe0f4196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 -- 172.21.15.44:0/849636471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0f40fee60 msgr2=0x7fe0f4196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0f40fee60 0x7fe0f4196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.804+0000 7fe0f9df7640 1 -- 172.21.15.44:0/849636471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4009560 con 0x7fe0f419ab40 2024-09-13T17:38:10.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.805+0000 7fe0f9df7640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 0x7fe0f419aff0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00e960 tx=0x7fe0ec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.805+0000 7fe0ea7fc640 1 -- 172.21.15.44:0/849636471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec004d20 con 0x7fe0f419ab40 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.805+0000 7fe0ea7fc640 1 -- 172.21.15.44:0/849636471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0ec004e80 con 0x7fe0f419ab40 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.805+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f41a2df0 con 0x7fe0f419ab40 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.805+0000 7fe0ea7fc640 1 -- 172.21.15.44:0/849636471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec011470 con 0x7fe0f419ab40 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.805+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f41a3270 con 0x7fe0f419ab40 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.807+0000 7fe0ea7fc640 1 -- 172.21.15.44:0/849636471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe0ec00cb20 con 0x7fe0f419ab40 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.807+0000 7fe0fe65b740 1 Processor -- start 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.807+0000 7fe0fe65b740 1 -- start start 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a0c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a021cc0 con 0x563f0a0b80b0 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a01f450 con 0x563f0a0b7790 2024-09-13T17:38:10.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a01ad40 con 0x563f0a0c2950 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a0c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a0c2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38332/0 (socket says 172.21.15.44:38332) 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 -- 172.21.15.44:0/3908109643 learned_addr learned my addr 172.21.15.44:0/3908109643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0ea7fc640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe0e00a1200 0x7fe0e00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f9df7640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe0e00a1200 0x7fe0e00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 -- 172.21.15.44:0/3908109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 msgr2=0x563f0a0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 -- 172.21.15.44:0/3908109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 msgr2=0x563f0a0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 -- 172.21.15.44:0/3908109643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f09fe2620 con 0x563f0a0b80b0 2024-09-13T17:38:10.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0ea7fc640 1 -- 172.21.15.44:0/849636471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fe0ec0cd4f0 con 0x7fe0f419ab40 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a0c2410 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000bd10 tx=0x7fe0f00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f490b500be8ef85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0f9df7640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe0e00a1200 0x7fe0e00a36c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0092f0 tx=0x7fe0ec085040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.808+0000 7fe0df7fe640 1 -- 172.21.15.44:0/3908109643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004030 con 0x563f0a0b80b0 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0df7fe640 1 -- 172.21.15.44:0/3908109643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0f0015070 con 0x563f0a0b80b0 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0df7fe640 1 -- 172.21.15.44:0/3908109643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00116a0 con 0x563f0a0b80b0 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 -- 172.21.15.44:0/3908109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 msgr2=0x563f0a0c2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a0c2410 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000bd10 tx=0x7fe0f00107a0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 -- 172.21.15.44:0/3908109643 shutdown_connections 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a0c2410 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/3908109643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 -- 172.21.15.44:0/3908109643 >> 172.21.15.44:0/3908109643 conn(0x563f0a0b3d00 msgr2=0x563f0a0b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 -- 172.21.15.44:0/3908109643 shutdown_connections 2024-09-13T17:38:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.809+0000 7fe0fe65b740 1 -- 172.21.15.44:0/3908109643 wait complete. 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 -- start start 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 Processor -- start 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 0x563f0a0c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0c2950 0x563f0a0bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f09fd9180 con 0x563f0a0b7790 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f09cf3430 con 0x563f0a0c2950 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f09f9e5e0 con 0x563f0a0b80b0 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0fe65b740 10 client.0 _ll_register_callbacks cb 0x563f0a00aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 0x563f0a0c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 0x563f0a0c1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38346/0 (socket says 172.21.15.44:38346) 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 -- 172.21.15.44:0/2017420063 learned_addr learned my addr 172.21.15.44:0/2017420063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 -- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 msgr2=0x563f0a0c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 -- 172.21.15.44:0/2017420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0c2950 msgr2=0x563f0a0bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0c2950 0x563f0a0bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.810+0000 7fe0f8df5640 1 -- 172.21.15.44:0/2017420063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f000b940 con 0x563f0a0b7790 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 0x563f0a0c1e80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4072f50 tx=0x7fe0e40317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6202c17e3dd11533 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0dcff9640 1 -- 172.21.15.44:0/2017420063 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4031b60 con 0x563f0a0b7790 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0dcff9640 1 -- 172.21.15.44:0/2017420063 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fe0e4031cc0 con 0x563f0a0b7790 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f0a044b80 con 0x563f0a0b7790 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0dcff9640 1 -- 172.21.15.44:0/2017420063 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e4004500 con 0x563f0a0b7790 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 10 client.19179 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563f0a0193c0 con 0x563f0a0b7790 2024-09-13T17:38:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0dcff9640 1 -- 172.21.15.44:0/2017420063 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0e4050080 con 0x563f0a0b7790 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0dcff9640 1 -- 172.21.15.44:0/2017420063 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0e403c700 con 0x563f0a0b7790 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 10 client.19179 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 10 client.19179 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 20 client.19179 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a0d2230 0x563f0a0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.811+0000 7fe0fe65b740 4 client.19179 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.812+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f0a00d1f0 con 0x563f0a0d2230 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.812+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a0d2230 0x563f0a0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.812+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a0d2230 0x563f0a0d4650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a4440e864f30f7c server_cookie=7835bac73e1dd7b8 in_seq=0 out_seq=0 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.812+0000 7fe0dcff9640 10 client.19179 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0dcff9640 1 -- 172.21.15.44:0/2017420063 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563f0a00d1f0 con 0x563f0a0d2230 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0dcff9640 10 client.19179 handle_command_reply: tid=0 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 client.19179 shutdown 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 msgr2=0x563f0a0c1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 0x563f0a0c1e80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4072f50 tx=0x7fe0e40317f0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 shutdown_connections 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a0d2230 0x563f0a0d4650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0c2950 0x563f0a0bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0dcff9640 0 client.19179 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/2017420063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b7790 0x563f0a0c1e80 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 >> 172.21.15.44:0/2017420063 conn(0x563f0a0b3d00 msgr2=0x563f0a0b9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 shutdown_connections 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 1 -- 172.21.15.44:0/2017420063 wait complete. 2024-09-13T17:38:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.813+0000 7fe0fe65b740 20 client.19179 trim_cache size 0 max 16384 2024-09-13T17:38:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.824+0000 7fe0fe65b740 1 Processor -- start 2024-09-13T17:38:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.824+0000 7fe0fe65b740 1 -- start start 2024-09-13T17:38:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.824+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.824+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 0x563f0a0bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a0b39d0 con 0x563f0a0c2950 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a0b7510 con 0x563f0a0b7790 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a0d94d0 con 0x563f0a0b80b0 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 0x563f0a0bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 0x563f0a0bdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38354/0 (socket says 172.21.15.44:38354) 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 -- 172.21.15.44:0/1992462432 learned_addr learned my addr 172.21.15.44:0/1992462432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 -- 172.21.15.44:0/1992462432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 msgr2=0x563f0a0d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 -- 172.21.15.44:0/1992462432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 msgr2=0x563f0a0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 -- 172.21.15.44:0/1992462432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f09fbb770 con 0x563f0a0c2950 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 0x563f0a0bdb10 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005940 tx=0x7fe0e4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d053c959184a4c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0dffff640 1 -- 172.21.15.44:0/1992462432 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4059070 con 0x563f0a0c2950 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0dffff640 1 -- 172.21.15.44:0/1992462432 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563f0a044b80 con 0x563f0a0c2950 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.825+0000 7fe0dffff640 1 -- 172.21.15.44:0/1992462432 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4031c90 con 0x563f0a0c2950 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 -- 172.21.15.44:0/1992462432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 msgr2=0x563f0a0bdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 0x563f0a0bdb10 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005940 tx=0x7fe0e4005d70 comp rx=0 tx=0).stop 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 -- 172.21.15.44:0/1992462432 shutdown_connections 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0b80b0 0x563f0a0d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0c2950 0x563f0a0bdb10 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/1992462432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 -- 172.21.15.44:0/1992462432 >> 172.21.15.44:0/1992462432 conn(0x563f0a0b3d00 msgr2=0x563f0a0b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 -- 172.21.15.44:0/1992462432 shutdown_connections 2024-09-13T17:38:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 -- 172.21.15.44:0/1992462432 wait complete. 2024-09-13T17:38:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 -- start start 2024-09-13T17:38:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.826+0000 7fe0fe65b740 1 Processor -- start 2024-09-13T17:38:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a1251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a125710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a120810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a120dd0 con 0x563f0a0b80b0 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a120f40 con 0x563f0a0b7790 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a121100 con 0x563f0a0c2950 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0fe65b740 10 client.0 _ll_register_callbacks cb 0x7fe0f000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a125710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a125710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38370/0 (socket says 172.21.15.44:38370) 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 -- 172.21.15.44:0/819584649 learned_addr learned my addr 172.21.15.44:0/819584649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a1251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a120810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 -- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 msgr2=0x563f0a120810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a120810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 -- 172.21.15.44:0/819584649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 msgr2=0x563f0a1251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a1251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 -- 172.21.15.44:0/819584649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f0a0193c0 con 0x563f0a0b80b0 2024-09-13T17:38:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a120810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0f95f6640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a125710 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000c00 tx=0x7fe0f00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c508597e648c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.827+0000 7fe0dcff9640 1 -- 172.21.15.44:0/819584649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0021040 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0dcff9640 1 -- 172.21.15.44:0/819584649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7fe0f0017070 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f0015070 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0dcff9640 1 -- 172.21.15.44:0/819584649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f0003ac0 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 10 client.19185 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563f09fe0e80 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0dcff9640 1 -- 172.21.15.44:0/819584649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0f0015070 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0dcff9640 1 -- 172.21.15.44:0/819584649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0f0027480 con 0x563f0a0b80b0 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 10 client.19185 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 10 client.19185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-13T17:38:10.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 20 client.19185 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a12e1a0 0x563f0a1305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 4 client.19185 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 --> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f09d0e4e0 con 0x563f0a12e1a0 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.828+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a12e1a0 0x563f0a1305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0f8df5640 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a12e1a0 0x563f0a1305c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=899131a990647026 server_cookie=e045d87011f5988f in_seq=0 out_seq=0 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0dcff9640 10 client.19185 ms_handle_connect on v2:172.21.15.100:6836/288521582 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0dcff9640 1 -- 172.21.15.44:0/819584649 <== mds.1 v2:172.21.15.100:6836/288521582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f09d0e4e0 con 0x563f0a12e1a0 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0dcff9640 10 client.19185 handle_command_reply: tid=0 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 client.19185 shutdown 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 msgr2=0x563f0a125710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a125710 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000c00 tx=0x7fe0f00058e0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 shutdown_connections 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:6836/288521582,v1:172.21.15.100:6837/288521582] conn(0x563f0a12e1a0 0x563f0a1305c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f0a0c2950 0x563f0a120810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0dcff9640 0 client.19185 ms_handle_reset on v2:172.21.15.100:6836/288521582 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f0a0b80b0 0x563f0a125710 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 --2- 172.21.15.44:0/819584649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f0a0b7790 0x563f0a1251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 >> 172.21.15.44:0/819584649 conn(0x563f0a0b3d00 msgr2=0x563f0a0bdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 shutdown_connections 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.829+0000 7fe0fe65b740 1 -- 172.21.15.44:0/819584649 wait complete. 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fe65b740 20 client.19185 trim_cache size 0 max 16384 2024-09-13T17:38:10.838 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe0e00a1200 msgr2=0x7fe0e00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe0e00a1200 0x7fe0e00a36c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0092f0 tx=0x7fe0ec085040 comp rx=0 tx=0).stop 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 msgr2=0x7fe0f419aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 0x7fe0f419aff0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00e960 tx=0x7fe0ec00ee20 comp rx=0 tx=0).stop 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 shutdown_connections 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe0e00a1200 0x7fe0e00a36c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4103fb0 0x7fe0f4196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f419ab40 0x7fe0f419aff0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 --2- 172.21.15.44:0/849636471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0f40fee60 0x7fe0f4196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 >> 172.21.15.44:0/849636471 conn(0x7fe0f4075c60 msgr2=0x7fe0f4102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 shutdown_connections 2024-09-13T17:38:10.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.830+0000 7fe0fb85a640 1 -- 172.21.15.44:0/849636471 wait complete. 2024-09-13T17:38:10.841 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:10 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2545568982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:38:10.843 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T17:38:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/558814388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 msgr2=0x7f7ea410c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/558814388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea410c790 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009900 tx=0x7f7e9c02a480 comp rx=0 tx=0).stop 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/558814388 shutdown_connections 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/558814388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea410c790 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/558814388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea4101860 0x7f7ea4109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/558814388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100f40 0x7f7ea4101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/558814388 >> 172.21.15.44:0/558814388 conn(0x7f7ea4075c60 msgr2=0x7f7ea40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/558814388 shutdown_connections 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/558814388 wait complete. 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.945+0000 7f7eaa0aa640 1 Processor -- start 2024-09-13T17:38:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 -- start start 2024-09-13T17:38:10.949 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:10 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2545568982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:38:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100f40 0x7f7ea40feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea4101860 0x7f7ea40ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea40ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41a4510 con 0x7f7ea410a310 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41a4680 con 0x7f7ea4101860 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7eaa0aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41a4840 con 0x7f7ea4100f40 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea40ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea40ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38386/0 (socket says 172.21.15.44:38386) 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 -- 172.21.15.44:0/388156956 learned_addr learned my addr 172.21.15.44:0/388156956 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100f40 0x7f7ea40feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea4101860 0x7f7ea40ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 -- 172.21.15.44:0/388156956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100f40 msgr2=0x7f7ea40feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100f40 0x7f7ea40feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 -- 172.21.15.44:0/388156956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea4101860 msgr2=0x7f7ea40ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea4101860 0x7f7ea40ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 -- 172.21.15.44:0/388156956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c009560 con 0x7f7ea410a310 2024-09-13T17:38:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea3fff640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea40ff610 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c02aa70 tx=0x7f7e9c0329b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.946+0000 7f7ea0ff9640 1 -- 172.21.15.44:0/388156956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c032db0 con 0x7f7ea410a310 2024-09-13T17:38:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.947+0000 7f7ea0ff9640 1 -- 172.21.15.44:0/388156956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e9c005490 con 0x7f7ea410a310 2024-09-13T17:38:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.947+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea41a4ac0 con 0x7f7ea410a310 2024-09-13T17:38:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.947+0000 7f7ea0ff9640 1 -- 172.21.15.44:0/388156956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c003850 con 0x7f7ea410a310 2024-09-13T17:38:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.947+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea41a4fc0 con 0x7f7ea410a310 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.949+0000 7f7ea0ff9640 1 -- 172.21.15.44:0/388156956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7e9c0039b0 con 0x7f7ea410a310 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.949+0000 7f7eacdce740 1 Processor -- start 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.949+0000 7f7eacdce740 1 -- start start 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfdeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 0x55757dfebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7eacdce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757df48cc0 con 0x55757dfe9950 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7eacdce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757df46450 con 0x55757dfdf0b0 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7eacdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757df41d40 con 0x55757dfde790 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 0x55757dfebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea0ff9640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a1280 0x7f7e8c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 0x55757dfebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38394/0 (socket says 172.21.15.44:38394) 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/782790822 learned_addr learned my addr 172.21.15.44:0/782790822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea3fff640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a1280 0x7f7e8c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfdeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/782790822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 msgr2=0x55757dfdeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfdeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/782790822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 msgr2=0x55757dfe9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea0ff9640 1 -- 172.21.15.44:0/388156956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f7e9c0edc60 con 0x7f7ea410a310 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/782790822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55757df09620 con 0x55757dfe9950 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 0x55757dfebd40 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7e98012480 tx=0x7f7e98009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db7e928137b3a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7ea3fff640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a1280 0x7f7e8c0a3740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0eb5e0 tx=0x7f7e9c0d8f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.950+0000 7f7e89ffb640 1 -- 172.21.15.44:0/782790822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98004220 con 0x55757dfe9950 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7e89ffb640 1 -- 172.21.15.44:0/782790822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e98004380 con 0x55757dfe9950 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7e89ffb640 1 -- 172.21.15.44:0/782790822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98010690 con 0x55757dfe9950 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 -- 172.21.15.44:0/782790822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 msgr2=0x55757dfebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 0x55757dfebd40 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7e98012480 tx=0x7f7e98009710 comp rx=0 tx=0).stop 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 -- 172.21.15.44:0/782790822 shutdown_connections 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfe9950 0x55757dfebd40 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 --2- 172.21.15.44:0/782790822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfdeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 -- 172.21.15.44:0/782790822 >> 172.21.15.44:0/782790822 conn(0x55757dfdad00 msgr2=0x55757dfdb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 -- 172.21.15.44:0/782790822 shutdown_connections 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.951+0000 7f7eacdce740 1 -- 172.21.15.44:0/782790822 wait complete. 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 -- start start 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 Processor -- start 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfdf0b0 0x55757dfe63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757dfe6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757df00180 con 0x55757dfde790 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757dc1a430 con 0x55757dfe9950 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757dec55e0 con 0x55757dfdf0b0 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7eacdce740 10 client.0 _ll_register_callbacks cb 0x55757df31aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38410/0 (socket says 172.21.15.44:38410) 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2524955143 learned_addr learned my addr 172.21.15.44:0/2524955143 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757dfe6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfdf0b0 0x55757dfe63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfdf0b0 msgr2=0x55757dfe63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfdf0b0 0x55757dfe63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2524955143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 msgr2=0x55757dfe6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757dfe6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.952+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2524955143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e98009c30 con 0x55757dfde790 2024-09-13T17:38:10.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe8fe0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7e98003f20 tx=0x7f7e98003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a06d02ee62ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2524955143 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98005480 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2524955143 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f7e980055e0 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55757df6bb80 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2524955143 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e980046c0 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 10 client.19197 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55757df403c0 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2524955143 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55757df6bb80 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2524955143 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e980107e0 con 0x55757dfde790 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 10 client.19197 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 10 client.19197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 20 client.19197 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757dff9210 0x55757dffb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 4 client.19197 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.953+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55757df341f0 con 0x55757dff9210 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.954+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757dff9210 0x55757dffb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.954+0000 7f7e82ffd640 10 client.19197 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.954+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757dff9210 0x55757dffb630 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd10f336f387dd8 server_cookie=ffe164234d68f55e in_seq=0 out_seq=0 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2524955143 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55757df341f0 con 0x55757dff9210 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7e82ffd640 10 client.19197 handle_command_reply: tid=0 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 client.19197 shutdown 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 msgr2=0x55757dfe8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe8fe0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7e98003f20 tx=0x7f7e98003b90 comp rx=0 tx=0).stop 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 shutdown_connections 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757dff9210 0x55757dffb630 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfdf0b0 0x55757dfe63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757dfe6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7e82ffd640 0 client.19197 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2524955143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe8fe0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 >> 172.21.15.44:0/2524955143 conn(0x55757dfdad00 msgr2=0x55757dfe0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 shutdown_connections 2024-09-13T17:38:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 1 -- 172.21.15.44:0/2524955143 wait complete. 2024-09-13T17:38:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.955+0000 7f7eacdce740 20 client.19197 trim_cache size 0 max 16384 2024-09-13T17:38:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 Processor -- start 2024-09-13T17:38:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 -- start start 2024-09-13T17:38:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfe9950 0x55757dfe6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757dfe93e0 con 0x55757dfde790 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757dfd5300 con 0x55757dfdf0b0 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7eacdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757dfe56e0 con 0x55757dfe9950 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38422/0 (socket says 172.21.15.44:38422) 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.966+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2287679848 learned_addr learned my addr 172.21.15.44:0/2287679848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2287679848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfe9950 msgr2=0x55757dfe6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfe9950 0x55757dfe6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2287679848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 msgr2=0x55757dfe51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea2ffd640 1 -- 172.21.15.44:0/2287679848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55757dee2770 con 0x55757dfde790 2024-09-13T17:38:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe4c60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006010 tx=0x7f7e98009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da64fe9bcf7edd9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7e8a7fc640 1 -- 172.21.15.44:0/2287679848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98010dc0 con 0x55757dfde790 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7e8a7fc640 1 -- 172.21.15.44:0/2287679848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e98009e50 con 0x55757dfde790 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.967+0000 7f7e8a7fc640 1 -- 172.21.15.44:0/2287679848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800ed50 con 0x55757dfde790 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- 172.21.15.44:0/2287679848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 msgr2=0x55757dfe4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe4c60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006010 tx=0x7f7e98009530 comp rx=0 tx=0).stop 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- 172.21.15.44:0/2287679848 shutdown_connections 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfe9950 0x55757dfe6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfdf0b0 0x55757dfe51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2287679848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfde790 0x55757dfe4c60 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- 172.21.15.44:0/2287679848 >> 172.21.15.44:0/2287679848 conn(0x55757dfdad00 msgr2=0x55757dff50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- 172.21.15.44:0/2287679848 shutdown_connections 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- 172.21.15.44:0/2287679848 wait complete. 2024-09-13T17:38:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- start start 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 Processor -- start 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 0x55757dfece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757e048740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757e048cb0 con 0x55757dfdf0b0 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757e048e20 con 0x55757dfe9950 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757e048fe0 con 0x55757dfde790 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.968+0000 7f7eacdce740 10 client.0 _ll_register_callbacks cb 0x7f7e98009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 0x55757dfece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 0x55757dfece20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38434/0 (socket says 172.21.15.44:38434) 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 -- 172.21.15.44:0/2348257707 learned_addr learned my addr 172.21.15.44:0/2348257707 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757e048740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 -- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 msgr2=0x55757dfec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 -- 172.21.15.44:0/2348257707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 msgr2=0x55757e048740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757e048740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 -- 172.21.15.44:0/2348257707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55757df403c0 con 0x55757dfdf0b0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7ea37fe640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 0x55757dfece20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x55757dfe5c20 tx=0x7f7e94013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ed3e47b175f4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2348257707 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017040 con 0x55757dfdf0b0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2348257707 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f7e9401e520 con 0x55757dfdf0b0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e98004380 con 0x55757dfdf0b0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2348257707 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9401e6e0 con 0x55757dfdf0b0 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7eacdce740 10 client.19203 fetch_fsmap learned FSMap version 25 2024-09-13T17:38:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.969+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55757df07e80 con 0x55757dfdf0b0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2348257707 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e98004380 con 0x55757dfdf0b0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2348257707 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e9401e040 con 0x55757dfdf0b0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7eacdce740 10 client.19203 fetch_fsmap finished waiting for FSMap version 25 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7eacdce740 10 client.19203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7eacdce740 20 client.19203 populate_metadata read hostname 'smithi044' 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757e0507f0 0x55757e052c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7eacdce740 4 client.19203 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 --> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55757dc354e0 con 0x55757e0507f0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757e0507f0 0x55757e052c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7e82ffd640 10 client.19203 ms_handle_connect on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.970+0000 7f7ea2ffd640 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757e0507f0 0x55757e052c10 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc24c0d9d167b7 server_cookie=9000aba2780823ba in_seq=0 out_seq=0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7e82ffd640 1 -- 172.21.15.44:0/2348257707 <== mds.0 v2:172.21.15.100:6838/3416533473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55757dc354e0 con 0x55757e0507f0 2024-09-13T17:38:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7e82ffd640 10 client.19203 handle_command_reply: tid=0 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 client.19203 shutdown 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 msgr2=0x55757dfece20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 0x55757dfece20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x55757dfe5c20 tx=0x7f7e94013950 comp rx=0 tx=0).stop 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 shutdown_connections 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:6838/3416533473,v1:172.21.15.100:6839/3416533473] conn(0x55757e0507f0 0x55757e052c10 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55757dfe9950 0x55757e048740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55757dfdf0b0 0x55757dfece20 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7e82ffd640 0 client.19203 ms_handle_reset on v2:172.21.15.100:6838/3416533473 2024-09-13T17:38:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 --2- 172.21.15.44:0/2348257707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55757dfde790 0x55757dfec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 >> 172.21.15.44:0/2348257707 conn(0x55757dfdad00 msgr2=0x55757dff50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 shutdown_connections 2024-09-13T17:38:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 1 -- 172.21.15.44:0/2348257707 wait complete. 2024-09-13T17:38:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eacdce740 20 client.19203 trim_cache size 0 max 16384 2024-09-13T17:38:10.982 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-09-13T17:38:10.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a1280 msgr2=0x7f7e8c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a1280 0x7f7e8c0a3740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0eb5e0 tx=0x7f7e9c0d8f90 comp rx=0 tx=0).stop 2024-09-13T17:38:10.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 msgr2=0x7f7ea40ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:10.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.971+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea40ff610 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c02aa70 tx=0x7f7e9c0329b0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 shutdown_connections 2024-09-13T17:38:10.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f7e8c0a1280 0x7f7e8c0a3740 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea410a310 0x7f7ea40ff610 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea4101860 0x7f7ea40ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 --2- 172.21.15.44:0/388156956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100f40 0x7f7ea40feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:10.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 >> 172.21.15.44:0/388156956 conn(0x7f7ea4075c60 msgr2=0x7f7ea40fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:10.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 shutdown_connections 2024-09-13T17:38:10.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:10.972+0000 7f7eaa0aa640 1 -- 172.21.15.44:0/388156956 wait complete. 2024-09-13T17:38:11.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:10 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2545568982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:38:11.985 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-09-13T17:38:11.986 INFO:tasks.fwd_scrub:done joining 2024-09-13T17:38:11.986 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-13T17:38:12.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:38:12.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 -- 172.21.15.44:0/174306701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 msgr2=0x7f5fa00ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 --2- 172.21.15.44:0/174306701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa00ff260 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009930 tx=0x7f5f9002a420 comp rx=0 tx=0).stop 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 -- 172.21.15.44:0/174306701 shutdown_connections 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 --2- 172.21.15.44:0/174306701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00ff830 0x7f5fa0103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 --2- 172.21.15.44:0/174306701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fa0103fd0 0x7f5fa010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 --2- 172.21.15.44:0/174306701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa00ff260 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 -- 172.21.15.44:0/174306701 >> 172.21.15.44:0/174306701 conn(0x7f5fa0075c60 msgr2=0x7f5fa00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 -- 172.21.15.44:0/174306701 shutdown_connections 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.121+0000 7f5fa5496640 1 -- 172.21.15.44:0/174306701 wait complete. 2024-09-13T17:38:12.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 Processor -- start 2024-09-13T17:38:12.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 -- start start 2024-09-13T17:38:12.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fa0103fd0 0x7f5fa01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa019cce0 0x7f5fa019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa019d6d0 con 0x7f5fa00fee80 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa019d840 con 0x7f5fa0103fd0 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5fa5496640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa019d9b0 con 0x7f5fa019cce0 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5f9effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5f9effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38454/0 (socket says 172.21.15.44:38454) 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5f9effd640 1 -- 172.21.15.44:0/2559439109 learned_addr learned my addr 172.21.15.44:0/2559439109 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5f9e7fc640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fa0103fd0 0x7f5fa01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5f9f7fe640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa019cce0 0x7f5fa019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.122+0000 7f5f9effd640 1 -- 172.21.15.44:0/2559439109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa019cce0 msgr2=0x7f5fa019d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f9effd640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa019cce0 0x7f5fa019d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f9effd640 1 -- 172.21.15.44:0/2559439109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fa0103fd0 msgr2=0x7f5fa01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f9effd640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fa0103fd0 0x7f5fa01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f9effd640 1 -- 172.21.15.44:0/2559439109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90009560 con 0x7f5fa00fee80 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f9f7fe640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa019cce0 0x7f5fa019d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f9effd640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa01983b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005ab0 tx=0x7f5f90032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90032e10 con 0x7f5fa00fee80 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f900046c0 con 0x7f5fa00fee80 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa01a4f90 con 0x7f5fa00fee80 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90003850 con 0x7f5fa00fee80 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.123+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa01a5300 con 0x7f5fa00fee80 2024-09-13T17:38:12.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.124+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa0100350 con 0x7f5fa00fee80 2024-09-13T17:38:12.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.125+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5f90004830 con 0x7f5fa00fee80 2024-09-13T17:38:12.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.126+0000 7f5f87fff640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5f880a1280 0x7f5f880a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.126+0000 7f5f9e7fc640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5f880a1280 0x7f5f880a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.126+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f5f900edff0 con 0x7f5fa00fee80 2024-09-13T17:38:12.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.127+0000 7f5f9e7fc640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5f880a1280 0x7f5f880a3740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0199950 tx=0x7f5f9400b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:12.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.129+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5f900b70b0 con 0x7f5fa00fee80 2024-09-13T17:38:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.305+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fa00ff260 con 0x7f5fa00fee80 2024-09-13T17:38:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.306+0000 7f5f87fff640 1 -- 172.21.15.44:0/2559439109 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25054 (secure 0 0 0) 0x7f5f90040090 con 0x7f5fa00fee80 2024-09-13T17:38:12.309 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:12.309 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:38:11.319123+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5f880a1280 msgr2=0x7f5f880a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5f880a1280 0x7f5f880a3740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0199950 tx=0x7f5f9400b390 comp rx=0 tx=0).stop 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 msgr2=0x7f5fa01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa01983b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005ab0 tx=0x7f5f90032c20 comp rx=0 tx=0).stop 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 shutdown_connections 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5f880a1280 0x7f5f880a3740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa019cce0 0x7f5fa019d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fa0103fd0 0x7f5fa01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 --2- 172.21.15.44:0/2559439109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa00fee80 0x7f5fa01983b0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.308+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 >> 172.21.15.44:0/2559439109 conn(0x7f5fa0075c60 msgr2=0x7f5fa0102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.309+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 shutdown_connections 2024-09-13T17:38:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.309+0000 7f5fa5496640 1 -- 172.21.15.44:0/2559439109 wait complete. 2024-09-13T17:38:12.322 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:38:12.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:38:12.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 -- 172.21.15.44:0/2117322762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf400fee60 msgr2=0x7faf400ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 --2- 172.21.15.44:0/2117322762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf400fee60 0x7faf400ff240 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7faf30009cd0 tx=0x7faf3002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 -- 172.21.15.44:0/2117322762 shutdown_connections 2024-09-13T17:38:12.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 --2- 172.21.15.44:0/2117322762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf40103fb0 0x7faf4010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 --2- 172.21.15.44:0/2117322762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf400ff810 0x7faf40103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 --2- 172.21.15.44:0/2117322762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf400fee60 0x7faf400ff240 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 -- 172.21.15.44:0/2117322762 >> 172.21.15.44:0/2117322762 conn(0x7faf40075c60 msgr2=0x7faf400fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 -- 172.21.15.44:0/2117322762 shutdown_connections 2024-09-13T17:38:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 -- 172.21.15.44:0/2117322762 wait complete. 2024-09-13T17:38:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.426+0000 7faf44c53640 1 Processor -- start 2024-09-13T17:38:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 -- start start 2024-09-13T17:38:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf400fee60 0x7faf40196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 0x7faf401967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf4019aba0 0x7faf4019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4019b590 con 0x7faf40103fb0 2024-09-13T17:38:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4019b700 con 0x7faf4019aba0 2024-09-13T17:38:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf44c53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4019b870 con 0x7faf400fee60 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 0x7faf401967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 0x7faf401967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38480/0 (socket says 172.21.15.44:38480) 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 -- 172.21.15.44:0/1192088355 learned_addr learned my addr 172.21.15.44:0/1192088355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3e59c640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf400fee60 0x7faf40196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3ed9d640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf4019aba0 0x7faf4019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 -- 172.21.15.44:0/1192088355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf400fee60 msgr2=0x7faf40196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf400fee60 0x7faf40196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 -- 172.21.15.44:0/1192088355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf4019aba0 msgr2=0x7faf4019b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf4019aba0 0x7faf4019b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 -- 172.21.15.44:0/1192088355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009900 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.427+0000 7faf3dd9b640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 0x7faf401967b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7faf3400e9c0 tx=0x7faf3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.428+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34009700 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.428+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf3400cd80 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.428+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf401a2e50 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.428+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34010430 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.428+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf401a32d0 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.429+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf40100330 con 0x7faf40103fb0 2024-09-13T17:38:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.430+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf34004eb0 con 0x7faf40103fb0 2024-09-13T17:38:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.431+0000 7faf2f7fe640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf280a1280 0x7faf280a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.431+0000 7faf3e59c640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf280a1280 0x7faf280a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.431+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7faf34014070 con 0x7faf40103fb0 2024-09-13T17:38:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.431+0000 7faf3e59c640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf280a1280 0x7faf280a3740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ca0 tx=0x7faf30004a80 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.435+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faf340965c0 con 0x7faf40103fb0 2024-09-13T17:38:12.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:12 smithi044 ceph-mon[31175]: pgmap v358: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-13T17:38:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:12 smithi044 ceph-mon[31175]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T17:38:12.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:12 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2559439109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:12.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.607+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf400ff810 con 0x7faf40103fb0 2024-09-13T17:38:12.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.608+0000 7faf2f7fe640 1 -- 172.21.15.44:0/1192088355 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf34099be0 con 0x7faf40103fb0 2024-09-13T17:38:12.610 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:38:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf280a1280 msgr2=0x7faf280a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf280a1280 0x7faf280a3740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ca0 tx=0x7faf30004a80 comp rx=0 tx=0).stop 2024-09-13T17:38:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 msgr2=0x7faf401967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 0x7faf401967b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7faf3400e9c0 tx=0x7faf3400ee80 comp rx=0 tx=0).stop 2024-09-13T17:38:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 shutdown_connections 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7faf280a1280 0x7faf280a3740 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf4019aba0 0x7faf4019b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf40103fb0 0x7faf401967b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 --2- 172.21.15.44:0/1192088355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf400fee60 0x7faf40196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 >> 172.21.15.44:0/1192088355 conn(0x7faf40075c60 msgr2=0x7faf40067740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 shutdown_connections 2024-09-13T17:38:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.610+0000 7faf44c53640 1 -- 172.21.15.44:0/1192088355 wait complete. 2024-09-13T17:38:12.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:38:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:12 smithi070 ceph-mon[34502]: pgmap v358: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-13T17:38:12.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:12 smithi070 ceph-mon[34502]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T17:38:12.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:12 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2559439109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:12.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 -- 172.21.15.44:0/3133011008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc103fd0 msgr2=0x7f58bc10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 --2- 172.21.15.44:0/3133011008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc103fd0 0x7f58bc10bb20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009cd0 tx=0x7f58b402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 -- 172.21.15.44:0/3133011008 shutdown_connections 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 --2- 172.21.15.44:0/3133011008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc103fd0 0x7f58bc10bb20 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 --2- 172.21.15.44:0/3133011008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0ff830 0x7f58bc103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 --2- 172.21.15.44:0/3133011008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc0fee80 0x7f58bc0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 -- 172.21.15.44:0/3133011008 >> 172.21.15.44:0/3133011008 conn(0x7f58bc0fa930 msgr2=0x7f58bc0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 -- 172.21.15.44:0/3133011008 shutdown_connections 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 -- 172.21.15.44:0/3133011008 wait complete. 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.726+0000 7f58c40e5640 1 Processor -- start 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 -- start start 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc0fee80 0x7f58bc196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc103fd0 0x7f58bc197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 0x7f58bc1a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a26f0 con 0x7f58bc197a50 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a2860 con 0x7f58bc103fd0 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c40e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a2a20 con 0x7f58bc0fee80 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 0x7f58bc1a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 0x7f58bc1a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38500/0 (socket says 172.21.15.44:38500) 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 -- 172.21.15.44:0/2352747115 learned_addr learned my addr 172.21.15.44:0/2352747115 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c1e81640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc0fee80 0x7f58bc196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c1680640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc103fd0 0x7f58bc197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 -- 172.21.15.44:0/2352747115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc0fee80 msgr2=0x7f58bc196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc0fee80 0x7f58bc196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 -- 172.21.15.44:0/2352747115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc103fd0 msgr2=0x7f58bc197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc103fd0 0x7f58bc197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 -- 172.21.15.44:0/2352747115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009900 con 0x7f58bc197a50 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58c2682640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 0x7f58bc1a2050 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f58b4004af0 tx=0x7f58b4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.727+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4003970 con 0x7f58bc197a50 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.728+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58b4004040 con 0x7f58bc197a50 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.728+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc1a2ca0 con 0x7f58bc197a50 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.728+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4045650 con 0x7f58bc197a50 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.728+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1a3200 con 0x7f58bc197a50 2024-09-13T17:38:12.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.728+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc100350 con 0x7f58bc197a50 2024-09-13T17:38:12.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.730+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f58b400a040 con 0x7f58bc197a50 2024-09-13T17:38:12.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.731+0000 7f58b2ffd640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f58a80a1280 0x7f58a80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:12.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.731+0000 7f58c1e81640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f58a80a1280 0x7f58a80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:12.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.731+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f58b40ee940 con 0x7f58bc197a50 2024-09-13T17:38:12.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.732+0000 7f58c1e81640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f58a80a1280 0x7f58a80a3740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58b800bbe0 tx=0x7f58b800b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:12.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.734+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f58b407fe20 con 0x7f58bc197a50 2024-09-13T17:38:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:12 smithi100 ceph-mon[35055]: pgmap v358: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-13T17:38:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:12 smithi100 ceph-mon[35055]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T17:38:12.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:12 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2559439109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.905+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58bc1071e0 con 0x7f58bc197a50 2024-09-13T17:38:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.906+0000 7f58b2ffd640 1 -- 172.21.15.44:0/2352747115 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58b40b7a70 con 0x7f58bc197a50 2024-09-13T17:38:12.908 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f58a80a1280 msgr2=0x7f58a80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f58a80a1280 0x7f58a80a3740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58b800bbe0 tx=0x7f58b800b390 comp rx=0 tx=0).stop 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 msgr2=0x7f58bc1a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 0x7f58bc1a2050 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f58b4004af0 tx=0x7f58b4030ec0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 shutdown_connections 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f58a80a1280 0x7f58a80a3740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58bc197a50 0x7f58bc1a2050 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc103fd0 0x7f58bc197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 --2- 172.21.15.44:0/2352747115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc0fee80 0x7f58bc196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 >> 172.21.15.44:0/2352747115 conn(0x7f58bc0fa930 msgr2=0x7f58bc0fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 shutdown_connections 2024-09-13T17:38:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:12.908+0000 7f58c40e5640 1 -- 172.21.15.44:0/2352747115 wait complete. 2024-09-13T17:38:12.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:38:13.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 -- 172.21.15.44:0/1891897344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab580fff90 msgr2=0x7fab58100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1891897344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab580fff90 0x7fab58100370 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fab48009930 tx=0x7fab4802a420 comp rx=0 tx=0).stop 2024-09-13T17:38:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 -- 172.21.15.44:0/1891897344 shutdown_connections 2024-09-13T17:38:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1891897344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab58100940 0x7fab58104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1891897344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab581050e0 0x7fab5810cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1891897344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab580fff90 0x7fab58100370 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 -- 172.21.15.44:0/1891897344 >> 172.21.15.44:0/1891897344 conn(0x7fab580fba20 msgr2=0x7fab580fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:13.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 -- 172.21.15.44:0/1891897344 shutdown_connections 2024-09-13T17:38:13.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 -- 172.21.15.44:0/1891897344 wait complete. 2024-09-13T17:38:13.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.024+0000 7fab5dec9640 1 Processor -- start 2024-09-13T17:38:13.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 -- start start 2024-09-13T17:38:13.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab580fff90 0x7fab58196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab581050e0 0x7fab581967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 0x7fab5819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819b580 con 0x7fab5819ab90 2024-09-13T17:38:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819b6f0 con 0x7fab580fff90 2024-09-13T17:38:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab5dec9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819b860 con 0x7fab581050e0 2024-09-13T17:38:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 0x7fab5819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab577fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab580fff90 0x7fab58196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 0x7fab5819b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38516/0 (socket says 172.21.15.44:38516) 2024-09-13T17:38:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 -- 172.21.15.44:0/1373134729 learned_addr learned my addr 172.21.15.44:0/1373134729 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab56ffd640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab581050e0 0x7fab581967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 -- 172.21.15.44:0/1373134729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab581050e0 msgr2=0x7fab581967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab581050e0 0x7fab581967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 -- 172.21.15.44:0/1373134729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab580fff90 msgr2=0x7fab58196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab580fff90 0x7fab58196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 -- 172.21.15.44:0/1373134729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009560 con 0x7fab5819ab90 2024-09-13T17:38:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab577fe640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab580fff90 0x7fab58196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:38:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.025+0000 7fab57fff640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 0x7fab5819b040 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fab5000eaa0 tx=0x7fab5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.026+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000cdb0 con 0x7fab5819ab90 2024-09-13T17:38:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.026+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab50005020 con 0x7fab5819ab90 2024-09-13T17:38:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.026+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab581a2e40 con 0x7fab5819ab90 2024-09-13T17:38:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.026+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500116d0 con 0x7fab5819ab90 2024-09-13T17:38:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.026+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581a3270 con 0x7fab5819ab90 2024-09-13T17:38:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.027+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58101460 con 0x7fab5819ab90 2024-09-13T17:38:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.028+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab50005190 con 0x7fab5819ab90 2024-09-13T17:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.029+0000 7fab54ff9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fab400a1200 0x7fab400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.029+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fab500cd460 con 0x7fab5819ab90 2024-09-13T17:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.029+0000 7fab577fe640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fab400a1200 0x7fab400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.029+0000 7fab577fe640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fab400a1200 0x7fab400a36c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fab480057c0 tx=0x7fab48005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.032+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fab50096690 con 0x7fab5819ab90 2024-09-13T17:38:13.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.204+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab58108290 con 0x7fab5819ab90 2024-09-13T17:38:13.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.205+0000 7fab54ff9640 1 -- 172.21.15.44:0/1373134729 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab5001f0a0 con 0x7fab5819ab90 2024-09-13T17:38:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:38:13.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fab400a1200 msgr2=0x7fab400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fab400a1200 0x7fab400a36c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fab480057c0 tx=0x7fab48005a20 comp rx=0 tx=0).stop 2024-09-13T17:38:13.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 msgr2=0x7fab5819b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 0x7fab5819b040 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fab5000eaa0 tx=0x7fab5000ef60 comp rx=0 tx=0).stop 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 shutdown_connections 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fab400a1200 0x7fab400a36c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab581050e0 0x7fab581967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5819ab90 0x7fab5819b040 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 --2- 172.21.15.44:0/1373134729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab580fff90 0x7fab58196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 >> 172.21.15.44:0/1373134729 conn(0x7fab580fba20 msgr2=0x7fab580fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 shutdown_connections 2024-09-13T17:38:13.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.207+0000 7fab5dec9640 1 -- 172.21.15.44:0/1373134729 wait complete. 2024-09-13T17:38:13.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:38:13.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 -- 172.21.15.44:0/3950028019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500ff830 msgr2=0x7f4050103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 --2- 172.21.15.44:0/3950028019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500ff830 0x7f4050103870 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f403c009930 tx=0x7f403c02a420 comp rx=0 tx=0).stop 2024-09-13T17:38:13.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 -- 172.21.15.44:0/3950028019 shutdown_connections 2024-09-13T17:38:13.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 --2- 172.21.15.44:0/3950028019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4050103fd0 0x7f405010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 --2- 172.21.15.44:0/3950028019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500ff830 0x7f4050103870 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 --2- 172.21.15.44:0/3950028019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40500fee80 0x7f40500ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 -- 172.21.15.44:0/3950028019 >> 172.21.15.44:0/3950028019 conn(0x7f40500fa8d0 msgr2=0x7f40500fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 -- 172.21.15.44:0/3950028019 shutdown_connections 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.325+0000 7f40571b5640 1 -- 172.21.15.44:0/3950028019 wait complete. 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 Processor -- start 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 -- start start 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 0x7f40501970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4050103fd0 0x7f4050197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197b70 0x7f40501a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501a2810 con 0x7f40500fee80 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501a2980 con 0x7f4050197b70 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f40571b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501a2b40 con 0x7f4050103fd0 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f4054f51640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 0x7f40501970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f4054f51640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 0x7f40501970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38526/0 (socket says 172.21.15.44:38526) 2024-09-13T17:38:13.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f4054f51640 1 -- 172.21.15.44:0/2152086665 learned_addr learned my addr 172.21.15.44:0/2152086665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.326+0000 7f404ffff640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4050103fd0 0x7f4050197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4055752640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197b70 0x7f40501a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4054f51640 1 -- 172.21.15.44:0/2152086665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4050103fd0 msgr2=0x7f4050197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4054f51640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4050103fd0 0x7f4050197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4054f51640 1 -- 172.21.15.44:0/2152086665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197b70 msgr2=0x7f40501a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4054f51640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197b70 0x7f40501a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4054f51640 1 -- 172.21.15.44:0/2152086665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c009560 con 0x7f40500fee80 2024-09-13T17:38:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.327+0000 7f4054f51640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 0x7f40501970f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f404400e9f0 tx=0x7f404400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.328+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044009700 con 0x7f40500fee80 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.328+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f404400cd80 con 0x7f40500fee80 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.328+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40501a2e20 con 0x7f40500fee80 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.328+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40440187e0 con 0x7f40500fee80 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.328+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40501a3240 con 0x7f40500fee80 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.329+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4050100350 con 0x7f40500fee80 2024-09-13T17:38:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.330+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4044010430 con 0x7f40500fee80 2024-09-13T17:38:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.331+0000 7f404dffb640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f40400a1200 0x7f40400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.333+0000 7f404ffff640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f40400a1200 0x7f40400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.333+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4044014070 con 0x7f40500fee80 2024-09-13T17:38:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.333+0000 7f404ffff640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f40400a1200 0x7f40400a36c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f403c005ab0 tx=0x7f403c005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.336+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4044096920 con 0x7f40500fee80 2024-09-13T17:38:13.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.507+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f40501071e0 con 0x7f40500fee80 2024-09-13T17:38:13.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.508+0000 7f404dffb640 1 -- 172.21.15.44:0/2152086665 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4044099f40 con 0x7f40500fee80 2024-09-13T17:38:13.510 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f40400a1200 msgr2=0x7f40400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f40400a1200 0x7f40400a36c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f403c005ab0 tx=0x7f403c005a20 comp rx=0 tx=0).stop 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 msgr2=0x7f40501970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 0x7f40501970f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f404400e9f0 tx=0x7f404400eeb0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 shutdown_connections 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f40400a1200 0x7f40400a36c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4050103fd0 0x7f4050197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197b70 0x7f40501a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.509+0000 7f40571b5640 1 --2- 172.21.15.44:0/2152086665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40500fee80 0x7f40501970f0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.510+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 >> 172.21.15.44:0/2152086665 conn(0x7f40500fa8d0 msgr2=0x7f40500fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:13.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.510+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 shutdown_connections 2024-09-13T17:38:13.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.510+0000 7f40571b5640 1 -- 172.21.15.44:0/2152086665 wait complete. 2024-09-13T17:38:13.522 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1192088355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.522 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2352747115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.522 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:13 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1373134729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.523 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-13T17:38:13.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 -- 172.21.15.44:0/3864873475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 msgr2=0x7faccc10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 --2- 172.21.15.44:0/3864873475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc10bb30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7facc4009900 tx=0x7facc402a3f0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 -- 172.21.15.44:0/3864873475 shutdown_connections 2024-09-13T17:38:13.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 --2- 172.21.15.44:0/3864873475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc10bb30 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 --2- 172.21.15.44:0/3864873475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ff840 0x7faccc103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 --2- 172.21.15.44:0/3864873475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 0x7faccc0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 -- 172.21.15.44:0/3864873475 >> 172.21.15.44:0/3864873475 conn(0x7faccc0fa900 msgr2=0x7faccc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:13.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 -- 172.21.15.44:0/3864873475 shutdown_connections 2024-09-13T17:38:13.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.623+0000 7facd33aa640 1 -- 172.21.15.44:0/3864873475 wait complete. 2024-09-13T17:38:13.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 Processor -- start 2024-09-13T17:38:13.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 -- start start 2024-09-13T17:38:13.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 0x7faccc196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc19ab40 0x7faccc19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19b530 con 0x7faccc103fe0 2024-09-13T17:38:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19b6a0 con 0x7faccc19ab40 2024-09-13T17:38:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd33aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19b810 con 0x7faccc0fee90 2024-09-13T17:38:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd1146640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 0x7faccc196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38542/0 (socket says 172.21.15.44:38542) 2024-09-13T17:38:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 -- 172.21.15.44:0/2166283056 learned_addr learned my addr 172.21.15.44:0/2166283056 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd1947640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc19ab40 0x7faccc19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 -- 172.21.15.44:0/2166283056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 msgr2=0x7faccc196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 0x7faccc196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 -- 172.21.15.44:0/2166283056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc19ab40 msgr2=0x7faccc19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc19ab40 0x7faccc19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd0945640 1 -- 172.21.15.44:0/2166283056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009560 con 0x7faccc103fe0 2024-09-13T17:38:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.624+0000 7facd1146640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 0x7faccc196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:38:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.625+0000 7facd0945640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc196750 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b450 tx=0x7facbc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.625+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc011890 con 0x7faccc103fe0 2024-09-13T17:38:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.625+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facbc011e80 con 0x7faccc103fe0 2024-09-13T17:38:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.625+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc1a2df0 con 0x7faccc103fe0 2024-09-13T17:38:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.625+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00f340 con 0x7faccc103fe0 2024-09-13T17:38:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.625+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc1a3240 con 0x7faccc103fe0 2024-09-13T17:38:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.626+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc100360 con 0x7faccc103fe0 2024-09-13T17:38:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.627+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7facbc0119f0 con 0x7faccc103fe0 2024-09-13T17:38:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.627+0000 7facc27fc640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facb80a1200 0x7facb80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.628+0000 7facd1146640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facb80a1200 0x7facb80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.628+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7facbc0ccc40 con 0x7faccc103fe0 2024-09-13T17:38:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.628+0000 7facd1146640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facb80a1200 0x7facb80a36c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7facc8009fd0 tx=0x7facc8009680 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:13.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.632+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7facbc095e30 con 0x7faccc103fe0 2024-09-13T17:38:13.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1192088355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2352747115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:13 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1373134729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1192088355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2352747115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:13 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1373134729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:13.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.856+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7faccc0ff270 con 0x7faccc103fe0 2024-09-13T17:38:13.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.864+0000 7facc27fc640 1 -- 172.21.15.44:0/2166283056 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733940 (secure 0 0 0) 0x7facbc015090 con 0x7faccc103fe0 2024-09-13T17:38:13.878 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:13.878 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:38:13.878 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 26, 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "active_gid": 15468, 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "active_name": "x", 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addrs": { 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4124099065 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4124099065 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addr": "172.21.15.44:6801/4124099065", 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "active_change": "2024-09-13T17:26:42.345052+0000", 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "active_mgr_features": 4540138322906710015, 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T17:38:13.880 INFO:teuthology.orchestra.run.smithi044.stdout: "standbys": [ 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 25085, 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "y", 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540138322906710015, 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.881 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.885 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.886 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.887 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.891 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.892 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.894 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-09-13T17:38:13.895 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.896 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.897 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T17:38:13.900 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T17:38:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T17:38:13.902 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.903 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.904 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T17:38:13.904 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T17:38:13.904 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.904 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.904 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.905 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T17:38:13.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T17:38:13.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.907 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-09-13T17:38:13.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.910 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.910 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.910 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:13.910 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:13.910 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:13.910 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:13.911 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:13.911 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:13.911 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.911 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.911 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.912 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.912 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:13.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:13.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.913 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:13.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.916 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.918 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.919 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap" 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.920 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.921 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.922 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T17:38:13.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.923 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.923 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-09-13T17:38:13.924 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-09-13T17:38:13.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.924 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.924 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:13.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:13.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.925 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.925 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T17:38:13.925 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T17:38:13.925 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.926 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.926 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-09-13T17:38:13.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-09-13T17:38:13.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:13.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.927 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:13.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.927 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T17:38:13.928 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.928 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.928 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:13.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:13.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:13.929 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-09-13T17:38:13.929 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:13.929 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:13.929 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-09-13T17:38:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-09-13T17:38:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-09-13T17:38:13.931 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.931 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.931 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.931 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T17:38:13.931 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.932 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.932 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.932 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-09-13T17:38:13.932 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-09-13T17:38:13.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.933 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.933 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-09-13T17:38:13.933 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.933 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.933 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.934 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T17:38:13.934 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.934 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-09-13T17:38:13.935 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-09-13T17:38:13.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.935 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.935 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.935 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-09-13T17:38:13.935 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.936 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.936 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T17:38:13.936 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.936 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.937 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.937 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-09-13T17:38:13.937 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-09-13T17:38:13.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.938 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.938 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.938 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.938 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.938 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.939 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T17:38:13.939 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T17:38:13.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.939 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.940 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-09-13T17:38:13.942 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-09-13T17:38:13.943 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-09-13T17:38:13.944 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.945 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.948 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-09-13T17:38:13.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-09-13T17:38:13.950 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.952 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T17:38:13.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.954 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.955 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.956 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.961 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.962 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.968 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.969 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-09-13T17:38:13.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-09-13T17:38:13.971 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-09-13T17:38:13.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.974 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.975 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.977 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.978 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-09-13T17:38:13.978 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-09-13T17:38:13.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.979 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T17:38:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.981 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:13.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:13.983 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.989 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.990 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.991 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.993 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.994 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.995 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.996 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:13.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-09-13T17:38:14.000 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T17:38:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.003 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-09-13T17:38:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T17:38:14.006 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T17:38:14.007 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.008 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.008 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.008 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-09-13T17:38:14.008 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-09-13T17:38:14.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.010 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.012 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.013 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.014 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.015 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.015 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.015 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.015 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.016 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.017 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.019 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.020 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.025 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.026 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.027 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.027 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.027 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T17:38:14.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T17:38:14.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.028 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.028 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.028 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.028 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.028 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.029 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.029 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.029 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.029 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.029 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.030 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T17:38:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T17:38:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T17:38:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T17:38:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T17:38:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T17:38:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-09-13T17:38:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-09-13T17:38:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T17:38:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T17:38:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T17:38:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T17:38:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.068 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.069 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.070 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.071 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.072 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.074 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.075 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.077 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.078 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.079 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.080 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.084 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.085 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.087 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.089 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.090 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.092 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.093 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.094 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.095 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.096 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.098 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.100 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.101 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.102 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.103 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-09-13T17:38:14.104 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-09-13T17:38:14.105 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.107 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.108 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.109 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.110 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.111 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.112 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.113 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.114 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.115 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.117 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.118 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T17:38:14.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T17:38:14.120 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-09-13T17:38:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.123 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.128 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.130 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-09-13T17:38:14.131 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.132 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.133 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T17:38:14.135 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.136 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-09-13T17:38:14.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.142 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.144 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.145 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.149 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.150 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.152 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.157 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.159 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T17:38:14.162 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.163 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.166 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.167 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.167 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.168 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.168 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.169 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.170 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-09-13T17:38:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-09-13T17:38:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-09-13T17:38:14.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.184 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.186 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.187 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.188 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.189 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.190 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.191 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.192 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.195 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.199 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.202 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.204 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.204 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.204 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.204 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.205 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.205 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.206 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.206 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.206 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.209 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.209 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-09-13T17:38:14.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.213 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.214 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.216 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.217 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.218 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-09-13T17:38:14.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.220 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.221 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.222 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.223 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.224 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.225 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.227 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.227 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.227 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.227 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.228 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.228 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.228 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.228 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.228 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-09-13T17:38:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.235 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-09-13T17:38:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-09-13T17:38:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.261 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.262 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-09-13T17:38:14.264 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.264 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.264 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.264 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-09-13T17:38:14.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-09-13T17:38:14.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.265 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.265 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.265 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.267 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.268 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.270 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.271 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.273 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.275 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-09-13T17:38:14.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-09-13T17:38:14.278 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.279 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-09-13T17:38:14.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.283 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.285 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.286 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-09-13T17:38:14.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T17:38:14.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-09-13T17:38:14.289 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.292 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.294 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.295 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.296 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.298 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.299 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.300 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.302 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.303 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.304 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.305 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.307 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.308 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.309 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.310 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.313 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.314 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.315 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.318 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.319 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.320 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.321 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.322 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.326 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.328 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-09-13T17:38:14.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-09-13T17:38:14.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-09-13T17:38:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.335 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.337 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-09-13T17:38:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.341 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.344 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.348 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.350 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.351 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-09-13T17:38:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.353 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.355 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.356 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.358 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.361 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.363 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.364 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.367 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.368 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-09-13T17:38:14.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.371 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.373 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.375 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.377 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.378 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.379 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T17:38:14.382 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.383 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.384 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.386 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.387 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.388 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.389 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.390 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.391 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.392 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.393 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.394 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.395 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.397 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.398 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.399 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.400 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.401 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.402 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.403 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.404 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.405 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-09-13T17:38:14.407 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-09-13T17:38:14.408 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-09-13T17:38:14.409 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 25132, 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "z", 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540138322906710015, 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.411 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.413 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.414 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.421 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.423 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.424 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-09-13T17:38:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-09-13T17:38:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-09-13T17:38:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap" 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-09-13T17:38:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-09-13T17:38:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-09-13T17:38:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-09-13T17:38:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.461 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-09-13T17:38:14.462 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.464 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.465 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T17:38:14.466 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T17:38:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.468 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.470 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-09-13T17:38:14.471 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.472 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.473 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.475 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.476 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T17:38:14.477 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.478 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.479 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.480 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.481 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-09-13T17:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-09-13T17:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T17:38:14.500 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.501 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.502 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.506 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.507 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T17:38:14.508 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.509 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.510 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.512 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.513 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.514 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.515 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-09-13T17:38:14.516 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.517 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.518 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.519 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.522 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.523 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.524 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.527 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.528 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.529 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.530 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.531 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.533 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T17:38:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T17:38:14.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T17:38:14.537 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.538 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.540 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.541 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.542 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.543 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.544 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.545 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.546 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.547 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.548 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.550 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.551 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T17:38:14.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T17:38:14.553 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.554 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.557 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.558 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.563 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.564 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T17:38:14.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T17:38:14.566 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.567 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.568 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.569 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.570 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.573 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.575 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.576 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.576 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-09-13T17:38:14.576 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.576 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.577 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.579 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.580 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.582 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.583 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.584 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-09-13T17:38:14.586 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.587 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.588 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.590 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.591 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.593 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.596 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.598 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.600 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-09-13T17:38:14.601 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-09-13T17:38:14.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-09-13T17:38:14.603 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.605 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T17:38:14.607 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T17:38:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.610 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.611 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.612 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.614 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-09-13T17:38:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T17:38:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-09-13T17:38:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.622 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.626 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.627 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.628 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.630 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-09-13T17:38:14.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-09-13T17:38:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-09-13T17:38:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.634 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.635 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-09-13T17:38:14.640 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.641 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.642 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.643 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.646 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.647 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T17:38:14.648 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.649 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.650 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.653 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.654 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.655 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.656 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.659 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.660 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-09-13T17:38:14.662 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.663 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.664 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.665 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.667 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.668 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.670 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.673 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.674 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.676 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.679 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.680 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.681 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.682 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.684 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.685 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.687 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.689 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.691 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.694 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-09-13T17:38:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-09-13T17:38:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-09-13T17:38:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-09-13T17:38:14.740 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-09-13T17:38:14.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.743 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.745 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.746 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.750 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.751 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-09-13T17:38:14.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.755 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.756 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.757 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.758 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.759 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.760 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.763 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.764 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.765 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.766 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.767 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.769 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.770 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.771 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.773 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.775 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.776 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.777 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.778 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.779 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.780 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.781 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.791 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.794 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.795 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.797 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.798 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.799 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.800 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T17:38:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-09-13T17:38:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-09-13T17:38:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-09-13T17:38:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-09-13T17:38:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard", 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "restful", 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_schedule" 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-09-13T17:38:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-09-13T17:38:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T17:38:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-09-13T17:38:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-09-13T17:38:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-09-13T17:38:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-09-13T17:38:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-09-13T17:38:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap" 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T17:38:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-09-13T17:38:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T17:38:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T17:38:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-09-13T17:38:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-09-13T17:38:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T17:38:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-09-13T17:38:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T17:38:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T17:38:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-09-13T17:38:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-09-13T17:38:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-09-13T17:38:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T17:38:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-09-13T17:38:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-09-13T17:38:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-09-13T17:38:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-09-13T17:38:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T17:38:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-09-13T17:38:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-09-13T17:38:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T17:38:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T17:38:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-09-13T17:38:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T17:38:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T17:38:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.000 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.001 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.002 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.004 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.005 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.008 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.009 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.011 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.013 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.014 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.015 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-09-13T17:38:15.016 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.017 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.020 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.021 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T17:38:15.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T17:38:15.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T17:38:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-09-13T17:38:15.025 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-09-13T17:38:15.026 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.027 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.031 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.032 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.033 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.034 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.035 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-09-13T17:38:15.036 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.037 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.038 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.039 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.040 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.041 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.042 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-09-13T17:38:15.043 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.044 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-09-13T17:38:15.045 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.046 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.047 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.048 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.050 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.051 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-09-13T17:38:15.052 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-09-13T17:38:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-09-13T17:38:15.054 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.055 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.057 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-09-13T17:38:15.059 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.063 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-09-13T17:38:15.065 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-09-13T17:38:15.066 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.069 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.070 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.072 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.073 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.076 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.077 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.078 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-09-13T17:38:15.080 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-09-13T17:38:15.081 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.083 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.084 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.085 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.086 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.087 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.088 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.091 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.093 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.094 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.098 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:15.099 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.101 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.104 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.105 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-09-13T17:38:15.106 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.107 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.108 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.109 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.110 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.111 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.113 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.114 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-09-13T17:38:15.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-09-13T17:38:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-09-13T17:38:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.155 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.156 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.158 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.159 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:15.161 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.162 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T17:38:15.163 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.164 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.165 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.169 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.171 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-09-13T17:38:15.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-09-13T17:38:15.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.174 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.176 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.178 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.179 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.180 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.182 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-09-13T17:38:15.183 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-09-13T17:38:15.184 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.186 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-09-13T17:38:15.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.189 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.191 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.192 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.196 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.197 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.198 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.200 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.201 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.202 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.203 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.206 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.207 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.209 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.210 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.216 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.217 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.218 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.219 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.220 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.224 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.225 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.226 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.228 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.229 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.230 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.232 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-09-13T17:38:15.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.234 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.235 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-09-13T17:38:15.237 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.238 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.239 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.240 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.242 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.243 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.244 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.245 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.246 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.248 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.249 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.251 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.254 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.255 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.256 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.257 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.259 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.260 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.262 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-09-13T17:38:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-09-13T17:38:15.266 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-09-13T17:38:15.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.269 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.271 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.273 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.274 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.277 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.278 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.282 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.285 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.287 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.288 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.289 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.292 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.293 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.295 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.296 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.297 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-09-13T17:38:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-09-13T17:38:15.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-09-13T17:38:15.301 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-09-13T17:38:15.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.305 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.307 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.309 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-09-13T17:38:15.312 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "services": { 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard": "https://172.21.15.44:8443/" 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "always_on_modules": { 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus": [ 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-09-13T17:38:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific": [ 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-09-13T17:38:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "quincy": [ 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-09-13T17:38:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "reef": [ 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-09-13T17:38:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "last_failure_osd_epoch": 93, 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "active_clients": [ 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 459441937 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "libcephsqlite", 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1436364259 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2911905836 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 935945782 2024-09-13T17:38:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3115411277 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3170300172 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T17:38:15.321 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:38:15.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:14 smithi100 ceph-mon[35055]: pgmap v360: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T17:38:15.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:14 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2152086665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:15.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:14 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2166283056' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T17:38:15.323 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:14 smithi070 ceph-mon[34502]: pgmap v360: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T17:38:15.323 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:14 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2152086665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:15.323 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:14 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2166283056' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facb80a1200 msgr2=0x7facb80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facb80a1200 0x7facb80a36c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7facc8009fd0 tx=0x7facc8009680 comp rx=0 tx=0).stop 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 msgr2=0x7faccc196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc196750 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b450 tx=0x7facbc00b910 comp rx=0 tx=0).stop 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 shutdown_connections 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7facb80a1200 0x7facb80a36c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc19ab40 0x7faccc19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc103fe0 0x7faccc196750 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 --2- 172.21.15.44:0/2166283056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc0fee90 0x7faccc196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 >> 172.21.15.44:0/2166283056 conn(0x7faccc0fa900 msgr2=0x7faccc0fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 shutdown_connections 2024-09-13T17:38:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:13.870+0000 7facd33aa640 1 -- 172.21.15.44:0/2166283056 wait complete. 2024-09-13T17:38:15.330 DEBUG:tasks.check_counter:Getting stats from a 2024-09-13T17:38:15.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-13T17:38:15.331 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:14 smithi044 ceph-mon[31175]: pgmap v360: 129 pgs: 129 active+clean; 2.4 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T17:38:15.331 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:14 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2152086665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:15.331 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:14 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2166283056' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T17:38:15.549 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:38:16.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:16 smithi044 ceph-mon[31175]: pgmap v361: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-13T17:38:16.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:16 smithi070 ceph-mon[34502]: pgmap v361: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-13T17:38:16.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:16 smithi100 ceph-mon[35055]: pgmap v361: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-13T17:38:17.278 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:38:17.278 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 237, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 226, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1563472, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 128800, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.044214335, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.015856972, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.039072105, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:17.279 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 225, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013752449, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000061121 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1563472, 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 128800 2024-09-13T17:38:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 175, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3328, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 183360, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.016045137, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.009882530, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.003835089, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 173, 2024-09-13T17:38:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.019568108, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000113110 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3328, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 183360 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-09-13T17:38:17.282 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.000335579, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000259199, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:17.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-09-13T17:38:17.284 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:17.285 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:17.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 149011, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 32, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:17.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 239, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1529996, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 239, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 239, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1529996, 2024-09-13T17:38:17.289 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-09-13T17:38:17.290 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:17.290 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:17.290 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:17.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:17.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:17.290 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-13T17:38:18.136 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-13T17:38:18.136 DEBUG:tasks.check_counter:Getting stats from d 2024-09-13T17:38:18.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-13T17:38:18.350 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:38:18.471 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:18 smithi044 ceph-mon[31175]: pgmap v362: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-13T17:38:18.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:18 smithi070 ceph-mon[34502]: pgmap v362: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-13T17:38:18.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:18 smithi100 ceph-mon[35055]: pgmap v362: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:20.140 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.001220376, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000188624, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:20.141 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 178, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 320, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 183792, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.016304428, 2024-09-13T17:38:20.142 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.011765685, 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.002004055, 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 175, 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.018143828, 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000103679 2024-09-13T17:38:20.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 183792 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 246, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 233, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1567840, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 132208, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.047574394, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.016863013, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.043342294, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 233, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.018349279, 2024-09-13T17:38:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000078752 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1567840, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 132208 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148994, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 31, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-09-13T17:38:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 244, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1530646, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 244, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 244, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1530646, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:20.153 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:38:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:20 smithi044 ceph-mon[31175]: pgmap v363: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-13T17:38:20.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:20 smithi070 ceph-mon[34502]: pgmap v363: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-13T17:38:20.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:20 smithi100 ceph-mon[35055]: pgmap v363: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-13T17:38:20.958 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-13T17:38:20.958 DEBUG:tasks.check_counter:Getting stats from g 2024-09-13T17:38:20.959 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-13T17:38:21.172 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:38:22.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:22 smithi044 ceph-mon[31175]: pgmap v364: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-13T17:38:22.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:22 smithi070 ceph-mon[34502]: pgmap v364: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-13T17:38:22.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:22 smithi100 ceph-mon[35055]: pgmap v364: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-13T17:38:22.900 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:38:22.900 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:22.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-09-13T17:38:22.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-09-13T17:38:22.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-09-13T17:38:22.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-09-13T17:38:22.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:22.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-09-13T17:38:22.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.001385835, 2024-09-13T17:38:22.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T17:38:22.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000306702, 2024-09-13T17:38:22.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:22.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:22.902 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:22.903 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:22.903 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:22.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:22.903 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:22.904 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:22.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:22.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.904 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:22.904 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:22.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.905 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:22.905 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 243, 2024-09-13T17:38:22.905 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 230, 2024-09-13T17:38:22.905 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1567024, 2024-09-13T17:38:22.905 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 130432, 2024-09-13T17:38:22.906 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-09-13T17:38:22.906 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-09-13T17:38:22.906 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.045667926, 2024-09-13T17:38:22.906 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.016909374, 2024-09-13T17:38:22.906 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.040155845, 2024-09-13T17:38:22.907 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:22.907 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:22.907 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 230, 2024-09-13T17:38:22.907 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.017939691, 2024-09-13T17:38:22.907 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000077998 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1567024, 2024-09-13T17:38:22.908 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 130432 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 175, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 320, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 183264, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.015195074, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.010770649, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001872302, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:22.909 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 172, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.017188038, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000099930 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 183264 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-09-13T17:38:22.910 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:22.911 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:22.912 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:22.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148977, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 30, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-09-13T17:38:22.914 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-09-13T17:38:22.915 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 241, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1530256, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 241, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 241, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1530256, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:22.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:22.917 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:38:23.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:23 smithi044 ceph-mon[31175]: pgmap v365: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-13T17:38:23.665 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-13T17:38:23.665 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-13T17:38:23.666 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-13T17:38:23.666 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-13T17:38:23.666 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-13T17:38:23.666 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-13T17:38:23.666 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-13T17:38:23.666 DEBUG:tasks.check_counter:Getting stats from j 2024-09-13T17:38:23.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-13T17:38:23.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:23 smithi070 ceph-mon[34502]: pgmap v365: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-13T17:38:23.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:23 smithi100 ceph-mon[35055]: pgmap v365: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-09-13T17:38:23.874 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T17:38:25.557 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T17:38:25.557 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:25.557 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-09-13T17:38:25.557 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.001386214, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000276960, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:25.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 181, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3328, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 184416, 2024-09-13T17:38:25.559 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.017939119, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.011554425, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.003921998, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 179, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.019767339, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000110432 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:25.560 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3328, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 184416 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 244, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 233, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1565376, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 132944, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.046496509, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.017106541, 2024-09-13T17:38:25.561 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.041973810, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 232, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013757358, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000059298 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1565376, 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 132944 2024-09-13T17:38:25.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:25.564 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148977, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 30, 2024-09-13T17:38:25.565 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-09-13T17:38:25.566 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 246, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1530906, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:25.567 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 246, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 246, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1530906, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T17:38:25.568 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T17:38:25.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:25 smithi044 ceph-mon[31175]: pgmap v366: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-13T17:38:25.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:25 smithi070 ceph-mon[34502]: pgmap v366: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-13T17:38:26.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:25 smithi100 ceph-mon[35055]: pgmap v366: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-13T17:38:26.163 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-13T17:38:26.163 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-13T17:38:26.164 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-13T17:38:26.164 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-13T17:38:26.164 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-13T17:38:26.164 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-13T17:38:26.164 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-13T17:38:26.164 DEBUG:tasks.check_counter:Getting stats from b 2024-09-13T17:38:26.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-13T17:38:26.373 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:38:27.929 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:27 smithi070 ceph-mon[34502]: pgmap v367: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:28.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:27 smithi100 ceph-mon[35055]: pgmap v367: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:28.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:27 smithi044 ceph-mon[31175]: pgmap v367: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:28.393 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T17:38:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 2359, 2024-09-13T17:38:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 2359, 2024-09-13T17:38:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 671519, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 732860, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 6, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.187040528, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.060792838, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.048172939, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.063171089, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2358, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.081288581, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000034473 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 3008, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 265, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 250, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1604156, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 134140, 2024-09-13T17:38:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 3, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.045736968, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.017360707, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.040802454, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000058477, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 249, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.015111750, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000060689 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 1603600, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 133456 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 3, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 175, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 588, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 459607, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 6, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 4, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.011078665, 2024-09-13T17:38:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.007224230, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.001916005, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000023544, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 172, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.018151351, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000105531 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 459296 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3957, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.094781496, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000023952 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 0, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 0, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 0, 2024-09-13T17:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 1, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 0, 2024-09-13T17:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T17:38:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 0 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 0, 2024-09-13T17:38:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2024-09-13T17:38:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 3, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 0, 2024-09-13T17:38:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 0, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 0, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 0, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 1, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 0, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 4194304, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 4195730, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 4195338, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 3 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 3, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 3, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 1, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 1, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 0, 2024-09-13T17:38:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 48344, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 207124 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 0, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 0, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 0, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 0, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 0, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2024-09-13T17:38:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2024-09-13T17:38:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2024-09-13T17:38:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2024-09-13T17:38:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 2, 2024-09-13T17:38:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 2, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 2, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 1738, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 181353, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 48, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 9520, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 193, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 8368, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 13, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 2358, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 21, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 3, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 2358, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2358, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 1.210963278, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000513555 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2358, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 2359 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 2358, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 2358, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 0, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 1528, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 0, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 828, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 0, 2024-09-13T17:38:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 1, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2024-09-13T17:38:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2024-09-13T17:38:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 105, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 15, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 4, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 4, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2024-09-13T17:38:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 0, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 2, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 0 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 0, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2024-09-13T17:38:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed_ops": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2024-09-13T17:38:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 2623, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 2023652, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 2623, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 2623, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 2023652, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 2358, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 1431, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 2358, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1431, 2024-09-13T17:38:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 2358, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 2358, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 2358, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 2358, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x55c8501188c0": { 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:28.442 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T17:38:29.237 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-13T17:38:29.237 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-13T17:38:29.237 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-09-13T17:38:29.237 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-13T17:38:29.237 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=0 2024-09-13T17:38:29.238 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-13T17:38:29.238 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T17:38:29.238 DEBUG:tasks.check_counter:Getting stats from e 2024-09-13T17:38:29.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-13T17:38:29.440 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:38:29.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:29 smithi070 ceph-mon[34502]: pgmap v368: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-13T17:38:30.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:29 smithi100 ceph-mon[35055]: pgmap v368: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-13T17:38:30.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:29 smithi044 ceph-mon[31175]: pgmap v368: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 249, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 236, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1568656, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 133984, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 0, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.049009361, 2024-09-13T17:38:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.017415502, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.044747920, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 236, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.018121199, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000076784 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 1568656, 2024-09-13T17:38:31.211 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 133984 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 2, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 179, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 320, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 183968, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.012506700, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.008566266, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.001540164, 2024-09-13T17:38:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 176, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.017522609, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000099560 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 183968 2024-09-13T17:38:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 0, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 2, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 0, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 0, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 2, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 1, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.000287084, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.000222936, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 0, 2024-09-13T17:38:31.215 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:31.216 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:31.217 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 148955, 2024-09-13T17:38:31.218 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 30, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:31.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 928, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 2, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2024-09-13T17:38:31.220 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 247, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1531036, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 247, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 247, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1531036, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:31.221 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T17:38:31.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:31 smithi070 ceph-mon[34502]: pgmap v369: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-13T17:38:31.988 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-13T17:38:31.988 DEBUG:tasks.check_counter:Getting stats from h 2024-09-13T17:38:31.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-13T17:38:32.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:31 smithi100 ceph-mon[35055]: pgmap v369: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:32.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:31 smithi044 ceph-mon[31175]: pgmap v369: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:32.188 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:38:33.948 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:33 smithi070 ceph-mon[34502]: pgmap v370: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:33.973 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1334, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 878, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 359086, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 366694, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 12, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.099958887, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.036296415, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.063806545, 2024-09-13T17:38:33.974 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.005440268, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 879, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.054581684, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000062095 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2081, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000450605, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000216 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 3008, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.975 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 69, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 250, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 12889, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 905473, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 53, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 6, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.028831473, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.008893119, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.016490854, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000317787, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 247, 2024-09-13T17:38:33.976 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.029138043, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000117967 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 59, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000005164, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000087 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 2160, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 468272 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 327, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 337, 2024-09-13T17:38:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1613667, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 173773, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 10, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 5, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.052096284, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.018706643, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.048671267, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000115826, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 334, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.020786353, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000062234 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:33.978 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 69, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000012909, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000187 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 1610416, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 146352 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2024-09-13T17:38:33.979 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 58, 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.002283027, 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000039362 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000010542, 2024-09-13T17:38:33.980 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000005271 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 9, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 6, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 6, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000854371, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000142395 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 3, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2024-09-13T17:38:33.981 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 11, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 16, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 0, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 16, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 0, 2024-09-13T17:38:33.982 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 13, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 4, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 0, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 0, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 3, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 24, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 16, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 1, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 6, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2024-09-13T17:38:33.983 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 5, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 298, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 6, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2024-09-13T17:38:33.984 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 0, 2024-09-13T17:38:33.985 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 1 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 14, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 9, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 10, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2024-09-13T17:38:33.986 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.987 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 8, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 8, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 1, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 1, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 0, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 4194304, 2024-09-13T17:38:33.988 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 4196937, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 0, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 8, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.012069020, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001508627 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 0 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 20, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 22, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 2, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 19, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 22, 2024-09-13T17:38:33.989 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 3, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 16, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 20, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 4, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 0, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 6, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 6, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 49908, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 198932 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 10, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 0, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 9, 2024-09-13T17:38:33.990 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 22, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 0, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000465739, 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000155246 2024-09-13T17:38:33.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.992 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3, 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000388632, 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000129544 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2024-09-13T17:38:33.994 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.996 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.997 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.998 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2024-09-13T17:38:33.998 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.998 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.998 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:33.999 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.000 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.002 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.003 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2024-09-13T17:38:34.004 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 1, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 4, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 3, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 1, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 5, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 1, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 710, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:34.007 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:34.008 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 240375, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 90, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 88, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 1, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:34.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 9520, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 193, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 85453, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 342, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:34.010 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 47, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 13250, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 33, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 47, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 47, 2024-09-13T17:38:34.011 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.208993710, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.004446674 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 47, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 71 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 47, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 0, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 47, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2024-09-13T17:38:34.012 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 10, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 8, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 8, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 6, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2024-09-13T17:38:34.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 20, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2024-09-13T17:38:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2024-09-13T17:38:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 105, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 15, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 8, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 8, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 17, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 0, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 2 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.016 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 1, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2024-09-13T17:38:34.017 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 0, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed_ops": 0, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 0, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 1726, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1811308, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 1726, 2024-09-13T17:38:34.018 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 1726, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1811308, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:34.019 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 47, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 13250, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 47, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 13250, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 47, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 47, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 47, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 47, 2024-09-13T17:38:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x55a41c102500": { 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 8, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 2633, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 8, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 8, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 2633, 2024-09-13T17:38:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:34.025 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T17:38:34.026 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:33 smithi100 ceph-mon[35055]: pgmap v370: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:34.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:33 smithi044 ceph-mon[31175]: pgmap v370: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-13T17:38:34.743 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T17:38:34.744 DEBUG:tasks.check_counter:Getting stats from k 2024-09-13T17:38:34.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-13T17:38:34.940 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T17:38:35.908 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:35 smithi044 ceph-mon[31175]: pgmap v371: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-13T17:38:35.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:35 smithi070 ceph-mon[34502]: pgmap v371: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-13T17:38:36.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:35 smithi100 ceph-mon[35055]: pgmap v371: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 252, 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 239, 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1569472, 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 135760, 2024-09-13T17:38:36.745 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 0, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.049329061, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.017787953, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.044615463, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 239, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.018707708, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000078274 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.746 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 1569472, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 135760 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 2, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 182, 2024-09-13T17:38:36.747 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 320, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 184496, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.012673801, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.008794358, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.001599402, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 179, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.016859141, 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000094185 2024-09-13T17:38:36.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 184496 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 0, 2024-09-13T17:38:36.749 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 2, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 0, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 0, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 2, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 1, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.000263162, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.000191595, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:36.750 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 0, 2024-09-13T17:38:36.751 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:36.752 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:36.753 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:36.753 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:36.753 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:36.753 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:36.753 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:36.757 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:36.757 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:36.757 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:36.758 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 148994, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 31, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2024-09-13T17:38:36.759 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2024-09-13T17:38:36.760 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 928, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 2, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 250, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1531426, 2024-09-13T17:38:36.761 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 250, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 250, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1531426, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T17:38:36.762 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T17:38:37.548 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-13T17:38:37.548 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-13T17:38:37.548 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-13T17:38:37.548 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-13T17:38:37.548 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-13T17:38:37.549 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-13T17:38:37.549 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-13T17:38:37.549 DEBUG:tasks.check_counter:Getting stats from c 2024-09-13T17:38:37.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-13T17:38:37.760 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:38:37.881 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:37 smithi100 ceph-mon[35055]: pgmap v372: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:37.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:37 smithi070 ceph-mon[34502]: pgmap v372: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:38.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:37 smithi044 ceph-mon[31175]: pgmap v372: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:39.453 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 30, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 68, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 7705, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 26182, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 9, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.010805928, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.001710556, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.003823589, 2024-09-13T17:38:39.454 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000674819, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 66, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014850231, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000225003 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 59, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000011743, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000199 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3296, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 13312 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.455 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1656, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1195, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1972108, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 983375, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 52, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 9, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.181683960, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.072469376, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.130384681, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.004548277, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1195, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.079978640, 2024-09-13T17:38:39.456 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000066927 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2128, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000475339, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000223 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1614144, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 141888 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 29, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 243, 2024-09-13T17:38:39.457 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 6110, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 520037, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 14, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.018219737, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.008904849, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.006505729, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000427894, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 240, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.031948076, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000133116 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:39.458 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 63, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000014413, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000228 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 465104 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 49, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001635115, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000033369 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000013028, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000006514 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-09-13T17:38:39.460 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 9, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 6, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 6, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001957418, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000326236 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 3, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-09-13T17:38:39.461 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 16, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 16, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 13, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 4, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 3, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 4, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 3, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 21, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 15, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 1, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 4, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-09-13T17:38:39.462 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 5, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 298, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T17:38:39.463 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T17:38:39.464 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 14, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 9, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 10, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-09-13T17:38:39.465 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-09-13T17:38:39.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 3, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 3, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 1, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 1, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-09-13T17:38:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4195730, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.004188694, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001396231 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 20, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 22, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 2, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 19, 2024-09-13T17:38:39.468 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 23, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 4, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 16, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 20, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 4, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 4, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 4, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 48944, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 198932 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 12, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 9, 2024-09-13T17:38:39.469 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 19, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 1, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 6, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001957418, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000326236 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.470 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.471 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.472 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.475 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.476 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.479 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 2, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 0, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 2, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 5, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 2, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 708, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:39.482 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:39.483 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 264492, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 103, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 88, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 1, 2024-09-13T17:38:39.484 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9520, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 193, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 86893, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 346, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:39.485 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 40, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 11887, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 27, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 40, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 40, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.216994198, 2024-09-13T17:38:39.486 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005424854 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 40, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 63 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 40, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 0, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 40, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 0, 2024-09-13T17:38:39.487 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 3, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 7, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 6, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-09-13T17:38:39.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-09-13T17:38:39.489 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 15, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 10, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 10, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-09-13T17:38:39.490 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 16, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 1 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.491 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1711, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1813260, 2024-09-13T17:38:39.492 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1711, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1711, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1813260, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:39.493 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 40, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 11887, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 40, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 11887, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.494 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 40, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 40, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 40, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 40, 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:39.495 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:39.496 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x55e258662500": { 2024-09-13T17:38:39.497 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 3, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1426, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 3, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 3, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1426, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:39.498 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:39.499 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.499 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:39.499 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-13T17:38:39.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:39 smithi100 ceph-mon[35055]: pgmap v373: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:39.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:39 smithi070 ceph-mon[34502]: pgmap v373: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:40.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:39 smithi044 ceph-mon[31175]: pgmap v373: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-13T17:38:40.267 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T17:38:40.267 DEBUG:tasks.check_counter:Getting stats from f 2024-09-13T17:38:40.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-13T17:38:40.472 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:38:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:41 smithi100 ceph-mon[35055]: pgmap v374: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:41.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:41 smithi070 ceph-mon[34502]: pgmap v374: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:42.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:41 smithi044 ceph-mon[31175]: pgmap v374: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:42.298 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-13T17:38:42.298 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:42.298 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 2022, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2005, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 10510196, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 681810, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.230838241, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.070113097, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.102235809, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.057111507, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2005, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.087350200, 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000043566 2024-09-13T17:38:42.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1608944, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 138800 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 612, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 791, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 8542956, 2024-09-13T17:38:42.300 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 656640, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 7, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.071282728, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.025123355, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.023266872, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.016731553, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 788, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.040012285, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000050777 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.301 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 466864 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 786, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 788, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 5579803, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 244453, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.076470231, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.025725998, 2024-09-13T17:38:42.302 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.020929724, 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.022953726, 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 786, 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.031276404, 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000039791 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.303 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:42.304 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5373, 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.144417859, 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000026878 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:42.305 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.307 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-09-13T17:38:42.307 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-09-13T17:38:42.307 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-09-13T17:38:42.307 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-09-13T17:38:42.307 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 32, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 12, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 20, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 15, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-09-13T17:38:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-09-13T17:38:42.309 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T17:38:42.310 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-09-13T17:38:42.311 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-09-13T17:38:42.312 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 2487, 2024-09-13T17:38:42.313 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 25759559, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 26354805, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 26342362, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 2487 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 27, 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 28, 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 1, 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 11, 2024-09-13T17:38:42.315 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 11, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 0, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 32, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 32, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 49232, 2024-09-13T17:38:42.316 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 207124 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.317 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-09-13T17:38:42.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-09-13T17:38:42.319 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.320 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.322 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.325 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.326 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-09-13T17:38:42.327 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 2, 2024-09-13T17:38:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 7, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 5, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 5, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 3298, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:42.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:42.330 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:42.331 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 205543, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 58, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9520, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 193, 2024-09-13T17:38:42.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 132884, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 578, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 3142, 2024-09-13T17:38:42.333 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 26, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 4, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 3142, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3142, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.601956579, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000509852 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3142, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 3143 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 3142, 2024-09-13T17:38:42.334 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 3142, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 1668, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 1471, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-09-13T17:38:42.335 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 2, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-09-13T17:38:42.336 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-09-13T17:38:42.337 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 15, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 7, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 7, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-09-13T17:38:42.338 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-09-13T17:38:42.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 3416, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 24308512, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 3416, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 3416, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 24308512, 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 3142, 2024-09-13T17:38:42.341 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 22160511, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 3142, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 22160511, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:42.342 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 3142, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 3142, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 3142, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 3142, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:42.343 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x5570f4694820": { 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:42.346 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:42.346 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:42.346 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.346 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:42.346 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-13T17:38:43.141 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T17:38:43.141 DEBUG:tasks.check_counter:Getting stats from i 2024-09-13T17:38:43.141 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-13T17:38:43.338 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:38:43.765 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:43 smithi100 ceph-mon[35055]: pgmap v375: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:43.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:43 smithi070 ceph-mon[34502]: pgmap v375: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:44.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:43 smithi044 ceph-mon[31175]: pgmap v375: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:44.999 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-13T17:38:44.999 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 4547, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 4230, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1591204, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 14442737, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 15, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.474877291, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.164052615, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.166485711, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.116444536, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4231, 2024-09-13T17:38:45.000 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.352885941, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000319755 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3167, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000781592, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000246 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 416912, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 110816 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1783, 2024-09-13T17:38:45.001 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1757, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 2017336, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 9209090, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 287, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.264015680, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.068447356, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.159721677, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.037155299, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1755, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.180470373, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000672632 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.002 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 566, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000069656, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000123 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1730880, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 549632 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3860, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 6576, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1324027, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 6917837, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 18, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 7, 2024-09-13T17:38:45.003 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.468586026, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.179002485, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.222789430, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.075669630, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 6572, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.362671187, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000207344 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 7844, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001960237, 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000249 2024-09-13T17:38:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 42720, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 136592 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 6234, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.719695009, 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000275857 2024-09-13T17:38:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 16, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000388793, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000024299 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 795, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 795, 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-09-13T17:38:45.006 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 795, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 3.293751750, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004143083 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 340, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 42, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 13, 2024-09-13T17:38:45.007 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 29, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 23, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 3, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 3, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 873, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 859, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-09-13T17:38:45.008 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 315, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 315, 2024-09-13T17:38:45.009 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 203, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 23, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 180, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 112, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 619, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 1208, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 599, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 637, 2024-09-13T17:38:45.010 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 614, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 2, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 26 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 10, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 20, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 2, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 3, 2024-09-13T17:38:45.011 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 1, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 23, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-09-13T17:38:45.012 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 2487, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 2421, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 2421, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 66, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 51, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 56, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 55, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 55, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-09-13T17:38:45.013 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 1, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 25759559, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 26354805, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2423, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 4.934600622, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002036566 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 36, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 60, 2024-09-13T17:38:45.014 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 24, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 20, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 20, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 0, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 42, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 43, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 1, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 3, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 21, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 18, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 55436, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 207124 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.015 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 2057, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 1, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 795, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 42, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 2, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.003886498, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000777299 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 132, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.021536054, 2024-09-13T17:38:45.016 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000163151 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.017 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000401943, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000133981 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.019 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.020 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 12, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001844092, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000153674 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.011614083, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001451760 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 12, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.058175322, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004847943 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 599, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 3.127764909, 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005221644 2024-09-13T17:38:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.003447285, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001723642 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002643236, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001321618 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.011037723, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005518861 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 6, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.026638629, 2024-09-13T17:38:45.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004439771 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001535663, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000767831 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.025 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 9, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.021224777, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002358308 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002001536, 2024-09-13T17:38:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002001536 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 2, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 7, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 5, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 2, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 123, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 61, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 713, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:45.030 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:45.031 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 322802, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 126, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:45.032 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9520, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 193, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 175588, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 1095, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:45.033 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 4569, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 23287730, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 37, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 4569, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4569, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 25.883302442, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005664981 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-09-13T17:38:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4569, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 5291 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 4569, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 362, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 4207, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 320, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 30, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 24, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 2424, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 176, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-09-13T17:38:45.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 524, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 338, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 55, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-09-13T17:38:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 599, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-09-13T17:38:45.037 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 22, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 12, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 12, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 791, 2024-09-13T17:38:45.038 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 10 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-09-13T17:38:45.039 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 2, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 1, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 2, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 1, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 10186, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 3936714, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:45.040 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 10186, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 10186, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 3936714, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:45.041 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 4569, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 23287887, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 4569, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 23287887, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.042 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 4569, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 4569, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 4569, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 4569, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.043 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 157, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:45.044 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 157, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x560c06bf8500": { 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 2487, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 22160501, 2024-09-13T17:38:45.045 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 2487, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2423, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 22160501, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:45.046 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-13T17:38:45.757 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:45 smithi100 ceph-mon[35055]: pgmap v376: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:45.779 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-13T17:38:45.779 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-13T17:38:45.779 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=10 2024-09-13T17:38:45.779 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-13T17:38:45.780 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-13T17:38:45.780 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=12 2024-09-13T17:38:45.780 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=6 2024-09-13T17:38:45.780 DEBUG:tasks.check_counter:Getting stats from l 2024-09-13T17:38:45.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid ce020790-71f3-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-13T17:38:45.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:45 smithi070 ceph-mon[34502]: pgmap v376: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T17:38:46.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:45 smithi044 ceph-mon[31175]: pgmap v376: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-13T17:38:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T17:38:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 853, 2024-09-13T17:38:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 853, 2024-09-13T17:38:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 228606, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 264551, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.085615401, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.026367907, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.024950253, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.025125794, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 852, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.040453818, 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000047481 2024-09-13T17:38:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3008, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1028, 2024-09-13T17:38:47.645 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1016, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1783278, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 375681, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.126072191, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.039109539, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.063665682, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.030797102, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1015, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.044277462, 2024-09-13T17:38:47.646 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000043623 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1597280, 2024-09-13T17:38:47.647 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 140560 2024-09-13T17:38:47.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.648 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T17:38:47.648 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 841, 2024-09-13T17:38:47.648 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1023, 2024-09-13T17:38:47.648 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 282224, 2024-09-13T17:38:47.648 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 728321, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.112055279, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.037643766, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.029287148, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.032832076, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1020, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.069067903, 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000067713 2024-09-13T17:38:47.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 467392 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-09-13T17:38:47.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4017, 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.124192781, 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000030916 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.653 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 0, 2024-09-13T17:38:47.654 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 1, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-09-13T17:38:47.655 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-09-13T17:38:47.656 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T17:38:47.657 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T17:38:47.658 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-09-13T17:38:47.658 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-09-13T17:38:47.658 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T17:38:47.658 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-09-13T17:38:47.658 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-09-13T17:38:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-09-13T17:38:47.660 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-09-13T17:38:47.661 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 8, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 1, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-09-13T17:38:47.662 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4196937, 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 4196796, 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.663 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 8 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 3, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 3, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 1, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 1, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 0, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 0, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 0, 2024-09-13T17:38:47.664 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 48328, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 207124 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-09-13T17:38:47.665 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-09-13T17:38:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.667 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.668 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-09-13T17:38:47.669 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.670 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-09-13T17:38:47.672 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.673 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-09-13T17:38:47.675 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.676 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.677 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 4, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 3, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 3, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 3816, 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-09-13T17:38:47.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T17:38:47.680 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T17:38:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-09-13T17:38:47.682 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 189383, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 50, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-09-13T17:38:47.683 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9520, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 193, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 8368, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 13, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-09-13T17:38:47.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 2444, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 21, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 3, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 2444, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2444, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.389962329, 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000568724 2024-09-13T17:38:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2444, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 2445 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 2444, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 2444, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 1593, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-09-13T17:38:47.686 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 849, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 1, 2024-09-13T17:38:47.687 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-09-13T17:38:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-09-13T17:38:47.689 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-09-13T17:38:47.689 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-09-13T17:38:47.689 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-09-13T17:38:47.689 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-09-13T17:38:47.689 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-09-13T17:38:47.690 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 15, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 5, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 5, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-09-13T17:38:47.691 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-09-13T17:38:47.692 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 2718, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 2032462, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 2718, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2718, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2032462, 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:47.693 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:47.694 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 2444, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 2638, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2444, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2638, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 2444, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 2444, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2444, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2444, 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:47.696 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:47.697 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x5621ecd24780": { 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-09-13T17:38:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-13T17:38:47.699 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-13T17:38:47.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:47 smithi070 ceph-mon[34502]: pgmap v377: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:48.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:47 smithi100 ceph-mon[35055]: pgmap v377: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:48.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:47 smithi044 ceph-mon[31175]: pgmap v377: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-13T17:38:48.436 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T17:38:48.437 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:38:48.448 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:38:48.448 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:38:48.448 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:38:48.472 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:38:48.472 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:38:48.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 -- 172.21.15.44:0/3052755919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e01050e0 msgr2=0x7f59e010cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 --2- 172.21.15.44:0/3052755919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e01050e0 0x7f59e010cc30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009930 tx=0x7f59d802a420 comp rx=0 tx=0).stop 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 -- 172.21.15.44:0/3052755919 shutdown_connections 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 --2- 172.21.15.44:0/3052755919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59e0100940 0x7f59e0104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 --2- 172.21.15.44:0/3052755919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e01050e0 0x7f59e010cc30 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 --2- 172.21.15.44:0/3052755919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e00fff90 0x7f59e0100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 -- 172.21.15.44:0/3052755919 >> 172.21.15.44:0/3052755919 conn(0x7f59e00fba20 msgr2=0x7f59e00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:48.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 -- 172.21.15.44:0/3052755919 shutdown_connections 2024-09-13T17:38:48.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.687+0000 7f59e80b1640 1 -- 172.21.15.44:0/3052755919 wait complete. 2024-09-13T17:38:48.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 Processor -- start 2024-09-13T17:38:48.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 -- start start 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 0x7f59e00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59e01050e0 0x7f59e0072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0073230 0x7f59e01a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a4aa0 con 0x7f59e00fff90 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a4c10 con 0x7f59e0073230 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e80b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a4dd0 con 0x7f59e01050e0 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e5e4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 0x7f59e00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e5e4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 0x7f59e00727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39546/0 (socket says 172.21.15.44:39546) 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.688+0000 7f59e5e4d640 1 -- 172.21.15.44:0/1485040555 learned_addr learned my addr 172.21.15.44:0/1485040555 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e564c640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59e01050e0 0x7f59e0072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e664e640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0073230 0x7f59e01a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e5e4d640 1 -- 172.21.15.44:0/1485040555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59e01050e0 msgr2=0x7f59e0072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e5e4d640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59e01050e0 0x7f59e0072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e5e4d640 1 -- 172.21.15.44:0/1485040555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0073230 msgr2=0x7f59e01a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e5e4d640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0073230 0x7f59e01a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e5e4d640 1 -- 172.21.15.44:0/1485040555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8009560 con 0x7f59e00fff90 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e5e4d640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 0x7f59e00727b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00eab0 tx=0x7f59dc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00ce40 con 0x7f59e00fff90 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59dc011900 con 0x7f59e00fff90 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e01a50b0 con 0x7f59e00fff90 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0108c0 con 0x7f59e00fff90 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.689+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e01a55c0 con 0x7f59e00fff90 2024-09-13T17:38:48.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.690+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e0101460 con 0x7f59e00fff90 2024-09-13T17:38:48.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.691+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59dc011470 con 0x7f59e00fff90 2024-09-13T17:38:48.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.692+0000 7f59d6ffd640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f59cc0a1200 0x7f59cc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.692+0000 7f59e564c640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f59cc0a1200 0x7f59cc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.692+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f59dc0cd6f0 con 0x7f59e00fff90 2024-09-13T17:38:48.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.693+0000 7f59e564c640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f59cc0a1200 0x7f59cc0a36c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59e00742b0 tx=0x7f59d0005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:48.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.696+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f59dc096770 con 0x7f59e00fff90 2024-09-13T17:38:48.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.870+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59e0100370 con 0x7f59e00fff90 2024-09-13T17:38:48.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.872+0000 7f59d6ffd640 1 -- 172.21.15.44:0/1485040555 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25054 (secure 0 0 0) 0x7f59dc099d90 con 0x7f59e00fff90 2024-09-13T17:38:48.874 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:48.874 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:38:11.319123+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f59cc0a1200 msgr2=0x7f59cc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f59cc0a1200 0x7f59cc0a36c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59e00742b0 tx=0x7f59d0005f90 comp rx=0 tx=0).stop 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 msgr2=0x7f59e00727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 0x7f59e00727b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00eab0 tx=0x7f59dc00ef70 comp rx=0 tx=0).stop 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 shutdown_connections 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f59cc0a1200 0x7f59cc0a36c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59e01050e0 0x7f59e0072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0073230 0x7f59e01a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 --2- 172.21.15.44:0/1485040555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e00fff90 0x7f59e00727b0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 >> 172.21.15.44:0/1485040555 conn(0x7f59e00fba20 msgr2=0x7f59e00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:48.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 shutdown_connections 2024-09-13T17:38:48.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.874+0000 7f59e80b1640 1 -- 172.21.15.44:0/1485040555 wait complete. 2024-09-13T17:38:48.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:38:48.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:38:48.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 -- 172.21.15.44:0/3159702739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 msgr2=0x7f304010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 --2- 172.21.15.44:0/3159702739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010bb70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3038009cd0 tx=0x7f303802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 -- 172.21.15.44:0/3159702739 shutdown_connections 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 --2- 172.21.15.44:0/3159702739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010bb70 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 --2- 172.21.15.44:0/3159702739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400ff880 0x7f30401038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 --2- 172.21.15.44:0/3159702739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30400feed0 0x7f30400ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 -- 172.21.15.44:0/3159702739 >> 172.21.15.44:0/3159702739 conn(0x7f30400fa9a0 msgr2=0x7f30400fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 -- 172.21.15.44:0/3159702739 shutdown_connections 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 -- 172.21.15.44:0/3159702739 wait complete. 2024-09-13T17:38:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.994+0000 7f30451e2640 1 Processor -- start 2024-09-13T17:38:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 -- start start 2024-09-13T17:38:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400feed0 0x7f304010c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304010cf00 0x7f30401a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a49c0 con 0x7f3040104020 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a4b30 con 0x7f30400feed0 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f30451e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a4cf0 con 0x7f304010cf00 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39552/0 (socket says 172.21.15.44:39552) 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 -- 172.21.15.44:0/3980117690 learned_addr learned my addr 172.21.15.44:0/3980117690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303f59e640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304010cf00 0x7f30401a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303ed9d640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400feed0 0x7f304010c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 -- 172.21.15.44:0/3980117690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304010cf00 msgr2=0x7f30401a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304010cf00 0x7f30401a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 -- 172.21.15.44:0/3980117690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400feed0 msgr2=0x7f304010c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400feed0 0x7f304010c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 -- 172.21.15.44:0/3980117690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038009900 con 0x7f3040104020 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303ed9d640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400feed0 0x7f304010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.995+0000 7f303e59c640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010c9c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f303400c970 tx=0x7f303400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.996+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034004440 con 0x7f3040104020 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.996+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3034007cf0 con 0x7f3040104020 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.996+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30401a4fd0 con 0x7f3040104020 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.996+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303400f450 con 0x7f3040104020 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.996+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30401a5450 con 0x7f3040104020 2024-09-13T17:38:49.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.997+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30401003a0 con 0x7f3040104020 2024-09-13T17:38:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.998+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3034007860 con 0x7f3040104020 2024-09-13T17:38:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.999+0000 7f3027fff640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30280a1200 0x7f30280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.999+0000 7f303ed9d640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30280a1200 0x7f30280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.999+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f30340cd150 con 0x7f3040104020 2024-09-13T17:38:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:48.999+0000 7f303ed9d640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30280a1200 0x7f30280a36c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3030009c20 tx=0x7f3030005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.003+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3034096230 con 0x7f3040104020 2024-09-13T17:38:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.172+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3040107230 con 0x7f3040104020 2024-09-13T17:38:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.173+0000 7f3027fff640 1 -- 172.21.15.44:0/3980117690 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3034007e60 con 0x7f3040104020 2024-09-13T17:38:49.174 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.174+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30280a1200 msgr2=0x7f30280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.174+0000 7f30451e2640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30280a1200 0x7f30280a36c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3030009c20 tx=0x7f3030005f90 comp rx=0 tx=0).stop 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 msgr2=0x7f304010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010c9c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f303400c970 tx=0x7f303400ce30 comp rx=0 tx=0).stop 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 shutdown_connections 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f30280a1200 0x7f30280a36c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304010cf00 0x7f30401a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3040104020 0x7f304010c9c0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 --2- 172.21.15.44:0/3980117690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30400feed0 0x7f304010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 >> 172.21.15.44:0/3980117690 conn(0x7f30400fa9a0 msgr2=0x7f30400fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:49.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 shutdown_connections 2024-09-13T17:38:49.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.175+0000 7f30451e2640 1 -- 172.21.15.44:0/3980117690 wait complete. 2024-09-13T17:38:49.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:38:49.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.291+0000 7f0fcf85a640 1 -- 172.21.15.44:0/3132915402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc81050a0 msgr2=0x7f0fc810cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.291+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/3132915402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc81050a0 0x7f0fc810cbf0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009cd0 tx=0x7f0fc002a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 -- 172.21.15.44:0/3132915402 shutdown_connections 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/3132915402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc81050a0 0x7f0fc810cbf0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/3132915402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8100900 0x7f0fc8104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/3132915402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc80fff50 0x7f0fc8100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 -- 172.21.15.44:0/3132915402 >> 172.21.15.44:0/3132915402 conn(0x7f0fc80fb9e0 msgr2=0x7f0fc80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 -- 172.21.15.44:0/3132915402 shutdown_connections 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 -- 172.21.15.44:0/3132915402 wait complete. 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 Processor -- start 2024-09-13T17:38:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 -- start start 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc80fff50 0x7f0fc8196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.292+0000 7f0fcf85a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc81050a0 0x7f0fc8196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcf85a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 0x7f0fc819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcf85a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc819b360 con 0x7f0fc819a970 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcf85a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc819b4d0 con 0x7f0fc80fff50 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcf85a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc819b640 con 0x7f0fc81050a0 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 0x7f0fc819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 0x7f0fc819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39578/0 (socket says 172.21.15.44:39578) 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 -- 172.21.15.44:0/2538194298 learned_addr learned my addr 172.21.15.44:0/2538194298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fccdf5640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc81050a0 0x7f0fc8196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcd5f6640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc80fff50 0x7f0fc8196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 -- 172.21.15.44:0/2538194298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc81050a0 msgr2=0x7f0fc8196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc81050a0 0x7f0fc8196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 -- 172.21.15.44:0/2538194298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc80fff50 msgr2=0x7f0fc8196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc80fff50 0x7f0fc8196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 -- 172.21.15.44:0/2538194298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0009900 con 0x7f0fc819a970 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fccdf5640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc81050a0 0x7f0fc8196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcddf7640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 0x7f0fc819ae20 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0005cc0 tx=0x7f0fc0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0030f10 con 0x7f0fc819a970 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc81a2c20 con 0x7f0fc819a970 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fc0031570 con 0x7f0fc819a970 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0045670 con 0x7f0fc819a970 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.293+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc81a30d0 con 0x7f0fc819a970 2024-09-13T17:38:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.294+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc8101420 con 0x7f0fc819a970 2024-09-13T17:38:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.295+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0fc000a040 con 0x7f0fc819a970 2024-09-13T17:38:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.296+0000 7f0fbe7fc640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0fb40a1200 0x7f0fb40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.296+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f0fc00eece0 con 0x7f0fc819a970 2024-09-13T17:38:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.296+0000 7f0fcd5f6640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0fb40a1200 0x7f0fb40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.297+0000 7f0fcd5f6640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0fb40a1200 0x7f0fb40a36c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400ba40 tx=0x7f0fc4005cc0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.299+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0fc007fe20 con 0x7f0fc819a970 2024-09-13T17:38:49.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.470+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0fc81082b0 con 0x7f0fc819a970 2024-09-13T17:38:49.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.471+0000 7f0fbe7fc640 1 -- 172.21.15.44:0/2538194298 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0fc00b7d60 con 0x7f0fc819a970 2024-09-13T17:38:49.472 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.472+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0fb40a1200 msgr2=0x7f0fb40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.472+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0fb40a1200 0x7f0fb40a36c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400ba40 tx=0x7f0fc4005cc0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 msgr2=0x7f0fc819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 0x7f0fc819ae20 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0005cc0 tx=0x7f0fc0031a20 comp rx=0 tx=0).stop 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 shutdown_connections 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f0fb40a1200 0x7f0fb40a36c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fc81050a0 0x7f0fc8196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc819a970 0x7f0fc819ae20 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 --2- 172.21.15.44:0/2538194298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc80fff50 0x7f0fc8196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 >> 172.21.15.44:0/2538194298 conn(0x7f0fc80fb9e0 msgr2=0x7f0fc80fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:49.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 shutdown_connections 2024-09-13T17:38:49.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.473+0000 7f0fcf85a640 1 -- 172.21.15.44:0/2538194298 wait complete. 2024-09-13T17:38:49.485 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:38:49.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 -- 172.21.15.44:0/4096426719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c0ff7f0 msgr2=0x7ff92c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 --2- 172.21.15.44:0/4096426719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c0ff7f0 0x7ff92c103830 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009930 tx=0x7ff91c02a420 comp rx=0 tx=0).stop 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 -- 172.21.15.44:0/4096426719 shutdown_connections 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 --2- 172.21.15.44:0/4096426719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c103f90 0x7ff92c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 --2- 172.21.15.44:0/4096426719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c0ff7f0 0x7ff92c103830 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 --2- 172.21.15.44:0/4096426719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92c0fee40 0x7ff92c0ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 -- 172.21.15.44:0/4096426719 >> 172.21.15.44:0/4096426719 conn(0x7ff92c0fa890 msgr2=0x7ff92c0fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 -- 172.21.15.44:0/4096426719 shutdown_connections 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.590+0000 7ff933b65640 1 -- 172.21.15.44:0/4096426719 wait complete. 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 Processor -- start 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 -- start start 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92c0fee40 0x7ff92c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 0x7ff92c1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c19abb0 0x7ff92c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c19b5a0 con 0x7ff92c103f90 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c19b710 con 0x7ff92c19abb0 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff933b65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c19b880 con 0x7ff92c0fee40 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 0x7ff92c1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 0x7ff92c1967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60220/0 (socket says 172.21.15.44:60220) 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 -- 172.21.15.44:0/595909404 learned_addr learned my addr 172.21.15.44:0/595909404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff932102640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c19abb0 0x7ff92c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931901640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92c0fee40 0x7ff92c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 -- 172.21.15.44:0/595909404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92c0fee40 msgr2=0x7ff92c196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92c0fee40 0x7ff92c196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 -- 172.21.15.44:0/595909404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c19abb0 msgr2=0x7ff92c19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c19abb0 0x7ff92c19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.591+0000 7ff931100640 1 -- 172.21.15.44:0/595909404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c009560 con 0x7ff92c103f90 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.592+0000 7ff931100640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 0x7ff92c1967c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005ab0 tx=0x7ff91c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.592+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0044d0 con 0x7ff92c103f90 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.592+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c1a2e60 con 0x7ff92c103f90 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.592+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff91c032e10 con 0x7ff92c103f90 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.592+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c003850 con 0x7ff92c103f90 2024-09-13T17:38:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.592+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c1a3230 con 0x7ff92c103f90 2024-09-13T17:38:49.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.593+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c100310 con 0x7ff92c103f90 2024-09-13T17:38:49.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.594+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff91c004850 con 0x7ff92c103f90 2024-09-13T17:38:49.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.595+0000 7ff922ffd640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff9180a1200 0x7ff9180a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.595+0000 7ff931901640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff9180a1200 0x7ff9180a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.595+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff91c0edff0 con 0x7ff92c103f90 2024-09-13T17:38:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.595+0000 7ff931901640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff9180a1200 0x7ff9180a36c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff928009bd0 tx=0x7ff928005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.598+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff91c07fe20 con 0x7ff92c103f90 2024-09-13T17:38:49.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.772+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff92c1081b0 con 0x7ff92c103f90 2024-09-13T17:38:49.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.772+0000 7ff922ffd640 1 -- 172.21.15.44:0/595909404 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff91c0b7180 con 0x7ff92c103f90 2024-09-13T17:38:49.773 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:38:49.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff9180a1200 msgr2=0x7ff9180a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff9180a1200 0x7ff9180a36c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff928009bd0 tx=0x7ff928005f90 comp rx=0 tx=0).stop 2024-09-13T17:38:49.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 msgr2=0x7ff92c1967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 0x7ff92c1967c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005ab0 tx=0x7ff91c032c20 comp rx=0 tx=0).stop 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 shutdown_connections 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff9180a1200 0x7ff9180a36c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c19abb0 0x7ff92c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff92c103f90 0x7ff92c1967c0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 --2- 172.21.15.44:0/595909404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92c0fee40 0x7ff92c196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 >> 172.21.15.44:0/595909404 conn(0x7ff92c0fa890 msgr2=0x7ff92c068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 shutdown_connections 2024-09-13T17:38:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.774+0000 7ff933b65640 1 -- 172.21.15.44:0/595909404 wait complete. 2024-09-13T17:38:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:49 smithi044 ceph-mon[31175]: pgmap v378: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:49 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1485040555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:49 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3980117690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:49 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2538194298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:49.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 -- 172.21.15.44:0/3584158982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 msgr2=0x7f7524101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 --2- 172.21.15.44:0/3584158982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f7524101310 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7514009900 tx=0x7f751402a410 comp rx=0 tx=0).stop 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 -- 172.21.15.44:0/3584158982 shutdown_connections 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 --2- 172.21.15.44:0/3584158982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 0x7f752410c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 --2- 172.21.15.44:0/3584158982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7524101850 0x7f7524109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 --2- 172.21.15.44:0/3584158982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f7524101310 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 -- 172.21.15.44:0/3584158982 >> 172.21.15.44:0/3584158982 conn(0x7f75240fa8a0 msgr2=0x7f75240fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 -- 172.21.15.44:0/3584158982 shutdown_connections 2024-09-13T17:38:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.895+0000 7f752a553640 1 -- 172.21.15.44:0/3584158982 wait complete. 2024-09-13T17:38:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 Processor -- start 2024-09-13T17:38:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 -- start start 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f75241960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7524101850 0x7f7524196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 0x7f752419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752419b000 con 0x7f7524100f30 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752419b170 con 0x7f7524101850 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f752a553640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752419b330 con 0x7f752410a300 2024-09-13T17:38:49.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f7523fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f75241960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f7523fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f75241960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60238/0 (socket says 172.21.15.44:60238) 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.896+0000 7f7523fff640 1 -- 172.21.15.44:0/2398592113 learned_addr learned my addr 172.21.15.44:0/2398592113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f75237fe640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7524101850 0x7f7524196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7528af0640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 0x7f752419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7523fff640 1 -- 172.21.15.44:0/2398592113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 msgr2=0x7f752419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7523fff640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 0x7f752419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7523fff640 1 -- 172.21.15.44:0/2398592113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7524101850 msgr2=0x7f7524196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7523fff640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7524101850 0x7f7524196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7523fff640 1 -- 172.21.15.44:0/2398592113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514009560 con 0x7f7524100f30 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7528af0640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 0x7f752419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f7523fff640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f75241960d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7514005ab0 tx=0x7f7514032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514032e40 con 0x7f7524100f30 2024-09-13T17:38:49.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75140046c0 con 0x7f7524100f30 2024-09-13T17:38:49.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752419b5b0 con 0x7f7524100f30 2024-09-13T17:38:49.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514003850 con 0x7f7524100f30 2024-09-13T17:38:49.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.897+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752419b880 con 0x7f7524100f30 2024-09-13T17:38:49.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.898+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7524068df0 con 0x7f7524100f30 2024-09-13T17:38:49.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.899+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7514004af0 con 0x7f7524100f30 2024-09-13T17:38:49.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.900+0000 7f75217fa640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f750c0a1200 0x7f750c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:49.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.900+0000 7f75237fe640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f750c0a1200 0x7f750c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:49.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.901+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f75140ede50 con 0x7f7524100f30 2024-09-13T17:38:49.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.901+0000 7f75237fe640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f750c0a1200 0x7f750c0a36c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7524197670 tx=0x7f751800a2d0 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:49.905+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f75140b6ed0 con 0x7f7524100f30 2024-09-13T17:38:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:49 smithi070 ceph-mon[34502]: pgmap v378: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:49 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1485040555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:49 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3980117690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:49.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:49 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2538194298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:49 smithi100 ceph-mon[35055]: pgmap v378: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-13T17:38:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:49 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1485040555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:49 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3980117690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:50.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:49 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2538194298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:50.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.078+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7524197420 con 0x7f7524100f30 2024-09-13T17:38:50.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.079+0000 7f75217fa640 1 -- 172.21.15.44:0/2398592113 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7f75140ba4f0 con 0x7f7524100f30 2024-09-13T17:38:50.080 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:38:50.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f750c0a1200 msgr2=0x7f750c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f750c0a1200 0x7f750c0a36c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7524197670 tx=0x7f751800a2d0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 msgr2=0x7f75241960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f75241960d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7514005ab0 tx=0x7f7514032c50 comp rx=0 tx=0).stop 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 shutdown_connections 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f750c0a1200 0x7f750c0a36c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752410a300 0x7f752419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7524101850 0x7f7524196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 --2- 172.21.15.44:0/2398592113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7524100f30 0x7f75241960d0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 >> 172.21.15.44:0/2398592113 conn(0x7f75240fa8a0 msgr2=0x7f75240fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 shutdown_connections 2024-09-13T17:38:50.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.081+0000 7f752a553640 1 -- 172.21.15.44:0/2398592113 wait complete. 2024-09-13T17:38:50.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 -- 172.21.15.44:0/2908684063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c1018d0 msgr2=0x7fd17c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 --2- 172.21.15.44:0/2908684063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c1018d0 0x7fd17c109db0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd168009900 tx=0x7fd16802a480 comp rx=0 tx=0).stop 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 -- 172.21.15.44:0/2908684063 shutdown_connections 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 --2- 172.21.15.44:0/2908684063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c10a380 0x7fd17c10c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 --2- 172.21.15.44:0/2908684063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c1018d0 0x7fd17c109db0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 --2- 172.21.15.44:0/2908684063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd17c100fb0 0x7fd17c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 -- 172.21.15.44:0/2908684063 >> 172.21.15.44:0/2908684063 conn(0x7fd17c0fa940 msgr2=0x7fd17c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 -- 172.21.15.44:0/2908684063 shutdown_connections 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.196+0000 7fd182b9c640 1 -- 172.21.15.44:0/2908684063 wait complete. 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 Processor -- start 2024-09-13T17:38:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 -- start start 2024-09-13T17:38:50.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 0x7fd17c196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd17c1018d0 0x7fd17c196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 0x7fd17c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c19b160 con 0x7fd17c10a380 2024-09-13T17:38:50.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c19b2d0 con 0x7fd17c1018d0 2024-09-13T17:38:50.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd182b9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c19b490 con 0x7fd17c100fb0 2024-09-13T17:38:50.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd181139640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 0x7fd17c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd180938640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 0x7fd17c196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd181139640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 0x7fd17c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60258/0 (socket says 172.21.15.44:60258) 2024-09-13T17:38:50.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd181139640 1 -- 172.21.15.44:0/686888458 learned_addr learned my addr 172.21.15.44:0/686888458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:50.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd180938640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 0x7fd17c196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.44:60290/0 (socket says 172.21.15.44:60290) 2024-09-13T17:38:50.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd17bfff640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd17c1018d0 0x7fd17c196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.197+0000 7fd181139640 1 -- 172.21.15.44:0/686888458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 msgr2=0x7fd17c196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd181139640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 0x7fd17c196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd181139640 1 -- 172.21.15.44:0/686888458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd17c1018d0 msgr2=0x7fd17c196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd181139640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd17c1018d0 0x7fd17c196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd181139640 1 -- 172.21.15.44:0/686888458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009560 con 0x7fd17c10a380 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd180938640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 0x7fd17c196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd181139640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 0x7fd17c19ab60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fd17400eb10 tx=0x7fd17400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174004b20 con 0x7fd17c10a380 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd174004c80 con 0x7fd17c10a380 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c19b770 con 0x7fd17c10a380 2024-09-13T17:38:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174005a10 con 0x7fd17c10a380 2024-09-13T17:38:50.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.198+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c1a2ff0 con 0x7fd17c10a380 2024-09-13T17:38:50.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.199+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c068eb0 con 0x7fd17c10a380 2024-09-13T17:38:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.200+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd174005020 con 0x7fd17c10a380 2024-09-13T17:38:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.201+0000 7fd179ffb640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd16c0a1280 0x7fd16c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.201+0000 7fd180938640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd16c0a1280 0x7fd16c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.201+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fd174014070 con 0x7fd17c10a380 2024-09-13T17:38:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.201+0000 7fd180938640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd16c0a1280 0x7fd16c0a3740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd170005d00 tx=0x7fd17000b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.205+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd174096660 con 0x7fd17c10a380 2024-09-13T17:38:50.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.390+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd17c197580 con 0x7fd17c10a380 2024-09-13T17:38:50.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.390+0000 7fd179ffb640 1 -- 172.21.15.44:0/686888458 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) v1 ==== 81+0+942 (secure 0 0 0) 0x7fd174099c80 con 0x7fd17c10a380 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1409924258 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2038542363 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/771352345 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1586278527 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/4032970337 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1683772085 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/815607991 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:50.392 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/322597156 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2209604760 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/322597156 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2025177182 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:50.393 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3278871357 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:listed 16 entries 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.392+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd16c0a1280 msgr2=0x7fd16c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.392+0000 7fd182b9c640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd16c0a1280 0x7fd16c0a3740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd170005d00 tx=0x7fd17000b410 comp rx=0 tx=0).stop 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.392+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 msgr2=0x7fd17c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.392+0000 7fd182b9c640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 0x7fd17c19ab60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fd17400eb10 tx=0x7fd17400c2d0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.392+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 shutdown_connections 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fd16c0a1280 0x7fd16c0a3740 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c10a380 0x7fd17c19ab60 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd17c1018d0 0x7fd17c196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 --2- 172.21.15.44:0/686888458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c100fb0 0x7fd17c196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 >> 172.21.15.44:0/686888458 conn(0x7fd17c0fa940 msgr2=0x7fd17c0fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:50.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 shutdown_connections 2024-09-13T17:38:50.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.393+0000 7fd182b9c640 1 -- 172.21.15.44:0/686888458 wait complete. 2024-09-13T17:38:50.405 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T17:38:50.405 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:38:50.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:38:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 -- 172.21.15.44:0/335755689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4103fd0 msgr2=0x7fe6c410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 --2- 172.21.15.44:0/335755689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4103fd0 0x7fe6c410bb20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009cd0 tx=0x7fe6bc02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 -- 172.21.15.44:0/335755689 shutdown_connections 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 --2- 172.21.15.44:0/335755689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4103fd0 0x7fe6c410bb20 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 --2- 172.21.15.44:0/335755689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40ff830 0x7fe6c4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 --2- 172.21.15.44:0/335755689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c40fee80 0x7fe6c40ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 -- 172.21.15.44:0/335755689 >> 172.21.15.44:0/335755689 conn(0x7fe6c4075c60 msgr2=0x7fe6c40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 -- 172.21.15.44:0/335755689 shutdown_connections 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.563+0000 7fe6c9757640 1 -- 172.21.15.44:0/335755689 wait complete. 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 Processor -- start 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 -- start start 2024-09-13T17:38:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40fee80 0x7fe6c40fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c4103fd0 0x7fe6c40fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 0x7fe6c41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a49c0 con 0x7fe6c40fbb20 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a4b30 con 0x7fe6c40fee80 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c9757640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a4cf0 con 0x7fe6c4103fd0 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 0x7fe6c41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 0x7fe6c41a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60288/0 (socket says 172.21.15.44:60288) 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.564+0000 7fe6c37fe640 1 -- 172.21.15.44:0/2216264206 learned_addr learned my addr 172.21.15.44:0/2216264206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c2ffd640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40fee80 0x7fe6c40fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c27fc640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c4103fd0 0x7fe6c40fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c37fe640 1 -- 172.21.15.44:0/2216264206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c4103fd0 msgr2=0x7fe6c40fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c37fe640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c4103fd0 0x7fe6c40fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c37fe640 1 -- 172.21.15.44:0/2216264206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40fee80 msgr2=0x7fe6c40fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c37fe640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40fee80 0x7fe6c40fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c37fe640 1 -- 172.21.15.44:0/2216264206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc009900 con 0x7fe6c40fbb20 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c2ffd640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40fee80 0x7fe6c40fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c27fc640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c4103fd0 0x7fe6c40fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c37fe640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 0x7fe6c41a4370 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc004af0 tx=0x7fe6bc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003970 con 0x7fe6c40fbb20 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6bc004040 con 0x7fe6c40fbb20 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c41a4f70 con 0x7fe6c40fbb20 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc045650 con 0x7fe6c40fbb20 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.565+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c41a5450 con 0x7fe6c40fbb20 2024-09-13T17:38:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.566+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c4100350 con 0x7fe6c40fbb20 2024-09-13T17:38:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.567+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe6bc00a040 con 0x7fe6c40fbb20 2024-09-13T17:38:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.568+0000 7fe6abfff640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6ac0a1200 0x7fe6ac0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.568+0000 7fe6c2ffd640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6ac0a1200 0x7fe6ac0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.568+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fe6bc0ee980 con 0x7fe6c40fbb20 2024-09-13T17:38:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.568+0000 7fe6c2ffd640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6ac0a1200 0x7fe6ac0a36c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005d00 tx=0x7fe6b4009550 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.572+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe6bc0b7a00 con 0x7fe6c40fbb20 2024-09-13T17:38:50.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.748+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6c40ff830 con 0x7fe6c40fbb20 2024-09-13T17:38:50.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.750+0000 7fe6abfff640 1 -- 172.21.15.44:0/2216264206 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25054 (secure 0 0 0) 0x7fe6bc031c00 con 0x7fe6c40fbb20 2024-09-13T17:38:50.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:50.752 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:38:11.319123+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:38:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6ac0a1200 msgr2=0x7fe6ac0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6ac0a1200 0x7fe6ac0a36c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005d00 tx=0x7fe6b4009550 comp rx=0 tx=0).stop 2024-09-13T17:38:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 msgr2=0x7fe6c41a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 0x7fe6c41a4370 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc004af0 tx=0x7fe6bc030ec0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 shutdown_connections 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fe6ac0a1200 0x7fe6ac0a36c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c4103fd0 0x7fe6c40fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c40fbb20 0x7fe6c41a4370 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 --2- 172.21.15.44:0/2216264206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c40fee80 0x7fe6c40fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 >> 172.21.15.44:0/2216264206 conn(0x7fe6c4075c60 msgr2=0x7fe6c40677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 shutdown_connections 2024-09-13T17:38:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.752+0000 7fe6c9757640 1 -- 172.21.15.44:0/2216264206 wait complete. 2024-09-13T17:38:50.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:20:01.986440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:24:19.855133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:24:21.127863+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:24:24.170156+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:38:50.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- 172.21.15.44:0/3952456558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 msgr2=0x7ff7c4100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/3952456558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c4100330 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009cd0 tx=0x7ff7b402a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- 172.21.15.44:0/3952456558 shutdown_connections 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/3952456558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c4100900 0x7ff7c4104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/3952456558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7c41050a0 0x7ff7c410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/3952456558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c4100330 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- 172.21.15.44:0/3952456558 >> 172.21.15.44:0/3952456558 conn(0x7ff7c40fb9e0 msgr2=0x7ff7c40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- 172.21.15.44:0/3952456558 shutdown_connections 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- 172.21.15.44:0/3952456558 wait complete. 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 Processor -- start 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- start start 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c41961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c41050a0 0x7ff7c4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7c419aaf0 0x7ff7c419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419b4e0 con 0x7ff7c40fff50 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419b650 con 0x7ff7c419aaf0 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.870+0000 7ff7cb05a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419b7c0 con 0x7ff7c41050a0 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c41961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c41961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60312/0 (socket says 172.21.15.44:60312) 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 -- 172.21.15.44:0/4036790123 learned_addr learned my addr 172.21.15.44:0/4036790123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c3fff640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c41050a0 0x7ff7c4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c95f7640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7c419aaf0 0x7ff7c419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 -- 172.21.15.44:0/4036790123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c41050a0 msgr2=0x7ff7c4196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c41050a0 0x7ff7c4196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 -- 172.21.15.44:0/4036790123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7c419aaf0 msgr2=0x7ff7c419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7c419aaf0 0x7ff7c419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 -- 172.21.15.44:0/4036790123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4009900 con 0x7ff7c40fff50 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c8df6640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c41961c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4004af0 tx=0x7ff7b4030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4031b80 con 0x7ff7c40fff50 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7b4003e00 con 0x7ff7c40fff50 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c41a2da0 con 0x7ff7c40fff50 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.871+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4045410 con 0x7ff7c40fff50 2024-09-13T17:38:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.872+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c41a31a0 con 0x7ff7c40fff50 2024-09-13T17:38:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.873+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4101420 con 0x7ff7c40fff50 2024-09-13T17:38:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.874+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff7b400a040 con 0x7ff7c40fff50 2024-09-13T17:38:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.875+0000 7ff7c1ffb640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7ac0a1280 0x7ff7ac0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.875+0000 7ff7c3fff640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7ac0a1280 0x7ff7ac0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.875+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7ff7b40ee6a0 con 0x7ff7c40fff50 2024-09-13T17:38:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.875+0000 7ff7c3fff640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7ac0a1280 0x7ff7ac0a3740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4197760 tx=0x7ff7b800b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:50.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:50.879+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff7b40b7780 con 0x7ff7c40fff50 2024-09-13T17:38:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/595909404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2398592113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:50.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:50 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/686888458' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:38:51.011 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/595909404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.011 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2398592113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.011 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:50 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/686888458' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:38:51.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.049+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7c4100330 con 0x7ff7c40fff50 2024-09-13T17:38:51.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.049+0000 7ff7c1ffb640 1 -- 172.21.15.44:0/4036790123 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7b40bada0 con 0x7ff7c40fff50 2024-09-13T17:38:51.051 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T17:38:51.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7ac0a1280 msgr2=0x7ff7ac0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7ac0a1280 0x7ff7ac0a3740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4197760 tx=0x7ff7b800b390 comp rx=0 tx=0).stop 2024-09-13T17:38:51.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 msgr2=0x7ff7c41961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c41961c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4004af0 tx=0x7ff7b4030e50 comp rx=0 tx=0).stop 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 shutdown_connections 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7ff7ac0a1280 0x7ff7ac0a3740 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c41050a0 0x7ff7c4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7c419aaf0 0x7ff7c419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 --2- 172.21.15.44:0/4036790123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c40fff50 0x7ff7c41961c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 >> 172.21.15.44:0/4036790123 conn(0x7ff7c40fb9e0 msgr2=0x7ff7c40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 shutdown_connections 2024-09-13T17:38:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.051+0000 7ff7cb05a640 1 -- 172.21.15.44:0/4036790123 wait complete. 2024-09-13T17:38:51.063 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/595909404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.064 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2398592113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.064 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:50 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/686888458' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:38:51.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:38:51.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1648257279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d0103fb0 msgr2=0x7f27d010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1648257279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d0103fb0 0x7f27d010bb00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009cd0 tx=0x7f27c802a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1648257279 shutdown_connections 2024-09-13T17:38:51.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1648257279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d0103fb0 0x7f27d010bb00 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1648257279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00ff810 0x7f27d0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1648257279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d00fee60 0x7f27d00ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1648257279 >> 172.21.15.44:0/1648257279 conn(0x7f27d0075c60 msgr2=0x7f27d00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1648257279 shutdown_connections 2024-09-13T17:38:51.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1648257279 wait complete. 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.168+0000 7f27d4c5a640 1 Processor -- start 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 -- start start 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 0x7f27d00fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d0103fb0 0x7f27d00fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00fbb20 0x7f27d01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01a4940 con 0x7f27d00fee60 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01a4ab0 con 0x7f27d00fbb20 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27d4c5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01a4c70 con 0x7f27d0103fb0 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 0x7f27d00fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 0x7f27d00fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60320/0 (socket says 172.21.15.44:60320) 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 -- 172.21.15.44:0/1676388580 learned_addr learned my addr 172.21.15.44:0/1676388580 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ced9d640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00fbb20 0x7f27d01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27cdd9b640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d0103fb0 0x7f27d00fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 -- 172.21.15.44:0/1676388580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d0103fb0 msgr2=0x7f27d00fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d0103fb0 0x7f27d00fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 -- 172.21.15.44:0/1676388580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00fbb20 msgr2=0x7f27d01a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00fbb20 0x7f27d01a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 -- 172.21.15.44:0/1676388580 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8009900 con 0x7f27d00fee60 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.169+0000 7f27ce59c640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 0x7f27d00fb0a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c970 tx=0x7f27c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.170+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0007bb0 con 0x7f27d00fee60 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.170+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c0007d10 con 0x7f27d00fee60 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.170+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d01a4f50 con 0x7f27d00fee60 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.170+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004d70 con 0x7f27d00fee60 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.170+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d01a5380 con 0x7f27d00fee60 2024-09-13T17:38:51.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.171+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d0100330 con 0x7f27d00fee60 2024-09-13T17:38:51.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.172+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f27c0005000 con 0x7f27d00fee60 2024-09-13T17:38:51.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.172+0000 7f27bf7fe640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f27b80a1200 0x7f27b80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.173+0000 7f27cdd9b640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f27b80a1200 0x7f27b80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.173+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f27c00cd1d0 con 0x7f27d00fee60 2024-09-13T17:38:51.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.173+0000 7f27cdd9b640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f27b80a1200 0x7f27b80a36c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005e00 tx=0x7f27c4005c50 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:51.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.176+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f27c0096250 con 0x7f27d00fee60 2024-09-13T17:38:51.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.346+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27d01081d0 con 0x7f27d00fee60 2024-09-13T17:38:51.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.346+0000 7f27bf7fe640 1 -- 172.21.15.44:0/1676388580 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27c001d0a0 con 0x7f27d00fee60 2024-09-13T17:38:51.349 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f27b80a1200 msgr2=0x7f27b80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f27b80a1200 0x7f27b80a36c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005e00 tx=0x7f27c4005c50 comp rx=0 tx=0).stop 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 msgr2=0x7f27d00fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 0x7f27d00fb0a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c970 tx=0x7f27c000ce30 comp rx=0 tx=0).stop 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 shutdown_connections 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f27b80a1200 0x7f27b80a36c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d0103fb0 0x7f27d00fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00fbb20 0x7f27d01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 --2- 172.21.15.44:0/1676388580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d00fee60 0x7f27d00fb0a0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 >> 172.21.15.44:0/1676388580 conn(0x7f27d0075c60 msgr2=0x7f27d0067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 shutdown_connections 2024-09-13T17:38:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.348+0000 7f27d4c5a640 1 -- 172.21.15.44:0/1676388580 wait complete. 2024-09-13T17:38:51.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:38:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 -- 172.21.15.44:0/2461109517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97180ffbe0 msgr2=0x7f97180fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 --2- 172.21.15.44:0/2461109517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97180ffbe0 0x7f97180fffc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9700009930 tx=0x7f970002a420 comp rx=0 tx=0).stop 2024-09-13T17:38:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 -- 172.21.15.44:0/2461109517 shutdown_connections 2024-09-13T17:38:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 --2- 172.21.15.44:0/2461109517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9718105190 0x7f971810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 --2- 172.21.15.44:0/2461109517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718100590 0x7f9718104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 --2- 172.21.15.44:0/2461109517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97180ffbe0 0x7f97180fffc0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 -- 172.21.15.44:0/2461109517 >> 172.21.15.44:0/2461109517 conn(0x7f97180741b0 msgr2=0x7f97180745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 -- 172.21.15.44:0/2461109517 shutdown_connections 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 -- 172.21.15.44:0/2461109517 wait complete. 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 Processor -- start 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.465+0000 7f971f048640 1 -- start start 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97180ffbe0 0x7f9718197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718105190 0x7f9718197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 0x7f97181a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181a2770 con 0x7f9718197ad0 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181a28e0 con 0x7f9718105190 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181a2aa0 con 0x7f97180ffbe0 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 0x7f97181a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 0x7f97181a20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60342/0 (socket says 172.21.15.44:60342) 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 -- 172.21.15.44:0/2897770539 learned_addr learned my addr 172.21.15.44:0/2897770539 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971cde4640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97180ffbe0 0x7f9718197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f9717fff640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718105190 0x7f9718197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 -- 172.21.15.44:0/2897770539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97180ffbe0 msgr2=0x7f9718197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97180ffbe0 0x7f9718197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 -- 172.21.15.44:0/2897770539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718105190 msgr2=0x7f9718197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718105190 0x7f9718197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 -- 172.21.15.44:0/2897770539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700009560 con 0x7f9718197ad0 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f9717fff640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718105190 0x7f9718197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971d5e5640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 0x7f97181a20d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f971000eaa0 tx=0x7f971000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971000cdb0 con 0x7f9718197ad0 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9710005020 con 0x7f9718197ad0 2024-09-13T17:38:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.466+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97181a2d80 con 0x7f9718197ad0 2024-09-13T17:38:51.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.467+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97100116d0 con 0x7f9718197ad0 2024-09-13T17:38:51.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.467+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97181a3290 con 0x7f9718197ad0 2024-09-13T17:38:51.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.468+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9718067fc0 con 0x7f9718197ad0 2024-09-13T17:38:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.469+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9710005190 con 0x7f9718197ad0 2024-09-13T17:38:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.469+0000 7f9715ffb640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97040a1200 0x7f97040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.470+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f97100cd1b0 con 0x7f9718197ad0 2024-09-13T17:38:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.470+0000 7f971cde4640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97040a1200 0x7f97040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.470+0000 7f971cde4640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97040a1200 0x7f97040a36c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97000057c0 tx=0x7f9700005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:51.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.474+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9710096290 con 0x7f9718197ad0 2024-09-13T17:38:51.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.645+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97180fffc0 con 0x7f9718197ad0 2024-09-13T17:38:51.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.645+0000 7f9715ffb640 1 -- 172.21.15.44:0/2897770539 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f971001f0a0 con 0x7f9718197ad0 2024-09-13T17:38:51.648 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T17:38:51.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97040a1200 msgr2=0x7f97040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97040a1200 0x7f97040a36c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97000057c0 tx=0x7f9700005a20 comp rx=0 tx=0).stop 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 msgr2=0x7f97181a20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 0x7f97181a20d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f971000eaa0 tx=0x7f971000ef60 comp rx=0 tx=0).stop 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 shutdown_connections 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f97040a1200 0x7f97040a36c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9718197ad0 0x7f97181a20d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9718105190 0x7f9718197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 --2- 172.21.15.44:0/2897770539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97180ffbe0 0x7f9718197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 >> 172.21.15.44:0/2897770539 conn(0x7f97180741b0 msgr2=0x7f9718102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 shutdown_connections 2024-09-13T17:38:51.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.647+0000 7f971f048640 1 -- 172.21.15.44:0/2897770539 wait complete. 2024-09-13T17:38:51.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:38:51.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/78627592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac0ff800 msgr2=0x7f5cac103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/78627592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac0ff800 0x7f5cac103840 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009930 tx=0x7f5c9c02a420 comp rx=0 tx=0).stop 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/78627592 shutdown_connections 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/78627592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cac103fa0 0x7f5cac10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/78627592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac0ff800 0x7f5cac103840 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/78627592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac0fee50 0x7f5cac0ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/78627592 >> 172.21.15.44:0/78627592 conn(0x7f5cac0fa8c0 msgr2=0x7f5cac0fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/78627592 shutdown_connections 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.764+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/78627592 wait complete. 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 Processor -- start 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 -- start start 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cac0fee50 0x7f5cac196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac103fa0 0x7f5cac196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 0x7f5cac19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac19b550 con 0x7f5cac19ab60 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac19b6c0 con 0x7f5cac103fa0 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb2cb1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac19b830 con 0x7f5cac0fee50 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 0x7f5cac19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 0x7f5cac19b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60354/0 (socket says 172.21.15.44:60354) 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 -- 172.21.15.44:0/103588378 learned_addr learned my addr 172.21.15.44:0/103588378 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb0a4d640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cac0fee50 0x7f5cac196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cabfff640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac103fa0 0x7f5cac196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 -- 172.21.15.44:0/103588378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cac0fee50 msgr2=0x7f5cac196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cac0fee50 0x7f5cac196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 -- 172.21.15.44:0/103588378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac103fa0 msgr2=0x7f5cac196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac103fa0 0x7f5cac196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.765+0000 7f5cb124e640 1 -- 172.21.15.44:0/103588378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009560 con 0x7f5cac19ab60 2024-09-13T17:38:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.766+0000 7f5cb124e640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 0x7f5cac19b010 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400eaa0 tx=0x7f5ca400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:51.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.766+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400cdb0 con 0x7f5cac19ab60 2024-09-13T17:38:51.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.766+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ca4005020 con 0x7f5cac19ab60 2024-09-13T17:38:51.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.766+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac1a2e10 con 0x7f5cac19ab60 2024-09-13T17:38:51.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.766+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40116d0 con 0x7f5cac19ab60 2024-09-13T17:38:51.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.766+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac1a3290 con 0x7f5cac19ab60 2024-09-13T17:38:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.767+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac100320 con 0x7f5cac19ab60 2024-09-13T17:38:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.768+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5ca4005190 con 0x7f5cac19ab60 2024-09-13T17:38:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.769+0000 7f5ca9ffb640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c980a1200 0x7f5c980a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.769+0000 7f5cb0a4d640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c980a1200 0x7f5c980a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.769+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f5ca40cd160 con 0x7f5cac19ab60 2024-09-13T17:38:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.769+0000 7f5cb0a4d640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c980a1200 0x7f5c980a36c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005d00 tx=0x7f5c9400b410 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:51.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.773+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5ca4096290 con 0x7f5cac19ab60 2024-09-13T17:38:51.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.944+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5cac108160 con 0x7f5cac19ab60 2024-09-13T17:38:51.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.944+0000 7f5ca9ffb640 1 -- 172.21.15.44:0/103588378 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5ca401f0a0 con 0x7f5cac19ab60 2024-09-13T17:38:51.947 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T17:38:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:51 smithi070 ceph-mon[34502]: pgmap v379: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2216264206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4036790123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1676388580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:51 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/2897770539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.946+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c980a1200 msgr2=0x7f5c980a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.946+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c980a1200 0x7f5c980a36c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005d00 tx=0x7f5c9400b410 comp rx=0 tx=0).stop 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 msgr2=0x7f5cac19b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 0x7f5cac19b010 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400eaa0 tx=0x7f5ca400ef60 comp rx=0 tx=0).stop 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 shutdown_connections 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f5c980a1200 0x7f5c980a36c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cac19ab60 0x7f5cac19b010 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac103fa0 0x7f5cac196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 --2- 172.21.15.44:0/103588378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cac0fee50 0x7f5cac196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:51.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 >> 172.21.15.44:0/103588378 conn(0x7f5cac0fa8c0 msgr2=0x7f5cac068ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:51.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 shutdown_connections 2024-09-13T17:38:51.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:51.947+0000 7f5cb2cb1640 1 -- 172.21.15.44:0/103588378 wait complete. 2024-09-13T17:38:51.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:51 smithi044 ceph-mon[31175]: pgmap v379: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:51.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2216264206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:51.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4036790123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1676388580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:51 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/2897770539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:51.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:38:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:51 smithi100 ceph-mon[35055]: pgmap v379: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2216264206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:38:52.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4036790123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T17:38:52.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1676388580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T17:38:52.025 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:51 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/2897770539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 -- 172.21.15.44:0/3306065518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80fe880 msgr2=0x7f90d80fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/3306065518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80fe880 0x7f90d80fece0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00aa40 tx=0x7f90cc02b750 comp rx=0 tx=0).stop 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 -- 172.21.15.44:0/3306065518 shutdown_connections 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/3306065518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80ff220 0x7f90d810b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/3306065518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80fe880 0x7f90d80fece0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/3306065518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d8104470 0x7f90d8104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 -- 172.21.15.44:0/3306065518 >> 172.21.15.44:0/3306065518 conn(0x7f90d80741e0 msgr2=0x7f90d80745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 -- 172.21.15.44:0/3306065518 shutdown_connections 2024-09-13T17:38:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.061+0000 7f90dd7d5640 1 -- 172.21.15.44:0/3306065518 wait complete. 2024-09-13T17:38:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 Processor -- start 2024-09-13T17:38:52.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 -- start start 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80fe880 0x7f90d8196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 0x7f90d8196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d8104470 0x7f90d819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819b140 con 0x7f90d80ff220 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819b2b0 con 0x7f90d80fe880 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.062+0000 7f90dd7d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819b470 con 0x7f90d8104470 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 0x7f90d8196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 0x7f90d8196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60380/0 (socket says 172.21.15.44:60380) 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 -- 172.21.15.44:0/1368037641 learned_addr learned my addr 172.21.15.44:0/1368037641 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d6ffd640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80fe880 0x7f90d8196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d77fe640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d8104470 0x7f90d819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 -- 172.21.15.44:0/1368037641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d8104470 msgr2=0x7f90d819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d8104470 0x7f90d819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 -- 172.21.15.44:0/1368037641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80fe880 msgr2=0x7f90d8196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80fe880 0x7f90d8196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 -- 172.21.15.44:0/1368037641 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc00a670 con 0x7f90d80ff220 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d6ffd640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80fe880 0x7f90d8196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90d67fc640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 0x7f90d8196750 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005ab0 tx=0x7f90cc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc005200 con 0x7f90d80ff220 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d819b6f0 con 0x7f90d80ff220 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90cc0044c0 con 0x7f90d80ff220 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc003890 con 0x7f90d80ff220 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.063+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d81a2fd0 con 0x7f90d80ff220 2024-09-13T17:38:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.064+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d80ffaf0 con 0x7f90d80ff220 2024-09-13T17:38:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.065+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f90cc005360 con 0x7f90d80ff220 2024-09-13T17:38:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.066+0000 7f90bffff640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f90c00a1200 0x7f90c00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.066+0000 7f90d6ffd640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f90c00a1200 0x7f90c00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.066+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f90cc0ee120 con 0x7f90d80ff220 2024-09-13T17:38:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.066+0000 7f90d6ffd640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f90c00a1200 0x7f90c00a36c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009f40 tx=0x7f90c8005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:52.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.070+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f90cc0b71a0 con 0x7f90d80ff220 2024-09-13T17:38:52.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.253+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90d80fece0 con 0x7f90d80ff220 2024-09-13T17:38:52.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.253+0000 7f90bffff640 1 -- 172.21.15.44:0/1368037641 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) v1 ==== 81+0+942 (secure 0 0 0) 0x7f90cc0403c0 con 0x7f90d80ff220 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1409924258 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2038542363 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/771352345 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1586278527 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/4032970337 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1683772085 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/815607991 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/322597156 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/2265862199 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:52.256 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/3711196986 2024-09-14T17:17:17.599897+0000 2024-09-13T17:38:52.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2209604760 2024-09-14T17:17:41.472796+0000 2024-09-13T17:38:52.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/322597156 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:52.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2025177182 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:52.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3278871357 2024-09-14T17:26:42.344842+0000 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:listed 16 entries 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f90c00a1200 msgr2=0x7f90c00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f90c00a1200 0x7f90c00a36c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009f40 tx=0x7f90c8005f90 comp rx=0 tx=0).stop 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 msgr2=0x7f90d8196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 0x7f90d8196750 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005ab0 tx=0x7f90cc0040c0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 shutdown_connections 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f90c00a1200 0x7f90c00a36c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d8104470 0x7f90d819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80ff220 0x7f90d8196750 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 --2- 172.21.15.44:0/1368037641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90d80fe880 0x7f90d8196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 >> 172.21.15.44:0/1368037641 conn(0x7f90d80741e0 msgr2=0x7f90d80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 shutdown_connections 2024-09-13T17:38:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:52.255+0000 7f90dd7d5640 1 -- 172.21.15.44:0/1368037641 wait complete. 2024-09-13T17:38:52.268 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2024-09-13T17:38:52.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:38:52.269 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:38:52.293 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[84318]: fuse finished with error 0 and tester_r 0 2024-09-13T17:38:52.295 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:38:52.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:52 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/103588378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:52.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:52 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1368037641' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:38:53.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:52 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/103588378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:53.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:52 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1368037641' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:38:53.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:52 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/103588378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T17:38:53.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:52 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1368037641' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T17:38:53.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:53 smithi070 ceph-mon[34502]: pgmap v380: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:54.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:53 smithi100 ceph-mon[35055]: pgmap v380: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:54.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:53 smithi044 ceph-mon[31175]: pgmap v380: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-13T17:38:55.947 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:55 smithi070 ceph-mon[34502]: pgmap v381: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-13T17:38:56.024 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:55 smithi100 ceph-mon[35055]: pgmap v381: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-13T17:38:56.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:55 smithi044 ceph-mon[31175]: pgmap v381: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-13T17:38:58.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:38:58.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:38:58.397 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:38:58.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:38:58.420 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T17:38:58.479 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:38:58.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:38:58.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:38:58.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:38:58.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-09-13T17:38:58.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:58 smithi044 ceph-mon[31175]: pgmap v382: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-13T17:38:58.649 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-13T17:38:58.660 INFO:tasks.cephadm:Teardown begin 2024-09-13T17:38:58.661 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:38:58.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:38:58.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:58 smithi070 ceph-mon[34502]: pgmap v382: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-13T17:38:58.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:58 smithi100 ceph-mon[35055]: pgmap v382: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-13T17:38:58.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 -- 172.21.15.44:0/3289531220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc100f70 msgr2=0x7f9afc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:58.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/3289531220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc100f70 0x7f9afc101350 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009ca0 tx=0x7f9ae402a950 comp rx=0 tx=0).stop 2024-09-13T17:38:58.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 -- 172.21.15.44:0/3289531220 shutdown_connections 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/3289531220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 0x7f9afc109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/3289531220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc10a340 0x7f9afc10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/3289531220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc100f70 0x7f9afc101350 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 -- 172.21.15.44:0/3289531220 >> 172.21.15.44:0/3289531220 conn(0x7f9afc0fa900 msgr2=0x7f9afc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 -- 172.21.15.44:0/3289531220 shutdown_connections 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.794+0000 7f9b02dbc640 1 -- 172.21.15.44:0/3289531220 wait complete. 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 Processor -- start 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 -- start start 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc100f70 0x7f9afc1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 0x7f9afc196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 0x7f9afc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc19b120 con 0x7f9afc10a340 2024-09-13T17:38:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc19b290 con 0x7f9afc100f70 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b02dbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc19b450 con 0x7f9afc101890 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 0x7f9afc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 0x7f9afc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60408/0 (socket says 172.21.15.44:60408) 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b00b58640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc100f70 0x7f9afc1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 -- 172.21.15.44:0/1851519495 learned_addr learned my addr 172.21.15.44:0/1851519495 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9afbfff640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 0x7f9afc196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 -- 172.21.15.44:0/1851519495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 msgr2=0x7f9afc196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 0x7f9afc196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 -- 172.21.15.44:0/1851519495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc100f70 msgr2=0x7f9afc1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc100f70 0x7f9afc1961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9b01359640 1 -- 172.21.15.44:0/1851519495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4009900 con 0x7f9afc10a340 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.795+0000 7f9afbfff640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 0x7f9afc196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.796+0000 7f9b01359640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 0x7f9afc19ab20 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c970 tx=0x7f9af400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.796+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4007bb0 con 0x7f9afc10a340 2024-09-13T17:38:58.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.796+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc19b730 con 0x7f9afc10a340 2024-09-13T17:38:58.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.796+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9af4007d10 con 0x7f9afc10a340 2024-09-13T17:38:58.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.796+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4018620 con 0x7f9afc10a340 2024-09-13T17:38:58.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.796+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc1a2fb0 con 0x7f9afc10a340 2024-09-13T17:38:58.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.797+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc068e70 con 0x7f9afc10a340 2024-09-13T17:38:58.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.798+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9af400f450 con 0x7f9afc10a340 2024-09-13T17:38:58.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.799+0000 7f9af9ffb640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ae80a1200 0x7f9ae80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:58.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.799+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f9af40cd0b0 con 0x7f9afc10a340 2024-09-13T17:38:58.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.799+0000 7f9b00b58640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ae80a1200 0x7f9ae80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:58.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.799+0000 7f9b00b58640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ae80a1200 0x7f9ae80a36c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ae402af40 tx=0x7f9ae403d040 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:58.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:58.803+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9af4096190 con 0x7f9afc10a340 2024-09-13T17:38:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.010+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9afc197540 con 0x7f9afc10a340 2024-09-13T17:38:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.011+0000 7f9af9ffb640 1 -- 172.21.15.44:0/1851519495 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f9af40997b0 con 0x7f9afc10a340 2024-09-13T17:38:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:59.014 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:38:59.014 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:38:59.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ae80a1200 msgr2=0x7f9ae80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ae80a1200 0x7f9ae80a36c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ae402af40 tx=0x7f9ae403d040 comp rx=0 tx=0).stop 2024-09-13T17:38:59.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 msgr2=0x7f9afc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 0x7f9afc19ab20 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c970 tx=0x7f9af400ce30 comp rx=0 tx=0).stop 2024-09-13T17:38:59.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 shutdown_connections 2024-09-13T17:38:59.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f9ae80a1200 0x7f9ae80a36c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc101890 0x7f9afc196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9afc10a340 0x7f9afc19ab20 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 --2- 172.21.15.44:0/1851519495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc100f70 0x7f9afc1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 >> 172.21.15.44:0/1851519495 conn(0x7f9afc0fa900 msgr2=0x7f9afc0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 shutdown_connections 2024-09-13T17:38:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.013+0000 7f9b02dbc640 1 -- 172.21.15.44:0/1851519495 wait complete. 2024-09-13T17:38:59.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:38:59.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 -- 172.21.15.44:0/374779416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ff830 msgr2=0x7f8100103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 --2- 172.21.15.44:0/374779416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ff830 0x7f8100103870 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009930 tx=0x7f80f402a420 comp rx=0 tx=0).stop 2024-09-13T17:38:59.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 -- 172.21.15.44:0/374779416 shutdown_connections 2024-09-13T17:38:59.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 --2- 172.21.15.44:0/374779416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 0x7f810010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 --2- 172.21.15.44:0/374779416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ff830 0x7f8100103870 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 --2- 172.21.15.44:0/374779416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81000fee80 0x7f81000ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 -- 172.21.15.44:0/374779416 >> 172.21.15.44:0/374779416 conn(0x7f8100075c60 msgr2=0x7f81000fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 -- 172.21.15.44:0/374779416 shutdown_connections 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 -- 172.21.15.44:0/374779416 wait complete. 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.128+0000 7f8105757640 1 Processor -- start 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 -- start start 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 0x7f8100196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 0x7f81001967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810019abc0 0x7f810019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810019b5b0 con 0x7f81000fee80 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810019b720 con 0x7f8100103fd0 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810019b890 con 0x7f810019abc0 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 0x7f8100196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 0x7f8100196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60418/0 (socket says 172.21.15.44:60418) 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 -- 172.21.15.44:0/1338185270 learned_addr learned my addr 172.21.15.44:0/1338185270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80fe7fc640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 0x7f81001967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80ff7fe640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810019abc0 0x7f810019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 -- 172.21.15.44:0/1338185270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810019abc0 msgr2=0x7f810019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810019abc0 0x7f810019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 -- 172.21.15.44:0/1338185270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 msgr2=0x7f81001967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 0x7f81001967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 -- 172.21.15.44:0/1338185270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f4009560 con 0x7f81000fee80 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80fe7fc640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 0x7f81001967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80feffd640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 0x7f8100196290 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f80f000eab0 tx=0x7f80f000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000ce40 con 0x7f81000fee80 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80f0011900 con 0x7f81000fee80 2024-09-13T17:38:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81001a2e70 con 0x7f81000fee80 2024-09-13T17:38:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.129+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f00108c0 con 0x7f81000fee80 2024-09-13T17:38:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.130+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81001a3290 con 0x7f81000fee80 2024-09-13T17:38:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.131+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8100100350 con 0x7f81000fee80 2024-09-13T17:38:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.132+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f80f0011470 con 0x7f81000fee80 2024-09-13T17:38:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.132+0000 7f80e7fff640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80e80a1200 0x7f80e80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.133+0000 7f80fe7fc640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80e80a1200 0x7f80e80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.133+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f80f00cd740 con 0x7f81000fee80 2024-09-13T17:38:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.133+0000 7f80fe7fc640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80e80a1200 0x7f80e80a36c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009900 tx=0x7f80f4005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.137+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f80f0096820 con 0x7f81000fee80 2024-09-13T17:38:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.351+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81000ff260 con 0x7f81000fee80 2024-09-13T17:38:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.352+0000 7f80e7fff640 1 -- 172.21.15.44:0/1338185270 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f80f0099e40 con 0x7f81000fee80 2024-09-13T17:38:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:59.354 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/673823135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":673823135},{"type":"v1","addr":"172.21.15.44:6835","nonce":673823135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3952931988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3952931988},{"type":"v1","addr":"172.21.15.44:6837","nonce":3952931988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/4189498967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":4189498967},{"type":"v1","addr":"172.21.15.44:6839","nonce":4189498967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1193858292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1193858292},{"type":"v1","addr":"172.21.15.44:6841","nonce":1193858292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/982414748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":982414748},{"type":"v1","addr":"172.21.15.70:6833","nonce":982414748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4022128703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4022128703},{"type":"v1","addr":"172.21.15.70:6835","nonce":4022128703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T17:24:21.722369+0000","modified":"2024-09-13T17:26:42.979893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24496,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/459553980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":459553980},{"type":"v1","addr":"172.21.15.100:6833","nonce":459553980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6837/4202473697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":4202473697},{"type":"v1","addr":"172.21.15.70:6837","nonce":4202473697}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/2731404816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":2731404816},{"type":"v1","addr":"172.21.15.70:6839","nonce":2731404816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4065662345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4065662345},{"type":"v1","addr":"172.21.15.100:6835","nonce":4065662345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.100:6837/288521582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":288521582},{"type":"v1","addr":"172.21.15.100:6837","nonce":288521582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.100:6839/3416533473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3416533473},{"type":"v1","addr":"172.21.15.100:6839","nonce":3416533473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:38:59.355 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 25 2024-09-13T17:38:59.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80e80a1200 msgr2=0x7f80e80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80e80a1200 0x7f80e80a36c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009900 tx=0x7f80f4005a20 comp rx=0 tx=0).stop 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 msgr2=0x7f8100196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 0x7f8100196290 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f80f000eab0 tx=0x7f80f000ef70 comp rx=0 tx=0).stop 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 shutdown_connections 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f80e80a1200 0x7f80e80a36c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810019abc0 0x7f810019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8100103fd0 0x7f81001967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 --2- 172.21.15.44:0/1338185270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000fee80 0x7f8100196290 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 >> 172.21.15.44:0/1338185270 conn(0x7f8100075c60 msgr2=0x7f8100102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 shutdown_connections 2024-09-13T17:38:59.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.354+0000 7f8105757640 1 -- 172.21.15.44:0/1338185270 wait complete. 2024-09-13T17:38:59.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 -- 172.21.15.44:0/4010547063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460101860 msgr2=0x7fa460109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 --2- 172.21.15.44:0/4010547063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460101860 0x7fa460109d40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa454009900 tx=0x7fa45402a480 comp rx=0 tx=0).stop 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 -- 172.21.15.44:0/4010547063 shutdown_connections 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 --2- 172.21.15.44:0/4010547063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa46010a310 0x7fa46010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 --2- 172.21.15.44:0/4010547063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460101860 0x7fa460109d40 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 --2- 172.21.15.44:0/4010547063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa460100f40 0x7fa460101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 -- 172.21.15.44:0/4010547063 >> 172.21.15.44:0/4010547063 conn(0x7fa460075c60 msgr2=0x7fa4600fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 -- 172.21.15.44:0/4010547063 shutdown_connections 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.475+0000 7fa466202640 1 -- 172.21.15.44:0/4010547063 wait complete. 2024-09-13T17:38:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 Processor -- start 2024-09-13T17:38:59.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 -- start start 2024-09-13T17:38:59.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 0x7fa460196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa460101860 0x7fa4601966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa46010a310 0x7fa46019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019b0b0 con 0x7fa460100f40 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019b220 con 0x7fa46010a310 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa466202640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019b3e0 con 0x7fa460101860 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 0x7fa460196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 0x7fa460196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34830/0 (socket says 172.21.15.44:34830) 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.476+0000 7fa45f7fe640 1 -- 172.21.15.44:0/821153499 learned_addr learned my addr 172.21.15.44:0/821153499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45ffff640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa46010a310 0x7fa46019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45effd640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa460101860 0x7fa4601966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45f7fe640 1 -- 172.21.15.44:0/821153499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa460101860 msgr2=0x7fa4601966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45f7fe640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa460101860 0x7fa4601966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45f7fe640 1 -- 172.21.15.44:0/821153499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa46010a310 msgr2=0x7fa46019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45f7fe640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa46010a310 0x7fa46019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45f7fe640 1 -- 172.21.15.44:0/821153499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454009560 con 0x7fa460100f40 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45f7fe640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 0x7fa460196180 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d950 tx=0x7fa45000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450009c30 con 0x7fa460100f40 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46019b6c0 con 0x7fa460100f40 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa450009d90 con 0x7fa460100f40 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45000f620 con 0x7fa460100f40 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.477+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4601a2f40 con 0x7fa460100f40 2024-09-13T17:38:59.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.478+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa460067e20 con 0x7fa460100f40 2024-09-13T17:38:59.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.480+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa4500109f0 con 0x7fa460100f40 2024-09-13T17:38:59.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.480+0000 7fa45cff9640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa4480a1200 0x7fa4480a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.480+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7fa4500ccc00 con 0x7fa460100f40 2024-09-13T17:38:59.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.480+0000 7fa45effd640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa4480a1200 0x7fa4480a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.481+0000 7fa45effd640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa4480a1200 0x7fa4480a36c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa45402aa70 tx=0x7fa454005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:59.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.487+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa450095d90 con 0x7fa460100f40 2024-09-13T17:38:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:59 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1851519495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:38:59.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:38:59 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:38:59.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.661+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4601974d0 con 0x7fa460100f40 2024-09-13T17:38:59.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.662+0000 7fa45cff9640 1 -- 172.21.15.44:0/821153499 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25054 (secure 0 0 0) 0x7fa450017090 con 0x7fa460100f40 2024-09-13T17:38:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T17:38:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"fsid":"ce020790-71f3-11ef-bceb-c7b262605968","created":"2024-09-13T17:16:55.305803+0000","modified":"2024-09-13T17:38:11.319123+0000","last_up_change":"2024-09-13T17:23:33.214850+0000","last_in_change":"2024-09-13T17:23:13.820324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:20:01.986440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:24:19.855133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:24:21.127863+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:24:24.170156+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff8d9259-48ec-4ece-86c1-326e4bc5ab62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6803","nonce":637354587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6805","nonce":637354587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6809","nonce":637354587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":637354587},{"type":"v1","addr":"172.21.15.44:6807","nonce":637354587}]},"public_addr":"172.21.15.44:6803/637354587","cluster_addr":"172.21.15.44:6805/637354587","heartbeat_back_addr":"172.21.15.44:6809/637354587","heartbeat_front_addr":"172.21.15.44:6807/637354587","state":["exists","up"]},{"osd":1,"uuid":"8eb850b3-92e7-40d5-a891-9c21f9f591dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6801","nonce":3261074014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6803","nonce":3261074014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6807","nonce":3261074014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":3261074014},{"type":"v1","addr":"172.21.15.70:6805","nonce":3261074014}]},"public_addr":"172.21.15.70:6801/3261074014","cluster_addr":"172.21.15.70:6803/3261074014","heartbeat_back_addr":"172.21.15.70:6807/3261074014","heartbeat_front_addr":"172.21.15.70:6805/3261074014","state":["exists","up"]},{"osd":2,"uuid":"1cfc25ea-477f-4e25-b425-2481547b68c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6801","nonce":4273545793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6803","nonce":4273545793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6807","nonce":4273545793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4273545793},{"type":"v1","addr":"172.21.15.100:6805","nonce":4273545793}]},"public_addr":"172.21.15.100:6801/4273545793","cluster_addr":"172.21.15.100:6803/4273545793","heartbeat_back_addr":"172.21.15.100:6807/4273545793","heartbeat_front_addr":"172.21.15.100:6805/4273545793","state":["exists","up"]},{"osd":3,"uuid":"65f620c4-46d1-40d5-9346-56d4f1cb4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6811","nonce":581674337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6813","nonce":581674337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6817","nonce":581674337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":581674337},{"type":"v1","addr":"172.21.15.44:6815","nonce":581674337}]},"public_addr":"172.21.15.44:6811/581674337","cluster_addr":"172.21.15.44:6813/581674337","heartbeat_back_addr":"172.21.15.44:6817/581674337","heartbeat_front_addr":"172.21.15.44:6815/581674337","state":["exists","up"]},{"osd":4,"uuid":"d9180bdb-24d0-42c8-9d13-51f4ae9812ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6809","nonce":2214036944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6811","nonce":2214036944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6815","nonce":2214036944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2214036944},{"type":"v1","addr":"172.21.15.70:6813","nonce":2214036944}]},"public_addr":"172.21.15.70:6809/2214036944","cluster_addr":"172.21.15.70:6811/2214036944","heartbeat_back_addr":"172.21.15.70:6815/2214036944","heartbeat_front_addr":"172.21.15.70:6813/2214036944","state":["exists","up"]},{"osd":5,"uuid":"55e04db8-8d0a-42bc-a507-aa94db69fa62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6809","nonce":2555541619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6811","nonce":2555541619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6815","nonce":2555541619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2555541619},{"type":"v1","addr":"172.21.15.100:6813","nonce":2555541619}]},"public_addr":"172.21.15.100:6809/2555541619","cluster_addr":"172.21.15.100:6811/2555541619","heartbeat_back_addr":"172.21.15.100:6815/2555541619","heartbeat_front_addr":"172.21.15.100:6813/2555541619","state":["exists","up"]},{"osd":6,"uuid":"18a8c0b4-0801-40db-88a0-ee66c59f28bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6819","nonce":3425095841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6821","nonce":3425095841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6825","nonce":3425095841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3425095841},{"type":"v1","addr":"172.21.15.44:6823","nonce":3425095841}]},"public_addr":"172.21.15.44:6819/3425095841","cluster_addr":"172.21.15.44:6821/3425095841","heartbeat_back_addr":"172.21.15.44:6825/3425095841","heartbeat_front_addr":"172.21.15.44:6823/3425095841","state":["exists","up"]},{"osd":7,"uuid":"f7c62e54-ff2e-4578-8eb7-93a848d283c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6817","nonce":4007129132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6819","nonce":4007129132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6823","nonce":4007129132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4007129132},{"type":"v1","addr":"172.21.15.70:6821","nonce":4007129132}]},"public_addr":"172.21.15.70:6817/4007129132","cluster_addr":"172.21.15.70:6819/4007129132","heartbeat_back_addr":"172.21.15.70:6823/4007129132","heartbeat_front_addr":"172.21.15.70:6821/4007129132","state":["exists","up"]},{"osd":8,"uuid":"c07c1084-7ab4-4531-bb14-893e93b0e1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6817","nonce":2612262054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6819","nonce":2612262054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6823","nonce":2612262054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2612262054},{"type":"v1","addr":"172.21.15.100:6821","nonce":2612262054}]},"public_addr":"172.21.15.100:6817/2612262054","cluster_addr":"172.21.15.100:6819/2612262054","heartbeat_back_addr":"172.21.15.100:6823/2612262054","heartbeat_front_addr":"172.21.15.100:6821/2612262054","state":["exists","up"]},{"osd":9,"uuid":"b38cbad2-3c65-4acc-bc76-68184079cc48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6827","nonce":280929733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6829","nonce":280929733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6833","nonce":280929733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":280929733},{"type":"v1","addr":"172.21.15.44:6831","nonce":280929733}]},"public_addr":"172.21.15.44:6827/280929733","cluster_addr":"172.21.15.44:6829/280929733","heartbeat_back_addr":"172.21.15.44:6833/280929733","heartbeat_front_addr":"172.21.15.44:6831/280929733","state":["exists","up"]},{"osd":10,"uuid":"27a2977e-4fde-4a5f-9be6-9811159c13d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6825","nonce":752114837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6827","nonce":752114837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6831","nonce":752114837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":752114837},{"type":"v1","addr":"172.21.15.70:6829","nonce":752114837}]},"public_addr":"172.21.15.70:6825/752114837","cluster_addr":"172.21.15.70:6827/752114837","heartbeat_back_addr":"172.21.15.70:6831/752114837","heartbeat_front_addr":"172.21.15.70:6829/752114837","state":["exists","up"]},{"osd":11,"uuid":"1fdeaf9a-fb39-48a9-a42c-76de2ae09f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6825","nonce":1155650564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6827","nonce":1155650564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6831","nonce":1155650564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1155650564},{"type":"v1","addr":"172.21.15.100:6829","nonce":1155650564}]},"public_addr":"172.21.15.100:6825/1155650564","cluster_addr":"172.21.15.100:6827/1155650564","heartbeat_back_addr":"172.21.15.100:6831/1155650564","heartbeat_front_addr":"172.21.15.100:6829/1155650564","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:13.240727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:36.474102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:19:58.149026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:21.206475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:20:44.140450+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:07.030830+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:30.383392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:21:54.286630+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:17.339209+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:22:43.019799+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:07.113024+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:23:31.449961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/1409924258":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2038542363":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6801/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/771352345":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/1586278527":"2024-09-14T17:17:41.472796+0000","172.21.15.44:0/4032970337":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/1683772085":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/815607991":"2024-09-14T17:17:17.599897+0000","172.21.15.44:6800/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:6801/2265862199":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6800/3711196986":"2024-09-14T17:17:17.599897+0000","172.21.15.44:0/2209604760":"2024-09-14T17:17:41.472796+0000","172.21.15.44:6801/322597156":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/2025177182":"2024-09-14T17:26:42.344842+0000","172.21.15.44:0/3278871357":"2024-09-14T17:26:42.344842+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa4480a1200 msgr2=0x7fa4480a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa4480a1200 0x7fa4480a36c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa45402aa70 tx=0x7fa454005a20 comp rx=0 tx=0).stop 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 msgr2=0x7fa460196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 0x7fa460196180 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d950 tx=0x7fa45000de10 comp rx=0 tx=0).stop 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 shutdown_connections 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7fa4480a1200 0x7fa4480a36c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa460101860 0x7fa4601966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa46010a310 0x7fa46019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 --2- 172.21.15.44:0/821153499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa460100f40 0x7fa460196180 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 >> 172.21.15.44:0/821153499 conn(0x7fa460075c60 msgr2=0x7fa4600fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 shutdown_connections 2024-09-13T17:38:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.665+0000 7fa466202640 1 -- 172.21.15.44:0/821153499 wait complete. 2024-09-13T17:38:59.679 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:38:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:59 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1851519495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:38:59.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:38:59 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:38:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:59 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1851519495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:38:59.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:38:59 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T17:38:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/2523823178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 msgr2=0x7f4ff410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/2523823178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff410baf0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009cd0 tx=0x7f4fec02a8e0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/2523823178 shutdown_connections 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/2523823178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff410baf0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/2523823178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff40ff850 0x7f4ff4103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/2523823178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff40feea0 0x7f4ff40ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/2523823178 >> 172.21.15.44:0/2523823178 conn(0x7f4ff4075c60 msgr2=0x7f4ff40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/2523823178 shutdown_connections 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/2523823178 wait complete. 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 Processor -- start 2024-09-13T17:38:59.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.781+0000 7f4ffa7f4640 1 -- start start 2024-09-13T17:38:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ffa7f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff40feea0 0x7f4ff4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ffa7f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ffa7f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff4197a00 0x7f4ff41a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ffa7f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41a26a0 con 0x7f4ff4103fa0 2024-09-13T17:38:59.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ffa7f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41a2810 con 0x7f4ff40feea0 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ffa7f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41a29d0 con 0x7f4ff4197a00 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff41974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34840/0 (socket says 172.21.15.44:34840) 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 -- 172.21.15.44:0/4289616164 learned_addr learned my addr 172.21.15.44:0/4289616164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff3fff640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff40feea0 0x7f4ff4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff8d91640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff4197a00 0x7f4ff41a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 -- 172.21.15.44:0/4289616164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff4197a00 msgr2=0x7f4ff41a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff4197a00 0x7f4ff41a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 -- 172.21.15.44:0/4289616164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff40feea0 msgr2=0x7f4ff4196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff40feea0 0x7f4ff4196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:38:59.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 -- 172.21.15.44:0/4289616164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec009900 con 0x7f4ff4103fa0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff37fe640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff41974c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800d770 tx=0x7f4fe800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.782+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8009c30 con 0x7f4ff4103fa0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.783+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff41a2cb0 con 0x7f4ff4103fa0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.783+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fe800de80 con 0x7f4ff4103fa0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.783+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800f620 con 0x7f4ff4103fa0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.783+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff41a31c0 con 0x7f4ff4103fa0 2024-09-13T17:38:59.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.784+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff4100320 con 0x7f4ff4103fa0 2024-09-13T17:38:59.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.785+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4fe8010460 con 0x7f4ff4103fa0 2024-09-13T17:38:59.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.785+0000 7f4ff17fa640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4fdc0a1200 0x7f4fdc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:38:59.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.786+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9403+0+0 (secure 0 0 0) 0x7f4fe80ccd30 con 0x7f4ff4103fa0 2024-09-13T17:38:59.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.786+0000 7f4ff3fff640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4fdc0a1200 0x7f4fdc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:38:59.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.786+0000 7f4ff3fff640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4fdc0a1200 0x7f4fdc0a36c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400bbe0 tx=0x7f4fe400b390 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:38:59.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:38:59.793+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4fe8095ec0 con 0x7f4ff4103fa0 2024-09-13T17:39:00.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.004+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4ff4067bf0 con 0x7f4ff4103fa0 2024-09-13T17:39:00.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.013+0000 7f4ff17fa640 1 -- 172.21.15.44:0/4289616164 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4fe8017090 con 0x7f4ff4103fa0 2024-09-13T17:39:00.015 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:39:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4fdc0a1200 msgr2=0x7f4fdc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4fdc0a1200 0x7f4fdc0a36c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400bbe0 tx=0x7f4fe400b390 comp rx=0 tx=0).stop 2024-09-13T17:39:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 msgr2=0x7f4ff41974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff41974c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800d770 tx=0x7f4fe800dc30 comp rx=0 tx=0).stop 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 shutdown_connections 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f4fdc0a1200 0x7f4fdc0a36c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff4197a00 0x7f4ff41a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff4103fa0 0x7f4ff41974c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 --2- 172.21.15.44:0/4289616164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff40feea0 0x7f4ff4196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 >> 172.21.15.44:0/4289616164 conn(0x7f4ff4075c60 msgr2=0x7f4ff41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 shutdown_connections 2024-09-13T17:39:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.015+0000 7f4ffa7f4640 1 -- 172.21.15.44:0/4289616164 wait complete. 2024-09-13T17:39:00.028 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:39:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.135+0000 7f6134444640 1 -- 172.21.15.44:0/1147732920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c101880 msgr2=0x7f612c109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.135+0000 7f6134444640 1 --2- 172.21.15.44:0/1147732920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c101880 0x7f612c109d60 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802a480 comp rx=0 tx=0).stop 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 -- 172.21.15.44:0/1147732920 shutdown_connections 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 --2- 172.21.15.44:0/1147732920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 0x7f612c10c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 --2- 172.21.15.44:0/1147732920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c101880 0x7f612c109d60 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 --2- 172.21.15.44:0/1147732920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f612c100f60 0x7f612c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 -- 172.21.15.44:0/1147732920 >> 172.21.15.44:0/1147732920 conn(0x7f612c075c60 msgr2=0x7f612c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 -- 172.21.15.44:0/1147732920 shutdown_connections 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 -- 172.21.15.44:0/1147732920 wait complete. 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 Processor -- start 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 -- start start 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.136+0000 7f6134444640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 0x7f612c0febb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f6134444640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f612c101880 0x7f612c0ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f6134444640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 0x7f612c0ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f6134444640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1a4550 con 0x7f612c100f60 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f6134444640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1a46c0 con 0x7f612c10a330 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f6134444640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1a4880 con 0x7f612c101880 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 0x7f612c0febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 0x7f612c0febb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34852/0 (socket says 172.21.15.44:34852) 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 -- 172.21.15.44:0/3170472846 learned_addr learned my addr 172.21.15.44:0/3170472846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61319df640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f612c101880 0x7f612c0ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61329e1640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 0x7f612c0ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 -- 172.21.15.44:0/3170472846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f612c101880 msgr2=0x7f612c0ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f612c101880 0x7f612c0ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 -- 172.21.15.44:0/3170472846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 msgr2=0x7f612c0ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 0x7f612c0ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 -- 172.21.15.44:0/3170472846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128009560 con 0x7f612c100f60 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61329e1640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 0x7f612c0ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61321e0640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 0x7f612c0febb0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f611c00d950 tx=0x7f611c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c009c30 con 0x7f612c100f60 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f611c009d90 con 0x7f612c100f60 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c1a4b60 con 0x7f612c100f60 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.137+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00f620 con 0x7f612c100f60 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.138+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c1a5090 con 0x7f612c100f60 2024-09-13T17:39:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.138+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c067de0 con 0x7f612c100f60 2024-09-13T17:39:00.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.141+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f611c0109f0 con 0x7f612c100f60 2024-09-13T17:39:00.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.141+0000 7f61237fe640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f61180a1280 0x7f61180a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:00.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.141+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9573+0+0 (secure 0 0 0) 0x7f611c0ccd90 con 0x7f612c100f60 2024-09-13T17:39:00.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.141+0000 7f61319df640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f61180a1280 0x7f61180a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:00.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.142+0000 7f61319df640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f61180a1280 0x7f61180a3740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f6128005a20 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:00.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.144+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f611c095de0 con 0x7f612c100f60 2024-09-13T17:39:00.303 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a[31171]: 2024-09-13T17:39:00.005+0000 7f8a74a0b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:39:00.344 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b[64131]: ignoring --setuser ceph since I am not root 2024-09-13T17:39:00.345 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b[64131]: ignoring --setgroup ceph since I am not root 2024-09-13T17:39:00.345 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mds[64135]: -- 172.21.15.70:0/2931806578 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555bfe67ed00 con 0x555bfe653400 2024-09-13T17:39:00.345 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b[64131]: starting mds.b at 2024-09-13T17:39:00.345 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h[63827]: ignoring --setuser ceph since I am not root 2024-09-13T17:39:00.345 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h[63827]: ignoring --setgroup ceph since I am not root 2024-09-13T17:39:00.346 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mds[63831]: -- 172.21.15.70:0/2221542156 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56398498cd00 con 0x563984961000 2024-09-13T17:39:00.346 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h[63827]: starting mds.h at 2024-09-13T17:39:00.346 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c[65592]: ignoring --setuser ceph since I am not root 2024-09-13T17:39:00.346 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c[65592]: ignoring --setgroup ceph since I am not root 2024-09-13T17:39:00.346 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mds[65596]: -- 172.21.15.100:0/3665307645 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d61fee2d00 con 0x55d61feb7400 2024-09-13T17:39:00.347 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c[65592]: starting mds.c at 2024-09-13T17:39:00.347 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f[65287]: ignoring --setuser ceph since I am not root 2024-09-13T17:39:00.347 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f[65287]: ignoring --setgroup ceph since I am not root 2024-09-13T17:39:00.347 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mds[65291]: -- 172.21.15.100:0/850748274 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56070a452d00 con 0x56070a427400 2024-09-13T17:39:00.347 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f[65287]: starting mds.f at 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: ignoring --setuser ceph since I am not root 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: ignoring --setgroup ceph since I am not root 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mds[65906]: -- 172.21.15.100:0/1827232500 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c71c81ed00 con 0x55c71c7f3400 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: starting mds.i at 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l[64984]: ignoring --setuser ceph since I am not root 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l[64984]: ignoring --setgroup ceph since I am not root 2024-09-13T17:39:00.348 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mds[64988]: -- 172.21.15.100:0/601541017 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563bbdc7ed00 con 0x563bbdc53000 2024-09-13T17:39:00.349 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l[64984]: starting mds.l at 2024-09-13T17:39:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:00.355+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f612c105380 con 0x7f612c100f60 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: pgmap v383: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/1338185270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/821153499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:39:00.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4289616164' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/4289616164' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T17:39:00.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:00 smithi044 ceph-mon[31175]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-13T17:39:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: pgmap v383: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-13T17:39:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/1338185270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:39:00.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/821153499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4289616164' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T17:39:00.698 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/4289616164' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T17:39:00.699 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:00 smithi070 ceph-mon[34502]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-13T17:39:00.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: pgmap v383: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-13T17:39:00.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/1338185270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T17:39:00.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/821153499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='mgr.15468 ' entity='mgr.x' 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4289616164' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:39:00.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T17:39:00.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/4289616164' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T17:39:00.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:00 smithi100 ceph-mon[35055]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-13T17:39:01.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.015+0000 7f61237fe640 1 -- 172.21.15.44:0/3170472846 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f611c0200a0 con 0x7f612c100f60 2024-09-13T17:39:01.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f61180a1280 msgr2=0x7f61180a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f61180a1280 0x7f61180a3740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f6128005a20 comp rx=0 tx=0).stop 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 msgr2=0x7f612c0febb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 0x7f612c0febb0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f611c00d950 tx=0x7f611c00de10 comp rx=0 tx=0).stop 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 shutdown_connections 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f61180a1280 0x7f61180a3740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f612c101880 0x7f612c0ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c10a330 0x7f612c0ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 --2- 172.21.15.44:0/3170472846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612c100f60 0x7f612c0febb0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 >> 172.21.15.44:0/3170472846 conn(0x7f612c075c60 msgr2=0x7f612c0fb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:01.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 shutdown_connections 2024-09-13T17:39:01.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.017+0000 7f6134444640 1 -- 172.21.15.44:0/3170472846 wait complete. 2024-09-13T17:39:01.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:39:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.136+0000 7f837dbd4640 1 -- 172.21.15.44:0/274121435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 msgr2=0x7f837810c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.136+0000 7f837dbd4640 1 --2- 172.21.15.44:0/274121435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f837810c130 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f836c009900 tx=0x7f836c02a410 comp rx=0 tx=0).stop 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.136+0000 7f837dbd4640 1 -- 172.21.15.44:0/274121435 shutdown_connections 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.136+0000 7f837dbd4640 1 --2- 172.21.15.44:0/274121435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 0x7f837810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 --2- 172.21.15.44:0/274121435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f837810c130 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 --2- 172.21.15.44:0/274121435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 0x7f83780feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 -- 172.21.15.44:0/274121435 >> 172.21.15.44:0/274121435 conn(0x7f8378075c60 msgr2=0x7f83780fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 -- 172.21.15.44:0/274121435 shutdown_connections 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 -- 172.21.15.44:0/274121435 wait complete. 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 Processor -- start 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.137+0000 7f837dbd4640 1 -- start start 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f837dbd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 0x7f83781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f837dbd4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f8378198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f837dbd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 0x7f837819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f837dbd4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837819d300 con 0x7f83780ff470 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f837dbd4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837819d470 con 0x7f83780feac0 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f837dbd4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837819d630 con 0x7f837810c670 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f8378198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f8378198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34868/0 (socket says 172.21.15.44:34868) 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 -- 172.21.15.44:0/3559854953 learned_addr learned my addr 172.21.15.44:0/3559854953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f83777fe640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 0x7f83781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8377fff640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 0x7f837819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 -- 172.21.15.44:0/3559854953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 msgr2=0x7f837819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 0x7f837819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 -- 172.21.15.44:0/3559854953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 msgr2=0x7f83781983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 0x7f83781983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:01.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 -- 172.21.15.44:0/3559854953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c009560 con 0x7f83780ff470 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8377fff640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 0x7f837819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f83777fe640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 0x7f83781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8376ffd640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f8378198910 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f836c0057c0 tx=0x7f836c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0044d0 con 0x7f83780ff470 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.138+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f836c032e10 con 0x7f83780ff470 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.139+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837819d8b0 con 0x7f83780ff470 2024-09-13T17:39:01.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.139+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c003850 con 0x7f83780ff470 2024-09-13T17:39:01.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.139+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83781a5190 con 0x7f83780ff470 2024-09-13T17:39:01.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.139+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8378067e00 con 0x7f83780ff470 2024-09-13T17:39:01.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.141+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f836c004850 con 0x7f83780ff470 2024-09-13T17:39:01.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.142+0000 7f8374ff9640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f83600a1280 0x7f83600a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:01.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.142+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9573+0+0 (secure 0 0 0) 0x7f836c0ee140 con 0x7f83780ff470 2024-09-13T17:39:01.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.142+0000 7f83777fe640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f83600a1280 0x7f83600a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:01.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.143+0000 7f83777fe640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f83600a1280 0x7f83600a3740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8368009fc0 tx=0x7f8368005f90 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:01.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.144+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f836c0b7170 con 0x7f83780ff470 2024-09-13T17:39:01.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:01.318+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8378107800 con 0x7f83780ff470 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3170472846' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: Cluster is now healthy 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6832/2974991677,v1:172.21.15.100:6833/2974991677] up:boot 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6834/1096315189,v1:172.21.15.100:6835/1096315189] up:boot 2024-09-13T17:39:01.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6836/844822554,v1:172.21.15.100:6837/844822554] up:boot 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6837/1393991735,v1:172.21.15.70:6839/1393991735] up:boot 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.70:6836/617214297,v1:172.21.15.70:6838/617214297] up:boot 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: mds.? [v2:172.21.15.100:6838/2859767114,v1:172.21.15.100:6839/2859767114] up:boot 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3170472846' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: fsmap 12 up:standby 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:01 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3559854953' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:01.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3170472846' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:39:01.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T17:39:01.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T17:39:01.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: Cluster is now healthy 2024-09-13T17:39:01.676 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6832/2974991677,v1:172.21.15.100:6833/2974991677] up:boot 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6834/1096315189,v1:172.21.15.100:6835/1096315189] up:boot 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6836/844822554,v1:172.21.15.100:6837/844822554] up:boot 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6837/1393991735,v1:172.21.15.70:6839/1393991735] up:boot 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.70:6836/617214297,v1:172.21.15.70:6838/617214297] up:boot 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: mds.? [v2:172.21.15.100:6838/2859767114,v1:172.21.15.100:6839/2859767114] up:boot 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3170472846' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: fsmap 12 up:standby 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:39:01.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:39:01.678 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:39:01.678 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:39:01.678 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:39:01.678 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:39:01.678 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:01 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3559854953' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:01.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3170472846' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T17:39:01.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T17:39:01.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T17:39:01.700 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: Cluster is now healthy 2024-09-13T17:39:01.700 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6832/2974991677,v1:172.21.15.100:6833/2974991677] up:boot 2024-09-13T17:39:01.700 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6834/1096315189,v1:172.21.15.100:6835/1096315189] up:boot 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6836/844822554,v1:172.21.15.100:6837/844822554] up:boot 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6837/1393991735,v1:172.21.15.70:6839/1393991735] up:boot 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.70:6836/617214297,v1:172.21.15.70:6838/617214297] up:boot 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: mds.? [v2:172.21.15.100:6838/2859767114,v1:172.21.15.100:6839/2859767114] up:boot 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3170472846' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: fsmap 12 up:standby 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T17:39:01.701 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T17:39:01.702 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T17:39:01.702 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T17:39:01.702 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T17:39:01.702 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='mgr.15468 172.21.15.44:0/1053005314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T17:39:01.702 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:01 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3559854953' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:02.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.021+0000 7f8374ff9640 1 -- 172.21.15.44:0/3559854953 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7f836c0350d0 con 0x7f83780ff470 2024-09-13T17:39:02.023 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.023+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f83600a1280 msgr2=0x7f83600a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.023+0000 7f837dbd4640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f83600a1280 0x7f83600a3740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8368009fc0 tx=0x7f8368005f90 comp rx=0 tx=0).stop 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.023+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 msgr2=0x7f8378198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.023+0000 7f837dbd4640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f8378198910 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f836c0057c0 tx=0x7f836c032c20 comp rx=0 tx=0).stop 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 shutdown_connections 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f83600a1280 0x7f83600a3740 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810c670 0x7f837819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83780ff470 0x7f8378198910 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 --2- 172.21.15.44:0/3559854953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83780feac0 0x7f83781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 >> 172.21.15.44:0/3559854953 conn(0x7f8378075c60 msgr2=0x7f83780fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:02.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 shutdown_connections 2024-09-13T17:39:02.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.024+0000 7f837dbd4640 1 -- 172.21.15.44:0/3559854953 wait complete. 2024-09-13T17:39:02.040 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 -- 172.21.15.44:0/2934476058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 msgr2=0x7f3474100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 --2- 172.21.15.44:0/2934476058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f3474100370 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3464009930 tx=0x7f346402a420 comp rx=0 tx=0).stop 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 -- 172.21.15.44:0/2934476058 shutdown_connections 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 --2- 172.21.15.44:0/2934476058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474100940 0x7f3474104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 --2- 172.21.15.44:0/2934476058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f347410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 --2- 172.21.15.44:0/2934476058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f3474100370 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 -- 172.21.15.44:0/2934476058 >> 172.21.15.44:0/2934476058 conn(0x7f34740fba20 msgr2=0x7f34740fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 -- 172.21.15.44:0/2934476058 shutdown_connections 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.152+0000 7f3479f57640 1 -- 172.21.15.44:0/2934476058 wait complete. 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 Processor -- start 2024-09-13T17:39:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 -- start start 2024-09-13T17:39:02.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f34740727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:02.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f3474072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:02.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474073230 0x7f34741a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34741a4a20 con 0x7f34740fff90 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34741a4b90 con 0x7f34741050e0 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3479f57640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34741a4d50 con 0x7f3474073230 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f346affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f3474072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f346affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f3474072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36736/0 (socket says 172.21.15.44:36736) 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f346affd640 1 -- 172.21.15.44:0/3651208615 learned_addr learned my addr 172.21.15.44:0/3651208615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.153+0000 7f3473fff640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474073230 0x7f34741a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f34737fe640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f34740727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f346affd640 1 -- 172.21.15.44:0/3651208615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474073230 msgr2=0x7f34741a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:02.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f346affd640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474073230 0x7f34741a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f346affd640 1 -- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 msgr2=0x7f34740727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f346affd640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f34740727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f346affd640 1 -- 172.21.15.44:0/3651208615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464009560 con 0x7f34741050e0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f3473fff640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474073230 0x7f34741a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f34737fe640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f34740727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f346affd640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f3474072cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f345c00e9f0 tx=0x7f345c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c0096c0 con 0x7f34741050e0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f345c00ccc0 con 0x7f34741050e0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34741a4fd0 con 0x7f34741050e0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c004e60 con 0x7f34741050e0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.154+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34741a5460 con 0x7f34741050e0 2024-09-13T17:39:02.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.155+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3474101460 con 0x7f34741050e0 2024-09-13T17:39:02.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.156+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f345c004fc0 con 0x7f34741050e0 2024-09-13T17:39:02.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.157+0000 7f34717fa640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34600a1230 0x7f34600a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:02.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.157+0000 7f34737fe640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34600a1230 0x7f34600a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:02.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.157+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9040+0+0 (secure 0 0 0) 0x7f345c014070 con 0x7f34741050e0 2024-09-13T17:39:02.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.158+0000 7f34737fe640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34600a1230 0x7f34600a36f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f3464005a40 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.160+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f345c096700 con 0x7f34741050e0 2024-09-13T17:39:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:02.333+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3474100370 con 0x7f34741050e0 2024-09-13T17:39:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:02 smithi044 ceph-mon[31175]: pgmap v385: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-13T17:39:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3559854953' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T17:39:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:02 smithi044 ceph-mon[31175]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T17:39:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:02 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3651208615' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:02.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:02 smithi044 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:02 smithi070 ceph-mon[34502]: pgmap v385: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-13T17:39:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:02 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3559854953' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T17:39:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:02 smithi070 ceph-mon[34502]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T17:39:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:02 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3651208615' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:02.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:02 smithi070 ceph-mon[34502]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:02 smithi100 ceph-mon[35055]: pgmap v385: 129 pgs: 129 active+clean; 2.4 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-13T17:39:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:02 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3559854953' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T17:39:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:02 smithi100 ceph-mon[35055]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T17:39:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:02 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3651208615' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:02.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:02 smithi100 ceph-mon[35055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:03.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.020+0000 7f34717fa640 1 -- 172.21.15.44:0/3651208615 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7f345c099d20 con 0x7f34741050e0 2024-09-13T17:39:03.023 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34600a1230 msgr2=0x7f34600a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34600a1230 0x7f34600a36f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f3464005a40 comp rx=0 tx=0).stop 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 msgr2=0x7f3474072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f3474072cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f345c00e9f0 tx=0x7f345c00eeb0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 shutdown_connections 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f34600a1230 0x7f34600a36f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3474073230 0x7f34741a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34741050e0 0x7f3474072cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 --2- 172.21.15.44:0/3651208615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34740fff90 0x7f34740727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 >> 172.21.15.44:0/3651208615 conn(0x7f34740fba20 msgr2=0x7f34740fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:03.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.024+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 shutdown_connections 2024-09-13T17:39:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.025+0000 7f3479f57640 1 -- 172.21.15.44:0/3651208615 wait complete. 2024-09-13T17:39:03.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T17:39:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.152+0000 7f07a985a640 1 -- 172.21.15.44:0/3857994432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a40728e0 msgr2=0x7f07a4072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.152+0000 7f07a985a640 1 --2- 172.21.15.44:0/3857994432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a40728e0 0x7f07a4072cc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f0794009900 tx=0x7f079402a480 comp rx=0 tx=0).stop 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 -- 172.21.15.44:0/3857994432 shutdown_connections 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 --2- 172.21.15.44:0/3857994432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a4073200 0x7f07a410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 --2- 172.21.15.44:0/3857994432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a410c520 0x7f07a410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 --2- 172.21.15.44:0/3857994432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a40728e0 0x7f07a4072cc0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 -- 172.21.15.44:0/3857994432 >> 172.21.15.44:0/3857994432 conn(0x7f07a40fb9e0 msgr2=0x7f07a40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 -- 172.21.15.44:0/3857994432 shutdown_connections 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 -- 172.21.15.44:0/3857994432 wait complete. 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 Processor -- start 2024-09-13T17:39:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.153+0000 7f07a985a640 1 -- start start 2024-09-13T17:39:03.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a985a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a40728e0 0x7f07a41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:03.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a985a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 0x7f07a41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:03.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a985a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a410c520 0x7f07a419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a985a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a419d2d0 con 0x7f07a4073200 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a985a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a419d440 con 0x7f07a40728e0 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a985a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a419d600 con 0x7f07a410c520 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 0x7f07a41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 0x7f07a41988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34908/0 (socket says 172.21.15.44:34908) 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 -- 172.21.15.44:0/3711536919 learned_addr learned my addr 172.21.15.44:0/3711536919 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a37fe640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a410c520 0x7f07a419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a2ffd640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a40728e0 0x7f07a41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 -- 172.21.15.44:0/3711536919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a410c520 msgr2=0x7f07a419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a410c520 0x7f07a419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 -- 172.21.15.44:0/3711536919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a40728e0 msgr2=0x7f07a41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a40728e0 0x7f07a41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:03.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 -- 172.21.15.44:0/3711536919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009560 con 0x7f07a4073200 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a2ffd640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a40728e0 0x7f07a41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.154+0000 7f07a27fc640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 0x7f07a41988e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f079800d950 tx=0x7f079800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.155+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798009c30 con 0x7f07a4073200 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.155+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0798009d90 con 0x7f07a4073200 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.155+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a419d8e0 con 0x7f07a4073200 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.155+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800f620 con 0x7f07a4073200 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.155+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a41a5160 con 0x7f07a4073200 2024-09-13T17:39:03.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.155+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a4073f90 con 0x7f07a4073200 2024-09-13T17:39:03.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.157+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f07980109f0 con 0x7f07a4073200 2024-09-13T17:39:03.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.158+0000 7f07a8858640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f078c0a1280 0x7f078c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:39:03.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.158+0000 7f07a2ffd640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f078c0a1280 0x7f078c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:39:03.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.158+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8510+0+0 (secure 0 0 0) 0x7f07980cc850 con 0x7f07a4073200 2024-09-13T17:39:03.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.158+0000 7f07a2ffd640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f078c0a1280 0x7f078c0a3740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0794000c00 tx=0x7f0794005e00 comp rx=0 tx=0).ready entity=mgr.15468 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:39:03.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.161+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0798095d30 con 0x7f07a4073200 2024-09-13T17:39:03.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:03.336+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f07a4072cc0 con 0x7f07a4073200 2024-09-13T17:39:04.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.021+0000 7f07a8858640 1 -- 172.21.15.44:0/3711536919 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0798020020 con 0x7f07a4073200 2024-09-13T17:39:04.024 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data_ec' removed 2024-09-13T17:39:04.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f078c0a1280 msgr2=0x7f078c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f078c0a1280 0x7f078c0a3740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0794000c00 tx=0x7f0794005e00 comp rx=0 tx=0).stop 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 msgr2=0x7f07a41988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 0x7f07a41988e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f079800d950 tx=0x7f079800de10 comp rx=0 tx=0).stop 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 shutdown_connections 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:6800/4124099065,v1:172.21.15.44:6801/4124099065] conn(0x7f078c0a1280 0x7f078c0a3740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07a410c520 0x7f07a419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a4073200 0x7f07a41988e0 secure :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f079800d950 tx=0x7f079800de10 comp rx=0 tx=0).stop 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 --2- 172.21.15.44:0/3711536919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a40728e0 0x7f07a41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.025+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 >> 172.21.15.44:0/3711536919 conn(0x7f07a40fb9e0 msgr2=0x7f07a40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:39:04.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.026+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 shutdown_connections 2024-09-13T17:39:04.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T17:39:04.026+0000 7f07a985a640 1 -- 172.21.15.44:0/3711536919 wait complete. 2024-09-13T17:39:04.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:39:04.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:39:04.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:39:04.139 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-13T17:39:04.140 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T17:39:04.158 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T17:39:04.176 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T17:39:04.197 INFO:tasks.cephadm:Stopping all daemons... 2024-09-13T17:39:04.198 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-13T17:39:04.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a 2024-09-13T17:39:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 ceph-mon[31175]: pgmap v387: 97 pgs: 97 active+clean; 577 KiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:39:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T17:39:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 ceph-mon[31175]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T17:39:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 ceph-mon[31175]: from='client.? 172.21.15.44:0/3711536919' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:04.357 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 systemd[1]: Stopping Ceph mon.a for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:04 smithi070 ceph-mon[34502]: pgmap v387: 97 pgs: 97 active+clean; 577 KiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:39:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:04 smithi070 ceph-mon[34502]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T17:39:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:04 smithi070 ceph-mon[34502]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T17:39:04.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:04 smithi070 ceph-mon[34502]: from='client.? 172.21.15.44:0/3711536919' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:04 smithi100 ceph-mon[35055]: pgmap v387: 97 pgs: 97 active+clean; 577 KiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T17:39:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:04 smithi100 ceph-mon[35055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T17:39:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:04 smithi100 ceph-mon[35055]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T17:39:04.524 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:04 smithi100 ceph-mon[35055]: from='client.? 172.21.15.44:0/3711536919' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T17:39:04.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a[31171]: 2024-09-13T17:39:04.513+0000 7f8a7a216640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:04.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a[31171]: 2024-09-13T17:39:04.513+0000 7f8a7a216640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-13T17:39:04.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 podman[103921]: 2024-09-13 17:39:04.545018525 +0000 UTC m=+0.149463506 container died 1c73e264c322c4a206bd3b3bfecd885131e6adc5f7b0051336234f2f55e60528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:39:05.226 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 podman[103921]: 2024-09-13 17:39:04.956343963 +0000 UTC m=+0.560788941 container cleanup 1c73e264c322c4a206bd3b3bfecd885131e6adc5f7b0051336234f2f55e60528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-13T17:39:05.226 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 13 17:39:04 smithi044 bash[103921]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-a 2024-09-13T17:39:05.352 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.a.service' 2024-09-13T17:39:05.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:05.400 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-13T17:39:05.400 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-13T17:39:05.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.b 2024-09-13T17:39:05.697 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:05 smithi070 systemd[1]: Stopping Ceph mon.b for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:06.161 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-b[34498]: 2024-09-13T17:39:05.732+0000 7f366d275640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:06.161 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-b[34498]: 2024-09-13T17:39:05.732+0000 7f366d275640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-13T17:39:06.161 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:05 smithi070 podman[69774]: 2024-09-13 17:39:05.814718999 +0000 UTC m=+0.207570073 container died 2596fd5d3a2a7fcd4d95fe9d78fc144ba81aa0e286d70aff0bc5e933a771abdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-13T17:39:06.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:06 smithi070 podman[69774]: 2024-09-13 17:39:06.259249558 +0000 UTC m=+0.652100586 container cleanup 2596fd5d3a2a7fcd4d95fe9d78fc144ba81aa0e286d70aff0bc5e933a771abdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:39:06.447 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:06 smithi070 bash[69774]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-b 2024-09-13T17:39:06.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.b.service' 2024-09-13T17:39:06.738 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:06 smithi070 podman[69786]: 2024-09-13 17:39:06.477232213 +0000 UTC m=+0.656786973 container remove 2596fd5d3a2a7fcd4d95fe9d78fc144ba81aa0e286d70aff0bc5e933a771abdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:39:06.739 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:06 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-13T17:39:06.739 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:06 smithi070 systemd[1]: Stopped Ceph mon.b for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:39:06.739 INFO:journalctl@ceph.mon.b.smithi070.stdout:Sep 13 17:39:06 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.b.service: Consumed 13.855s CPU time. 2024-09-13T17:39:06.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:06.784 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-13T17:39:06.784 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-13T17:39:06.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.c 2024-09-13T17:39:07.102 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:06 smithi100 systemd[1]: Stopping Ceph mon.c for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:07.357 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-c[35051]: 2024-09-13T17:39:07.100+0000 7f542c0cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:07.357 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-c[35051]: 2024-09-13T17:39:07.100+0000 7f542c0cb640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-13T17:39:07.612 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:07 smithi100 podman[72356]: 2024-09-13 17:39:07.356769508 +0000 UTC m=+0.374494466 container died 7b5c424e19e5a454de093fe54b23f85e0f2742b0d663f0bd7a86ebcf92d49c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-c, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:39:07.996 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:07 smithi100 podman[72356]: 2024-09-13 17:39:07.70977527 +0000 UTC m=+0.727500219 container cleanup 7b5c424e19e5a454de093fe54b23f85e0f2742b0d663f0bd7a86ebcf92d49c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-13T17:39:07.996 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 13 17:39:07 smithi100 bash[72356]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mon-c 2024-09-13T17:39:08.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mon.c.service' 2024-09-13T17:39:08.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:08.169 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-13T17:39:08.170 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-13T17:39:08.170 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x 2024-09-13T17:39:08.479 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:39:08 smithi044 systemd[1]: Stopping Ceph mgr.x for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:08.789 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:39:08 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:39:08.477+0000 7fb71a12a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:08.789 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:39:08 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x[31404]: 2024-09-13T17:39:08.477+0000 7fb71a12a640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-13T17:39:08.789 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:39:08 smithi044 podman[104043]: 2024-09-13 17:39:08.509393377 +0000 UTC m=+0.138198184 container died 09db5f4cccf04d4298e637d2d2f3377bf3cb6dc42d4e46f0262b2b74c92cf647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-13T17:39:09.107 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:39:08 smithi044 podman[104043]: 2024-09-13 17:39:08.904601841 +0000 UTC m=+0.533406647 container cleanup 09db5f4cccf04d4298e637d2d2f3377bf3cb6dc42d4e46f0262b2b74c92cf647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-13T17:39:09.107 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 13 17:39:08 smithi044 bash[104043]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-x 2024-09-13T17:39:09.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-13T17:39:09.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:09.384 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-13T17:39:09.384 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-13T17:39:09.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.y 2024-09-13T17:39:09.697 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:39:09 smithi070 systemd[1]: Stopping Ceph mgr.y for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:09.965 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:39:09 smithi070 podman[69896]: 2024-09-13 17:39:09.725469381 +0000 UTC m=+0.147371765 container died e0a4f1005dfb7979c705f9a898c8c85138053a5ef18157818b9d2d5985bdf961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2) 2024-09-13T17:39:10.367 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:39:10 smithi070 podman[69896]: 2024-09-13 17:39:10.147602003 +0000 UTC m=+0.569504387 container cleanup e0a4f1005dfb7979c705f9a898c8c85138053a5ef18157818b9d2d5985bdf961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:39:10.367 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:39:10 smithi070 bash[69896]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y 2024-09-13T17:39:10.617 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:39:10 smithi070 podman[69909]: 2024-09-13 17:39:10.366051718 +0000 UTC m=+0.635212857 container remove e0a4f1005dfb7979c705f9a898c8c85138053a5ef18157818b9d2d5985bdf961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-13T17:39:10.618 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 13 17:39:10 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-13T17:39:10.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-13T17:39:10.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:10.669 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-13T17:39:10.669 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-13T17:39:10.669 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.z 2024-09-13T17:39:11.002 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:39:10 smithi100 systemd[1]: Stopping Ceph mgr.z for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:11.274 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:39:11 smithi100 podman[72482]: 2024-09-13 17:39:11.002074792 +0000 UTC m=+0.134211248 container died f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T17:39:11.651 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:39:11 smithi100 podman[72482]: 2024-09-13 17:39:11.423583906 +0000 UTC m=+0.555720351 container cleanup f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:39:11.651 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:39:11 smithi100 bash[72482]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z 2024-09-13T17:39:11.906 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:39:11 smithi100 podman[72496]: 2024-09-13 17:39:11.650434211 +0000 UTC m=+0.640752267 container remove f5f8c60772257f367d6b7526e4196b31cd6f714017f202802b89e31d2dba356c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:39:11.906 INFO:journalctl@ceph.mgr.z.smithi100.stdout:Sep 13 17:39:11 smithi100 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-13T17:39:11.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-13T17:39:11.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:11.946 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-13T17:39:11.946 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-13T17:39:11.946 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.0 2024-09-13T17:39:12.269 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:11 smithi044 systemd[1]: Stopping Ceph osd.0 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:12.606 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0[45525]: 2024-09-13T17:39:12.266+0000 7f0784afa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:12.607 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0[45525]: 2024-09-13T17:39:12.266+0000 7f0784afa640 -1 osd.0 109 *** Got signal Terminated *** 2024-09-13T17:39:12.607 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0[45525]: 2024-09-13T17:39:12.266+0000 7f0784afa640 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:18.053 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:17 smithi044 podman[104165]: 2024-09-13 17:39:17.767063139 +0000 UTC m=+5.625191013 container died 2f3550ac5cc0d7f7a3e093455c91dd7e018c3dd526e312f8e4c6a455afc6f076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-13T17:39:18.356 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:18 smithi044 podman[104165]: 2024-09-13 17:39:18.134877649 +0000 UTC m=+5.993005530 container cleanup 2f3550ac5cc0d7f7a3e093455c91dd7e018c3dd526e312f8e4c6a455afc6f076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:39:18.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:18 smithi044 bash[104165]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0 2024-09-13T17:39:18.772 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:18 smithi044 podman[104184]: 2024-09-13 17:39:18.411499302 +0000 UTC m=+0.641128205 container remove 2f3550ac5cc0d7f7a3e093455c91dd7e018c3dd526e312f8e4c6a455afc6f076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-13T17:39:19.066 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:18 smithi044 podman[104273]: 2024-09-13 17:39:18.673855011 +0000 UTC m=+0.016615325 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:39:19.066 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:18 smithi044 podman[104273]: 2024-09-13 17:39:18.829346575 +0000 UTC m=+0.172106876 container create 93ad386b709e2b6ddafeaff87422e22290fe78b037bfa5ddb7edc585f9801089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T17:39:19.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:19 smithi044 podman[104273]: 2024-09-13 17:39:19.065869948 +0000 UTC m=+0.408630252 container init 93ad386b709e2b6ddafeaff87422e22290fe78b037bfa5ddb7edc585f9801089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:39:19.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:19 smithi044 podman[104273]: 2024-09-13 17:39:19.069550034 +0000 UTC m=+0.412310344 container start 93ad386b709e2b6ddafeaff87422e22290fe78b037bfa5ddb7edc585f9801089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-13T17:39:19.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:19 smithi044 podman[104273]: 2024-09-13 17:39:19.140713381 +0000 UTC m=+0.483473688 container attach 93ad386b709e2b6ddafeaff87422e22290fe78b037bfa5ddb7edc585f9801089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T17:39:19.357 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 13 17:39:19 smithi044 podman[104273]: 2024-09-13 17:39:19.259376464 +0000 UTC m=+0.602136776 container died 93ad386b709e2b6ddafeaff87422e22290fe78b037bfa5ddb7edc585f9801089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:39:19.822 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.0.service' 2024-09-13T17:39:19.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:19.864 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-13T17:39:19.864 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-13T17:39:19.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.1 2024-09-13T17:39:20.195 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:19 smithi070 systemd[1]: Stopping Ceph osd.1 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:20.447 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1[38995]: 2024-09-13T17:39:20.193+0000 7f9bb0f5a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:20.447 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1[38995]: 2024-09-13T17:39:20.193+0000 7f9bb0f5a640 -1 osd.1 109 *** Got signal Terminated *** 2024-09-13T17:39:20.447 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1[38995]: 2024-09-13T17:39:20.193+0000 7f9bb0f5a640 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:25.820 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:25 smithi070 podman[70035]: 2024-09-13 17:39:25.525297835 +0000 UTC m=+5.470264576 container died 5eb1a2557fe6530b5546b6edd7101c009da034302b8d13987f077a1fde05e38e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-13T17:39:26.179 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:25 smithi070 podman[70035]: 2024-09-13 17:39:25.918031922 +0000 UTC m=+5.862998662 container cleanup 5eb1a2557fe6530b5546b6edd7101c009da034302b8d13987f077a1fde05e38e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux ) 2024-09-13T17:39:26.179 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:25 smithi070 bash[70035]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1 2024-09-13T17:39:26.447 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:26 smithi070 podman[70050]: 2024-09-13 17:39:26.177978163 +0000 UTC m=+0.655016009 container remove 5eb1a2557fe6530b5546b6edd7101c009da034302b8d13987f077a1fde05e38e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-13T17:39:26.840 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:26 smithi070 podman[70142]: 2024-09-13 17:39:26.435495438 +0000 UTC m=+0.015917615 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:39:26.841 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:26 smithi070 podman[70142]: 2024-09-13 17:39:26.545553428 +0000 UTC m=+0.125975594 container create ba73cf261b1947317ee6226d24f7dd4454c0c2063631d85a4cddbdf66b38a306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-13T17:39:27.197 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:26 smithi070 podman[70142]: 2024-09-13 17:39:26.840010115 +0000 UTC m=+0.420432294 container init ba73cf261b1947317ee6226d24f7dd4454c0c2063631d85a4cddbdf66b38a306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, ceph=True) 2024-09-13T17:39:27.197 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:26 smithi070 podman[70142]: 2024-09-13 17:39:26.843395236 +0000 UTC m=+0.423817402 container start ba73cf261b1947317ee6226d24f7dd4454c0c2063631d85a4cddbdf66b38a306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:39:27.197 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:26 smithi070 podman[70142]: 2024-09-13 17:39:26.9231832 +0000 UTC m=+0.503605367 container attach ba73cf261b1947317ee6226d24f7dd4454c0c2063631d85a4cddbdf66b38a306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-13T17:39:27.197 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 13 17:39:27 smithi070 podman[70142]: 2024-09-13 17:39:27.026741175 +0000 UTC m=+0.607163342 container died ba73cf261b1947317ee6226d24f7dd4454c0c2063631d85a4cddbdf66b38a306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:39:27.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.1.service' 2024-09-13T17:39:27.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:27.709 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-13T17:39:27.709 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-13T17:39:27.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.2 2024-09-13T17:39:28.021 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:27 smithi100 systemd[1]: Stopping Ceph osd.2 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:28.274 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2[40248]: 2024-09-13T17:39:28.018+0000 7f927d74c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:28.274 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2[40248]: 2024-09-13T17:39:28.018+0000 7f927d74c640 -1 osd.2 109 *** Got signal Terminated *** 2024-09-13T17:39:28.274 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2[40248]: 2024-09-13T17:39:28.018+0000 7f927d74c640 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:33.638 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:33 smithi100 podman[72606]: 2024-09-13 17:39:33.371559283 +0000 UTC m=+5.465479844 container died 03562ee101f7c339e7ce170d66d891bd8c0f79efca421ed1fe1536f36d0051e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T17:39:33.955 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:33 smithi100 podman[72606]: 2024-09-13 17:39:33.727987893 +0000 UTC m=+5.821908457 container cleanup 03562ee101f7c339e7ce170d66d891bd8c0f79efca421ed1fe1536f36d0051e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-13T17:39:33.955 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:33 smithi100 bash[72606]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2 2024-09-13T17:39:34.274 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:33 smithi100 podman[72621]: 2024-09-13 17:39:33.954605418 +0000 UTC m=+0.584895037 container remove 03562ee101f7c339e7ce170d66d891bd8c0f79efca421ed1fe1536f36d0051e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-13T17:39:34.654 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:34 smithi100 podman[72712]: 2024-09-13 17:39:34.304889656 +0000 UTC m=+0.016128484 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:39:34.654 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:34 smithi100 podman[72712]: 2024-09-13 17:39:34.422294176 +0000 UTC m=+0.133532990 container create 34186ef79ee87fc0c0f066a7acd3a46bb8a45ce9c47c9003eae64ffd2ff765ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:39:34.654 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:34 smithi100 podman[72712]: 2024-09-13 17:39:34.649822493 +0000 UTC m=+0.361061307 container init 34186ef79ee87fc0c0f066a7acd3a46bb8a45ce9c47c9003eae64ffd2ff765ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:39:34.654 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:34 smithi100 podman[72712]: 2024-09-13 17:39:34.653215677 +0000 UTC m=+0.364454502 container start 34186ef79ee87fc0c0f066a7acd3a46bb8a45ce9c47c9003eae64ffd2ff765ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD) 2024-09-13T17:39:35.025 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:34 smithi100 podman[72712]: 2024-09-13 17:39:34.741562743 +0000 UTC m=+0.452801558 container attach 34186ef79ee87fc0c0f066a7acd3a46bb8a45ce9c47c9003eae64ffd2ff765ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-13T17:39:35.025 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 13 17:39:34 smithi100 podman[72712]: 2024-09-13 17:39:34.845381848 +0000 UTC m=+0.556620673 container died 34186ef79ee87fc0c0f066a7acd3a46bb8a45ce9c47c9003eae64ffd2ff765ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-13T17:39:35.547 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.2.service' 2024-09-13T17:39:35.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:35.593 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-13T17:39:35.593 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-13T17:39:35.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.3 2024-09-13T17:39:35.915 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:35 smithi044 systemd[1]: Stopping Ceph osd.3 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:36.191 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:35 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3[51955]: 2024-09-13T17:39:35.914+0000 7f3e9a4e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:36.192 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:35 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3[51955]: 2024-09-13T17:39:35.914+0000 7f3e9a4e4640 -1 osd.3 109 *** Got signal Terminated *** 2024-09-13T17:39:36.192 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:35 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3[51955]: 2024-09-13T17:39:35.914+0000 7f3e9a4e4640 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:40.357 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:39 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3[51955]: 2024-09-13T17:39:39.931+0000 7f3e92af0640 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:15.683391+0000 front 2024-09-13T17:39:15.683436+0000 (oldest deadline 2024-09-13T17:39:39.783551+0000) 2024-09-13T17:39:40.774 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:40.382+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:40.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:40 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:40.665+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:41.516 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:41 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:41.195+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:41.517 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:41 smithi044 podman[104375]: 2024-09-13 17:39:41.223323281 +0000 UTC m=+5.426833373 container died c6ad6fa2b107511a0f14d7375a5d600c40268ce4859ab82303b8fc07e558ae7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-13T17:39:41.774 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:41.341+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:41.856 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:41 smithi044 podman[104375]: 2024-09-13 17:39:41.731105237 +0000 UTC m=+5.934615315 container cleanup c6ad6fa2b107511a0f14d7375a5d600c40268ce4859ab82303b8fc07e558ae7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:39:41.857 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:41 smithi044 bash[104375]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3 2024-09-13T17:39:41.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:41 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:41.640+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:42.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:41.834+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:42.292 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:41 smithi044 podman[104401]: 2024-09-13 17:39:41.957522664 +0000 UTC m=+0.726038532 container remove c6ad6fa2b107511a0f14d7375a5d600c40268ce4859ab82303b8fc07e558ae7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T17:39:42.293 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:42 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:42.162+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:42.588 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:42 smithi044 podman[104492]: 2024-09-13 17:39:42.194717847 +0000 UTC m=+0.016523510 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:39:42.588 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:42 smithi044 podman[104492]: 2024-09-13 17:39:42.318858089 +0000 UTC m=+0.140663753 container create f9d5aadee91854840c72d03bfe22ef1b46c97cf6b4e3a3cdee3aa74876ec2a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-13T17:39:42.588 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:42 smithi044 podman[104492]: 2024-09-13 17:39:42.586257515 +0000 UTC m=+0.408063180 container init f9d5aadee91854840c72d03bfe22ef1b46c97cf6b4e3a3cdee3aa74876ec2a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-13T17:39:42.774 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:42.372+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:42.857 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:42 smithi044 podman[104492]: 2024-09-13 17:39:42.589926388 +0000 UTC m=+0.411732054 container start f9d5aadee91854840c72d03bfe22ef1b46c97cf6b4e3a3cdee3aa74876ec2a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-13T17:39:42.857 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:42 smithi044 podman[104492]: 2024-09-13 17:39:42.661345413 +0000 UTC m=+0.483151093 container attach f9d5aadee91854840c72d03bfe22ef1b46c97cf6b4e3a3cdee3aa74876ec2a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2) 2024-09-13T17:39:42.857 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 13 17:39:42 smithi044 podman[104492]: 2024-09-13 17:39:42.786178959 +0000 UTC m=+0.607984621 container died f9d5aadee91854840c72d03bfe22ef1b46c97cf6b4e3a3cdee3aa74876ec2a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-13T17:39:42.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:42 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:42.675+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:43.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:42.813+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:43.466 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:43 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:43.147+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:43.497 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.3.service' 2024-09-13T17:39:43.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:43.546 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-13T17:39:43.546 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-13T17:39:43.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.4 2024-09-13T17:39:43.774 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:43.335+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:43.880 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:43 smithi070 systemd[1]: Stopping Ceph osd.4 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:43.880 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:43 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:43.663+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:44.197 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:43 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4[45424]: 2024-09-13T17:39:43.878+0000 7fb310035640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:44.197 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:43 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4[45424]: 2024-09-13T17:39:43.878+0000 7fb310035640 -1 osd.4 109 *** Got signal Terminated *** 2024-09-13T17:39:44.197 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:43 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4[45424]: 2024-09-13T17:39:43.878+0000 7fb310035640 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:43.794+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:44.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:44 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:44.148+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:44.758 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:44 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:44.304+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:44.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:44 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:44.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:45.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:44 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:44.756+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:45.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:45 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:45.171+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:45.645 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:45 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:45.301+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:45.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:45 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:45.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:46.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:45 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:45.757+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:46.026 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:45 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:45.643+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:46.608 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:46 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:46.130+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:46.650 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:46 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:46.285+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:46.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:46 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:46.640+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:46.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:46 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:46.640+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:46.948 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:46 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:46.643+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:47.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:46 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:46.649+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:47.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:46 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:46.744+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:47.107 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:47.089+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:47.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:46 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:46.855+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:47.670 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:47 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:47.311+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:47.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:47 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:47.627+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:47.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:47 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:47.683+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:47.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:47 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:47.683+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:48.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:47 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:47.669+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:48.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:47 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:47.787+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:48.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:47 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:47.787+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:48.357 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:47 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:47.863+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:48.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:48 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:48.079+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:48.615 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:48 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4[45424]: 2024-09-13T17:39:48.340+0000 7fb308641640 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.294385+0000 front 2024-09-13T17:39:22.294470+0000 (oldest deadline 2024-09-13T17:39:48.194515+0000) 2024-09-13T17:39:48.675 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:48 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:48.300+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:48.675 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:48 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:48.300+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:48.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:48 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:48.613+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:48.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:48 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:48.651+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:48.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:48 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:48.651+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:49.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:48 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:48.673+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:49.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:48 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:48.834+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:49.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:48 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:48.834+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:49.107 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:49 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:49.041+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:49.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:48 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:48.837+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:49.521 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:49 smithi070 podman[70258]: 2024-09-13 17:39:49.219864184 +0000 UTC m=+5.462143922 container died 5a5040a244e5f8a2665684203466e799ec9adc53da1ec07e034d5294b9b8a384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-13T17:39:49.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:49.261+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:49.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:49.261+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:49.830 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:49 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:49.616+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:49.830 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:49 smithi070 podman[70258]: 2024-09-13 17:39:49.619458526 +0000 UTC m=+5.861738255 container cleanup 5a5040a244e5f8a2665684203466e799ec9adc53da1ec07e034d5294b9b8a384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-13T17:39:49.830 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:49 smithi070 bash[70258]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4 2024-09-13T17:39:49.830 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:49 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:49.665+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:49.830 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:49 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:49.665+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:50.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:49.664+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:50.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:49.862+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:50.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:49 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:49.862+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:50.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:49 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:49.819+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:50.107 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:50 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:50.066+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:50.107 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:50 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:50.066+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:50.197 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:49 smithi070 podman[70273]: 2024-09-13 17:39:49.828943517 +0000 UTC m=+0.602737167 container remove 5a5040a244e5f8a2665684203466e799ec9adc53da1ec07e034d5294b9b8a384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-13T17:39:50.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:50.240+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:50.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:50.240+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:50.525 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:50 smithi070 podman[70360]: 2024-09-13 17:39:50.137448412 +0000 UTC m=+0.016282976 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:39:50.525 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:50 smithi070 podman[70360]: 2024-09-13 17:39:50.246598353 +0000 UTC m=+0.125432954 container create 8c40d62dc56c5603aa1eb849d7be08315109d8192226b60fdc7b5cc2e408b08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:39:50.525 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:50 smithi070 podman[70360]: 2024-09-13 17:39:50.524430245 +0000 UTC m=+0.403264810 container init 8c40d62dc56c5603aa1eb849d7be08315109d8192226b60fdc7b5cc2e408b08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:39:50.947 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:50 smithi070 podman[70360]: 2024-09-13 17:39:50.527907906 +0000 UTC m=+0.406742459 container start 8c40d62dc56c5603aa1eb849d7be08315109d8192226b60fdc7b5cc2e408b08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2) 2024-09-13T17:39:50.948 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:50 smithi070 podman[70360]: 2024-09-13 17:39:50.624158517 +0000 UTC m=+0.502993087 container attach 8c40d62dc56c5603aa1eb849d7be08315109d8192226b60fdc7b5cc2e408b08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-13T17:39:50.948 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 13 17:39:50 smithi070 podman[70360]: 2024-09-13 17:39:50.712724708 +0000 UTC m=+0.591559261 container died 8c40d62dc56c5603aa1eb849d7be08315109d8192226b60fdc7b5cc2e408b08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-13T17:39:50.948 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:50 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:50.637+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:50.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:50 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:50.629+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:50.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:50 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:50.629+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:51.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:50.707+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:51.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:50.819+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:51.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:50 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:50.819+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:51.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.4.service' 2024-09-13T17:39:51.357 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:50 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:50.860+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:51.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:51 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:51.056+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:51.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:51 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:51.056+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:51.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:51.380 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-13T17:39:51.380 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-13T17:39:51.380 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.5 2024-09-13T17:39:51.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:51.258+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:51.525 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:51.258+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:51.525 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:51 smithi100 systemd[1]: Stopping Ceph osd.5 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:51.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:51.604+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:51.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:51.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:51.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:51 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:51.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:52.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:51.666+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:52.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:51.820+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:52.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:51.820+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:52.025 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:51.690+0000 7fb59ccd2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:52.025 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:51.690+0000 7fb59ccd2640 -1 osd.5 109 *** Got signal Terminated *** 2024-09-13T17:39:52.025 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:51 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:51.690+0000 7fb59ccd2640 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:52.356 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:51 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:51.902+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:52.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:52 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:52.102+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:52.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:52 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:52.102+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:52.620 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:52.290+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:52.620 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:52.290+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:52.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:52.561+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:52.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:52.603+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:52.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:52 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:52.603+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:53.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:52.618+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:53.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:52.805+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:53.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:52 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:52.805+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:53.357 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:52 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:52.941+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:53.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:53 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:53.144+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:53.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:53 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:53.144+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:53.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:53.254+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:53.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:53.254+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:53.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:53 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:53.527+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:53.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:53 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:53.563+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:53.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:53 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:53.563+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:54.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:53.576+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:54.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:53.767+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:54.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:53 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:53.767+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:54.180 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:53 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:53.897+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:54.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:54 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:54.231+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:54.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:54 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:54.231+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:54.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:54 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:54.179+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:54.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:54 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:54.179+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:54.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:54 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:54.179+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:39:54.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:54.489+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:54.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:54.550+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:54.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:54.550+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:54.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:54 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:54.550+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:39:55.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:54 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:54.801+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:55.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:54 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:54.801+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:55.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:54 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:54.587+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:55.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:54 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:54.587+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:39:55.198 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:54 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:54.885+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:55.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:55.262+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:55.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:55.262+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:55.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:55 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:55.196+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:55.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:55 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:55.196+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:55.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:55 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:55.196+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:39:55.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:55.487+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:55.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:55.592+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:55.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:55.592+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:55.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:55 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:55.592+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:39:56.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:55.573+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:56.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:55.573+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:39:56.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:55.803+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:56.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:55.803+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:56.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:55 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:55.803+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:39:56.227 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:55 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:55.871+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:56.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:56.244+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.920140+0000 front 2024-09-13T17:39:16.920318+0000 (oldest deadline 2024-09-13T17:39:39.820224+0000) 2024-09-13T17:39:56.524 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5[46671]: 2024-09-13T17:39:56.244+0000 7fb5992e6640 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.521365+0000 front 2024-09-13T17:39:24.521228+0000 (oldest deadline 2024-09-13T17:39:48.021292+0000) 2024-09-13T17:39:56.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:56.226+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:56.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:56.226+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:56.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:56.226+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:39:56.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:56 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:56.455+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:56.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:56 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:56.599+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:56.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:56 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:56.599+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:56.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:56 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:56.599+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:39:57.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:56.582+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:57.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:56.582+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:39:57.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:56.777+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:57.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:56.777+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:57.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:56 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:56.777+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:39:57.269 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:56.858+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:57.269 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:56.858+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:39:57.449 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:57 smithi100 podman[72826]: 2024-09-13 17:39:57.1013024 +0000 UTC m=+5.522557709 container died 37d5cc92d168869f936380ddc5414eb9ae355840915ecf0a0c7022ca0b93b914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-13T17:39:57.449 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:57 smithi100 podman[72826]: 2024-09-13 17:39:57.448489765 +0000 UTC m=+5.869745073 container cleanup 37d5cc92d168869f936380ddc5414eb9ae355840915ecf0a0c7022ca0b93b914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS) 2024-09-13T17:39:57.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:57 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:57.267+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:57.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:57 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:57.267+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:57.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:57 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:57.267+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:39:57.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:57 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:57.441+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:57.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:57 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:57.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:57.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:57 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:57.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:57.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:57 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:57.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:39:57.739 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:57 smithi100 bash[72826]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5 2024-09-13T17:39:57.739 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:57 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:57.611+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:57.739 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:57 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:57.611+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:39:58.024 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:57 smithi100 podman[72842]: 2024-09-13 17:39:57.775196212 +0000 UTC m=+0.672045376 container remove 37d5cc92d168869f936380ddc5414eb9ae355840915ecf0a0c7022ca0b93b914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-13T17:39:58.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:57 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:57.738+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:58.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:57 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:57.738+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:58.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:57 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:57.738+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:39:58.106 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:57 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:57.823+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:58.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:57 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:57.823+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:39:58.454 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:58 smithi100 podman[72928]: 2024-09-13 17:39:58.021233841 +0000 UTC m=+0.015616101 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:39:58.454 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:58 smithi100 podman[72928]: 2024-09-13 17:39:58.142516434 +0000 UTC m=+0.136898686 container create fb90ec2aa60598ab8cbfaf48e8fedb1a4bf98c5123fbfff6568cd101f20d4e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T17:39:58.454 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:58 smithi100 podman[72928]: 2024-09-13 17:39:58.361872809 +0000 UTC m=+0.356255063 container init fb90ec2aa60598ab8cbfaf48e8fedb1a4bf98c5123fbfff6568cd101f20d4e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:39:58.454 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:58 smithi100 podman[72928]: 2024-09-13 17:39:58.365019624 +0000 UTC m=+0.359401875 container start fb90ec2aa60598ab8cbfaf48e8fedb1a4bf98c5123fbfff6568cd101f20d4e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-13T17:39:58.454 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:58 smithi100 podman[72928]: 2024-09-13 17:39:58.453335569 +0000 UTC m=+0.447717821 container attach fb90ec2aa60598ab8cbfaf48e8fedb1a4bf98c5123fbfff6568cd101f20d4e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-13T17:39:58.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:58 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:58.250+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:58.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:58 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:58.250+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:58.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:58 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:58.250+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:39:58.774 INFO:journalctl@ceph.osd.5.smithi100.stdout:Sep 13 17:39:58 smithi100 podman[72928]: 2024-09-13 17:39:58.548015184 +0000 UTC m=+0.542397438 container died fb90ec2aa60598ab8cbfaf48e8fedb1a4bf98c5123fbfff6568cd101f20d4e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-13T17:39:58.774 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:58 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:58.579+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:39:58.774 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:58 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:58.579+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:39:58.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:58 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:58.471+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:58.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:58 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:58.471+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:39:58.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:58 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:58.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:58.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:58 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:58.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:58.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:58 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:58.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:39:59.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:58 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:58.818+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:59.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:58 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:58.818+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:39:59.150 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:58 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:58.784+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:39:59.150 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:58 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:58.784+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:39:59.150 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:58 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:58.784+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:39:59.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.5.service' 2024-09-13T17:39:59.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:39:59.212 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-13T17:39:59.213 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-13T17:39:59.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.6 2024-09-13T17:39:59.553 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:59 smithi044 systemd[1]: Stopping Ceph osd.6 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:39:59.553 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:59.250+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:39:59.553 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:59.250+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:39:59.553 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:39:59.250+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:39:59.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:59.427+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:39:59.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:39:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:39:59.427+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:39:59.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:59.579+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:39:59.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:59.579+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:39:59.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:39:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:39:59.579+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:39:59.818 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:59.550+0000 7f53fc2d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:39:59.819 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:59.550+0000 7f53fc2d4640 -1 osd.6 109 *** Got signal Terminated *** 2024-09-13T17:39:59.819 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:59.550+0000 7f53fc2d4640 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:39:59.819 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:59.816+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:39:59.819 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:39:59 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:39:59.816+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:40:00.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:59.627+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:00.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:39:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:39:59.627+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:00.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:59.829+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:00.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:59.829+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:00.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:39:59 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:39:59.829+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:00.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:00.254+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:00.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:00.254+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:00.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:00.254+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:00.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:00.460+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:00.947 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:00.460+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:00.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:00.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:00.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:00.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:00.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:00 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:00.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:01.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:00.652+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:01.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:00.652+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:01.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:00.831+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:01.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:00.831+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:01.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:00 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:00.831+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:01.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:00.816+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:40:01.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:00 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:00.816+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:40:01.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:01 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:01.256+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:01.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:01 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:01.256+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:01.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:01 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:01.256+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:01.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:01.424+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:01.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:01.424+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:01.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:01.567+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:01.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:01.567+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:01.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:01.567+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:02.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:01 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:01.626+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:02.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:01 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:01.626+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:02.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:01 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:01.842+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:02.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:01 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:01.842+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:02.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:01 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:01.842+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:02.106 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:01 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:01.795+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:40:02.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:01 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:01.795+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:40:02.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:01 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:01.795+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:36.139619+0000 front 2024-09-13T17:39:36.139540+0000 (oldest deadline 2024-09-13T17:40:01.439714+0000) 2024-09-13T17:40:02.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:02.268+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:02.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:02.268+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:02.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:02.268+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:02.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:02 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:02.392+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:02.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:02 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:02.392+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:02.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:02 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:02.566+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:02.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:02 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:02.566+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:02.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:02 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:02.566+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:02.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:02 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:02.566+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:03.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:02 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:02.630+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:03.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:02 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:02.630+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:03.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:02 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:02.831+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:03.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:02 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:02.831+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:03.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:02 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:02.831+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:03.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:02.760+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:40:03.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:02.760+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:40:03.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:02 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:02.760+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:36.139619+0000 front 2024-09-13T17:39:36.139540+0000 (oldest deadline 2024-09-13T17:40:01.439714+0000) 2024-09-13T17:40:03.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:03.247+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:03.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:03.247+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:03.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:03.247+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:03.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:03 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:03.364+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:03.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:03 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:03.364+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:03.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:03 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:03.549+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:03.697 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:03 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:03.549+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:03.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:03 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:03.549+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:03.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:03 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:03.549+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:04.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:03 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:03.679+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:04.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:03 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:03.679+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:04.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:03 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:03.874+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:04.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:03 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:03.874+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:04.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:03 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:03.874+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:04.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:03.749+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:22.638207+0000 front 2024-09-13T17:39:22.638258+0000 (oldest deadline 2024-09-13T17:39:46.738391+0000) 2024-09-13T17:40:04.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:03.749+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.839326+0000 front 2024-09-13T17:39:30.839361+0000 (oldest deadline 2024-09-13T17:39:56.139173+0000) 2024-09-13T17:40:04.107 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:03 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6[58818]: 2024-09-13T17:40:03.749+0000 7f53f88e8640 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:36.139619+0000 front 2024-09-13T17:39:36.139540+0000 (oldest deadline 2024-09-13T17:40:01.439714+0000) 2024-09-13T17:40:04.318 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:04.315+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:04.318 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:04.315+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:04.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:04.232+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:04.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:04.232+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:04.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:04 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:04.232+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:04.945 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:04 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:04.692+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:04.945 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:04 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:04.692+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:04.946 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:04 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:04.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:04.946 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:04 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:04.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:04.946 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:04 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:04.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:04.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:04.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:04.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:04.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:04.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:04.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:04.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:04.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:05.171 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:04 smithi044 podman[104594]: 2024-09-13 17:40:04.881482086 +0000 UTC m=+5.456050767 container died 543b5d1d4f06d101d1dcaef22b91f17dc3d3db125681868d3ca61363ae9444a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-13T17:40:05.530 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:05 smithi044 podman[104594]: 2024-09-13 17:40:05.268918417 +0000 UTC m=+5.843487100 container cleanup 543b5d1d4f06d101d1dcaef22b91f17dc3d3db125681868d3ca61363ae9444a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T17:40:05.530 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:05 smithi044 bash[104594]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6 2024-09-13T17:40:05.530 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:05.229+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:05.530 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:05.229+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:05.530 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:05.229+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:05.530 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:05 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:05.229+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:05.567 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:05.305+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:05.567 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:05.305+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:05.567 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:05.305+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:05.856 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:05 smithi044 podman[104610]: 2024-09-13 17:40:05.528422472 +0000 UTC m=+0.642131793 container remove 543b5d1d4f06d101d1dcaef22b91f17dc3d3db125681868d3ca61363ae9444a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T17:40:05.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:05.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:05.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:05.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:05.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:05.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:05.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:05 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:05.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:06.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:05 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:05.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:06.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:05 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:05.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:06.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:05 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:05.912+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:06.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:05 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:05.912+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:06.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:05 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:05.912+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:06.157 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:05 smithi044 podman[104701]: 2024-09-13 17:40:05.798519352 +0000 UTC m=+0.016263205 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:40:06.158 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:05 smithi044 podman[104701]: 2024-09-13 17:40:05.912758323 +0000 UTC m=+0.130502160 container create b8dcdb61af340aab7d5f7e5b6c53497c6fe6083ffe1ef9705ec05ee4100b22c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T17:40:06.158 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:06 smithi044 podman[104701]: 2024-09-13 17:40:06.156933389 +0000 UTC m=+0.374677234 container init b8dcdb61af340aab7d5f7e5b6c53497c6fe6083ffe1ef9705ec05ee4100b22c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:06.262+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:06.262+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:06.262+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:06 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:06.262+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:06 smithi044 podman[104701]: 2024-09-13 17:40:06.160452653 +0000 UTC m=+0.378196491 container start b8dcdb61af340aab7d5f7e5b6c53497c6fe6083ffe1ef9705ec05ee4100b22c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:06 smithi044 podman[104701]: 2024-09-13 17:40:06.240272935 +0000 UTC m=+0.458016798 container attach b8dcdb61af340aab7d5f7e5b6c53497c6fe6083ffe1ef9705ec05ee4100b22c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True) 2024-09-13T17:40:06.607 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 13 17:40:06 smithi044 podman[104701]: 2024-09-13 17:40:06.351667257 +0000 UTC m=+0.569411101 container died b8dcdb61af340aab7d5f7e5b6c53497c6fe6083ffe1ef9705ec05ee4100b22c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-13T17:40:06.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:06.317+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:06.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:06.317+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:06.697 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:06.317+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:06.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:06.561+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:06.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:06.561+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:06.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:06.561+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:06.698 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:06 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:06.561+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:06.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.6.service' 2024-09-13T17:40:06.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:06.977 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-13T17:40:06.977 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-13T17:40:06.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.7 2024-09-13T17:40:07.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:06.727+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:07.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:06.727+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:07.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:06.727+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:07.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:06.874+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:07.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:06.874+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:07.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:06 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:06.874+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:07.297 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 systemd[1]: Stopping Ceph osd.7 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:07.575 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:07.295+0000 7f490c6a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:07.575 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:07.295+0000 7f490c6a4640 -1 osd.7 109 *** Got signal Terminated *** 2024-09-13T17:40:07.575 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:07.295+0000 7f490c6a4640 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:40:07.575 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:07.298+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:07.575 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:07.298+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:07.575 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:07.298+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:07.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:07 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:07.296+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:07.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:07 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:07.296+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:07.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:07 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:07.296+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:07.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:07 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:07.296+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:07.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:07.573+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:07.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:07.573+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:07.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:07.573+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:07.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:07 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:07.573+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:08.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:07.695+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:08.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:07.695+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:08.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:07.695+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:08.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:07.830+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:08.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:07.830+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:08.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:07 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:07.830+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:08.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:08 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:08.284+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:08.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:08 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:08.284+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:08.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:08 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:08.284+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:08.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:08 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:08.284+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:08.619 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:08.254+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:08.619 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:08.254+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:08.619 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:08.254+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:08.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:08.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:08.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:08.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:08.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:08.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:08.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:08 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:08.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:09.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:08.704+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:09.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:08.704+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:09.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:08.704+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:09.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:08.785+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:09.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:08.785+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:09.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:08 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:08.785+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:09.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:09 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:09.279+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:09.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:09 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:09.279+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:09.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:09 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:09.279+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:09.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:09 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:09.279+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:09.664 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:09.213+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:09.664 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:09.213+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:09.664 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:09.213+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:09.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:09.662+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:09.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:09.662+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:09.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:09.662+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:09.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:09.662+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:09.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:09.662+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:10.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:09.739+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:10.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:09.739+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:10.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:09.739+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:10.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:09.826+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:10.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:09.826+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:10.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:09.826+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:10.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:09 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:09.826+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:10.447 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:10.188+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:10.447 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:10.188+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:10.447 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:10.188+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:10.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:10 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:10.291+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:10.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:10 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:10.291+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:10.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:10 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:10.291+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:10.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:10 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:10.291+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:10.715 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:10.676+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:10.716 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:10.676+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:10.716 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:10.676+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:10.716 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:10.676+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:10.716 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:10 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:10.676+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:11.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:10.715+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:11.026 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:10.715+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:11.026 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:10.715+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:11.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:10.843+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:11.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:10.843+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:11.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:10.843+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:11.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:10 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:10.843+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:11.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:11 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:11.312+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:11.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:11 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:11.312+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:11.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:11 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:11.312+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:11.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:11 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:11.312+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:11.650 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:11.197+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:11.650 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:11.197+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:11.650 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:11.197+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:11.650 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:11.197+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.864718+0000 front 2024-09-13T17:39:48.864780+0000 (oldest deadline 2024-09-13T17:40:10.564805+0000) 2024-09-13T17:40:11.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:11.648+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:11.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:11.648+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:11.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:11.648+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:11.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:11.648+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:11.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:11 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:11.648+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:12.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:11.703+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:12.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:11.703+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:12.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:11.703+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:12.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:11.839+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:12.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:11.839+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:12.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:11.839+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:12.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:11 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:11.839+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:12.449 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:12.180+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.261782+0000 front 2024-09-13T17:39:24.261803+0000 (oldest deadline 2024-09-13T17:39:46.561988+0000) 2024-09-13T17:40:12.449 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:12.180+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:32.463296+0000 front 2024-09-13T17:39:32.462997+0000 (oldest deadline 2024-09-13T17:39:58.362912+0000) 2024-09-13T17:40:12.449 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:12.180+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.864020+0000 front 2024-09-13T17:39:38.863929+0000 (oldest deadline 2024-09-13T17:40:04.763849+0000) 2024-09-13T17:40:12.449 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7[52335]: 2024-09-13T17:40:12.180+0000 7f4904cb0640 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.864718+0000 front 2024-09-13T17:39:48.864780+0000 (oldest deadline 2024-09-13T17:40:10.564805+0000) 2024-09-13T17:40:12.608 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:12.269+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:12.608 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:12.269+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:12.609 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:12.269+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:12.609 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:12 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:12.269+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:12.948 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:12 smithi070 podman[70464]: 2024-09-13 17:40:12.779138277 +0000 UTC m=+5.608851544 container died 31d5bbd383f5f7349bc3fe1cad946319e65678709e322b03df849647d3db449d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:40:12.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:12.608+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:12.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:12.608+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:12.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:12.608+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:12.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:12.608+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:12.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:12 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:12.608+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:13.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:12.694+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:13.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:12.694+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:13.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:12.694+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:13.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:12.875+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:13.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:12.875+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:13.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:12.875+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:13.026 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:12.875+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:13.447 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:13 smithi070 podman[70464]: 2024-09-13 17:40:13.128536746 +0000 UTC m=+5.958250013 container cleanup 31d5bbd383f5f7349bc3fe1cad946319e65678709e322b03df849647d3db449d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:13.447 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:13 smithi070 bash[70464]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7 2024-09-13T17:40:13.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:13.300+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:13.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:13.300+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:13.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:13.300+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:13.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:13 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:13.300+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:13.899 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:13 smithi070 podman[70491]: 2024-09-13 17:40:13.488335431 +0000 UTC m=+0.705581533 container remove 31d5bbd383f5f7349bc3fe1cad946319e65678709e322b03df849647d3db449d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:13.900 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:13 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:13.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:13.900 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:13 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:13.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:13.900 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:13 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:13.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:13.900 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:13 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:13.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:13.900 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:13 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:13.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:14.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:13.698+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:14.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:13.698+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:14.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:13.698+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:14.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:13.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:14.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:13.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:14.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:13.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:14.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:13 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:13.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:14.197 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:13 smithi070 podman[70580]: 2024-09-13 17:40:13.801357626 +0000 UTC m=+0.015659532 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:40:14.197 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:13 smithi070 podman[70580]: 2024-09-13 17:40:13.922094112 +0000 UTC m=+0.136396012 container create 5cac9b6ab0c4a9415967a05d4002ebe7758d2f1e8a52d66893f952944dc9ad14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:40:14.599 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:14 smithi070 podman[70580]: 2024-09-13 17:40:14.199366332 +0000 UTC m=+0.413668232 container init 5cac9b6ab0c4a9415967a05d4002ebe7758d2f1e8a52d66893f952944dc9ad14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:14.599 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:14 smithi070 podman[70580]: 2024-09-13 17:40:14.202537648 +0000 UTC m=+0.416839548 container start 5cac9b6ab0c4a9415967a05d4002ebe7758d2f1e8a52d66893f952944dc9ad14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-13T17:40:14.599 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:14 smithi070 podman[70580]: 2024-09-13 17:40:14.307807671 +0000 UTC m=+0.522109586 container attach 5cac9b6ab0c4a9415967a05d4002ebe7758d2f1e8a52d66893f952944dc9ad14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-13T17:40:14.599 INFO:journalctl@ceph.osd.7.smithi070.stdout:Sep 13 17:40:14 smithi070 podman[70580]: 2024-09-13 17:40:14.386872558 +0000 UTC m=+0.601174458 container died 5cac9b6ab0c4a9415967a05d4002ebe7758d2f1e8a52d66893f952944dc9ad14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T17:40:14.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:14.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:14.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:14.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:14.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:14.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:14.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:14.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:14.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:14 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:14.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:14.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:14 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:14.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:14.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:14 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:14.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:14.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:14 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:14.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:14.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:14 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:14.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:14.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:14 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:14.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:15.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:14.666+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:14.666+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:14.666+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:14.666+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.801567+0000 front 2024-09-13T17:39:47.801731+0000 (oldest deadline 2024-09-13T17:40:13.701607+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:14.883+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:14.883+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:14.883+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:15.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:14 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:14.883+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:15.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.7.service' 2024-09-13T17:40:15.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:15.102 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-13T17:40:15.102 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-13T17:40:15.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.8 2024-09-13T17:40:15.402 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 systemd[1]: Stopping Ceph osd.8 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:15.675 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.399+0000 7f3dd99f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:15.675 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.399+0000 7f3dd99f2640 -1 osd.8 109 *** Got signal Terminated *** 2024-09-13T17:40:15.675 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.399+0000 7f3dd99f2640 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:40:15.675 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.672+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:15.676 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.672+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:15.676 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.672+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:15.676 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:15.672+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.801567+0000 front 2024-09-13T17:39:47.801731+0000 (oldest deadline 2024-09-13T17:40:13.701607+0000) 2024-09-13T17:40:15.857 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:15.367+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:15.857 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:15.367+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:15.857 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:15.367+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:15.857 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:15.367+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:15.857 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:15 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:15.367+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:15.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:15 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:15.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:15.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:15 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:15.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:15.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:15 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:15.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:15.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:15 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:15.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:15.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:15 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:15.614+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:16.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:15.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:16.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:15.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:16.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:15.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:16.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:15 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:15.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:16.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:16.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:16.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:16.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:16.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:16.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:16.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:16.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:16.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:16 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:16.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:16.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:16 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:16.589+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:16.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:16 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:16.589+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:16.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:16 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:16.589+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:16.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:16 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:16.589+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:16.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:16 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:16.589+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:17.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:16.680+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:17.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:16.680+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:17.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:16.680+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:17.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:16.680+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.801567+0000 front 2024-09-13T17:39:47.801731+0000 (oldest deadline 2024-09-13T17:40:13.701607+0000) 2024-09-13T17:40:17.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:16.853+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:17.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:16.853+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:17.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:16.853+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:17.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:16 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:16.853+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:17.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:17 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:17.311+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:17.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:17 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:17.311+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:17.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:17 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:17.311+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:17.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:17 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:17.311+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:17.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:17 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:17.311+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:17.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:17.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:17.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:17.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:17.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:17.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:17.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:17.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:17.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:17.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:17.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:17 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:17.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:18.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:17.683+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:18.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:17.683+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:18.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:17.683+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:18.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:17.683+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.801567+0000 front 2024-09-13T17:39:47.801731+0000 (oldest deadline 2024-09-13T17:40:13.701607+0000) 2024-09-13T17:40:18.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:17.816+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:18.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:17.816+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:18.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:17.816+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:18.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:17.816+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:18.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:17.816+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:18.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:18 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:18.293+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:18.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:18 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:18.293+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:18.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:18 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:18.293+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:18.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:18 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:18.293+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:18.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:18 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:18.293+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:18.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:18 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:18.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:18.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:18 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:18.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:18.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:18 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:18.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:18.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:18 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:18.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:18.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:18 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:18.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:18.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:18 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:18.547+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:19.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:18.732+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:19.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:18.732+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:19.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:18.732+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:19.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:18.732+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.801567+0000 front 2024-09-13T17:39:47.801731+0000 (oldest deadline 2024-09-13T17:40:13.701607+0000) 2024-09-13T17:40:19.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:18.793+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:19.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:18.793+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:19.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:18.793+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:19.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:18.793+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:19.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:18 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:18.793+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:19.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:19 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:19.283+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:19.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:19 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:19.283+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:19.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:19 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:19.283+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:19.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:19 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:19.283+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:19.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:19 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:19.283+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:19.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:19 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:19.283+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:19.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:19 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:19.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:19.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:19 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:19.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:19.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:19 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:19.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:19.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:19 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:19.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:19.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:19 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:19.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:19.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:19 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:19.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:20.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:19.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:23.298024+0000 front 2024-09-13T17:39:23.298079+0000 (oldest deadline 2024-09-13T17:39:45.598093+0000) 2024-09-13T17:40:20.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:19.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.899110+0000 front 2024-09-13T17:39:30.899085+0000 (oldest deadline 2024-09-13T17:39:54.299601+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:19.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:40.200501+0000 front 2024-09-13T17:39:40.200654+0000 (oldest deadline 2024-09-13T17:40:06.100460+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:19.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.801567+0000 front 2024-09-13T17:39:47.801731+0000 (oldest deadline 2024-09-13T17:40:13.701607+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8[53542]: 2024-09-13T17:40:19.709+0000 7f3dd6006640 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:53.701976+0000 front 2024-09-13T17:39:53.701999+0000 (oldest deadline 2024-09-13T17:40:19.602166+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:19.748+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:19.748+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:19.748+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:19.748+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:20.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:19 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:19.748+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:20.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:20.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:20.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:20.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:20.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:20.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:20.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:20.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:20.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:20.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:20.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:20 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:20.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:20.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:20.591+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:20.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:20.591+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:20.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:20.591+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:20.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:20.591+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:20.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:20.591+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:20.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:20 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:20.591+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:21.024 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:20 smithi100 podman[73045]: 2024-09-13 17:40:20.925393067 +0000 UTC m=+5.626646829 container died e6abddf68c7fa31174e31b7ad561f770199e6b5dcaecaf7a270c752a1f87b316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-13T17:40:21.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:20 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:20.729+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:21.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:20 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:20.729+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:21.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:20 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:20.729+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:21.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:20 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:20.729+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:21.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:20 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:20.729+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:21.524 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:21 smithi100 podman[73045]: 2024-09-13 17:40:21.275748845 +0000 UTC m=+5.977002600 container cleanup e6abddf68c7fa31174e31b7ad561f770199e6b5dcaecaf7a270c752a1f87b316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-13T17:40:21.524 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:21 smithi100 bash[73045]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8 2024-09-13T17:40:21.525 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:21 smithi100 podman[73062]: 2024-09-13 17:40:21.418582162 +0000 UTC m=+0.486919769 container remove e6abddf68c7fa31174e31b7ad561f770199e6b5dcaecaf7a270c752a1f87b316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True) 2024-09-13T17:40:21.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:21 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:21.330+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:21.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:21 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:21.330+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:21.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:21 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:21.330+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:21.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:21 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:21.330+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:21.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:21 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:21.330+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:21.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:21 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:21.330+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:21.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:21 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:21.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:21.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:21 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:21.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:21.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:21 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:21.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:21.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:21 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:21.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:21.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:21 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:21.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:21.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:21 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:21.565+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:21.952 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:21 smithi100 podman[73151]: 2024-09-13 17:40:21.717282845 +0000 UTC m=+0.015493909 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:40:21.952 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:21 smithi100 podman[73151]: 2024-09-13 17:40:21.836383307 +0000 UTC m=+0.134594430 container create 6245128a4c9a8cfdd332d0a4636868e5b46edf7418ee515bf7c1e750c28e3366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:40:21.952 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:21.688+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:21.952 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:21.688+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:21.952 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:21.688+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:21.953 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:21.688+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:21.953 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:21.688+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:22.257 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:22 smithi100 podman[73151]: 2024-09-13 17:40:22.063918553 +0000 UTC m=+0.362129615 container init 6245128a4c9a8cfdd332d0a4636868e5b46edf7418ee515bf7c1e750c28e3366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-13T17:40:22.257 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:22 smithi100 podman[73151]: 2024-09-13 17:40:22.067038653 +0000 UTC m=+0.365249721 container start 6245128a4c9a8cfdd332d0a4636868e5b46edf7418ee515bf7c1e750c28e3366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T17:40:22.257 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:22 smithi100 podman[73151]: 2024-09-13 17:40:22.138701754 +0000 UTC m=+0.436912842 container attach 6245128a4c9a8cfdd332d0a4636868e5b46edf7418ee515bf7c1e750c28e3366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:22.524 INFO:journalctl@ceph.osd.8.smithi100.stdout:Sep 13 17:40:22 smithi100 podman[73151]: 2024-09-13 17:40:22.2563248 +0000 UTC m=+0.554535869 container died 6245128a4c9a8cfdd332d0a4636868e5b46edf7418ee515bf7c1e750c28e3366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:22.606 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:22.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:22.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:22.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:22.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:22.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:22.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:22.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:22.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:22.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:22.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:22.290+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:22.835 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:22 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:22.681+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:22.835 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:22 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:22.681+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:22.835 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:22 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:22.681+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:22.835 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:22 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:22.681+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:22.836 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:22 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:22.681+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:22.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.8.service' 2024-09-13T17:40:22.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:22.898 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-13T17:40:22.898 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-13T17:40:22.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.9 2024-09-13T17:40:22.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:22 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:22.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:22.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:22 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:22.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:22.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:22 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:22.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:22.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:22 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:22.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:22.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:22 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:22.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:22.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:22 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:22.597+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:23.215 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:22 smithi044 systemd[1]: Stopping Ceph osd.9 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.213+0000 7fb91afe9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.213+0000 7fb91afe9640 -1 osd.9 109 *** Got signal Terminated *** 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.213+0000 7fb91afe9640 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.308+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.308+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.308+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.308+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.308+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:23.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:23 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:23.308+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:23.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:23 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:23.623+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:23.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:23 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:23.623+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:23.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:23 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:23.623+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:23.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:23 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:23.623+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:23.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:23 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:23.623+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:23.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:23 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:23.623+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:23.960 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:23 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:23.707+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:23.960 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:23 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:23.707+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:23.960 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:23 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:23.707+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:23.960 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:23 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:23.707+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:23.960 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:23 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:23.707+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:24.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:24 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:24.339+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:24.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:24 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:24.339+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:24.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:24 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:24.339+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:24.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:24 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:24.339+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:24.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:24 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:24.339+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:24.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:24 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:24.339+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:24.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:24 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:24.632+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:24.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:24 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:24.632+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:24.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:24 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:24.632+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:24.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:24 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:24.632+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:24.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:24 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:24.632+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:24.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:24 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:24.633+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:25.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:24 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:24.713+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:25.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:24 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:24.713+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:25.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:24 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:24.713+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:25.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:24 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:24.713+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:25.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:24 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:24.713+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:25.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:25 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:25.294+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:25.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:25 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:25.294+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:25.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:25 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:25.294+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:25.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:25 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:25.294+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:25.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:25 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:25.294+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:25.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:25 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:25.294+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:25.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:25 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:25.658+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:25.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:25 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:25.658+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:25.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:25 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:25.658+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:25.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:25 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:25.658+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:25.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:25 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:25.658+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:25.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:25 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:25.658+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:26.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:25.665+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:26.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:25.665+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:26.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:25.665+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:26.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:25.665+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:26.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:25.665+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:26.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:26 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:26.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:26.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:26 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:26.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:26.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:26 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:26.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:26.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:26 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:26.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:26.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:26 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:26.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:26.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:26 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:26.340+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:26.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:26 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:26.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:26.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:26 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:26.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:26.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:26 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:26.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:26.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:26 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:26.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:26.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:26 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:26.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:26.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:26 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:26.618+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:27.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:26 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:26.700+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:27.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:26 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:26.700+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:27.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:26 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:26.700+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:27.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:26 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:26.700+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:27.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:26 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:26.700+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:27.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:26 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:26.700+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:27.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:27 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:27.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.942112+0000 front 2024-09-13T17:39:14.942150+0000 (oldest deadline 2024-09-13T17:39:40.842337+0000) 2024-09-13T17:40:27.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:27 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:27.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:24.144555+0000 front 2024-09-13T17:39:24.144540+0000 (oldest deadline 2024-09-13T17:39:50.044296+0000) 2024-09-13T17:40:27.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:27 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:27.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.044677+0000 front 2024-09-13T17:39:30.044695+0000 (oldest deadline 2024-09-13T17:39:53.544757+0000) 2024-09-13T17:40:27.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:27 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:27.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:38.845589+0000 front 2024-09-13T17:39:38.845566+0000 (oldest deadline 2024-09-13T17:40:04.745735+0000) 2024-09-13T17:40:27.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:27 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:27.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:48.147622+0000 front 2024-09-13T17:39:48.147518+0000 (oldest deadline 2024-09-13T17:40:14.047449+0000) 2024-09-13T17:40:27.607 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:27 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9[65490]: 2024-09-13T17:40:27.335+0000 7fb9175fd640 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.148911+0000 front 2024-09-13T17:39:55.148779+0000 (oldest deadline 2024-09-13T17:40:18.549114+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:27.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:27 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:27.611+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:28.024 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:27 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:27.747+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:28.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:27 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:27.747+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:28.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:27 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:27.747+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:28.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:27 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:27.747+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:28.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:27 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:27.747+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:28.025 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:27 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:27.747+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:28.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:28.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:28.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:28.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:28.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:28.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:28.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:28 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:28.582+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:28.990 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:28 smithi044 podman[104815]: 2024-09-13 17:40:28.62077831 +0000 UTC m=+5.515148824 container died 97a6d2ae3fa968e477dbe864121324208d28416606cd8917f620bb2ed47ed074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-13T17:40:29.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:28.791+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:29.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:28.791+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:29.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:28.791+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:29.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:28.791+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:29.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:28.791+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:29.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:28 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:28.791+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:29.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:29 smithi044 podman[104815]: 2024-09-13 17:40:29.087775182 +0000 UTC m=+5.982145678 container cleanup 97a6d2ae3fa968e477dbe864121324208d28416606cd8917f620bb2ed47ed074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T17:40:29.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:29 smithi044 bash[104815]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9 2024-09-13T17:40:29.765 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:29 smithi044 podman[104831]: 2024-09-13 17:40:29.405886989 +0000 UTC m=+0.779863375 container remove 97a6d2ae3fa968e477dbe864121324208d28416606cd8917f620bb2ed47ed074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:40:29.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:29.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:29.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:29.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:29.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:29.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:29.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:29 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:29.621+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:30.094 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:29 smithi044 podman[104922]: 2024-09-13 17:40:29.667412208 +0000 UTC m=+0.016115715 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:40:30.094 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:29 smithi044 podman[104922]: 2024-09-13 17:40:29.773476975 +0000 UTC m=+0.122180475 container create 44981327a156d520355fa44a56eac48061cc3b9637aa2279d56b28e0a8f0eec8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:30.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:29 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:29.789+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:30.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:29 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:29.789+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:30.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:29 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:29.789+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:30.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:29 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:29.789+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:30.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:29 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:29.789+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:30.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:29 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:29.789+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:30.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:30 smithi044 podman[104922]: 2024-09-13 17:40:30.092944194 +0000 UTC m=+0.441647699 container init 44981327a156d520355fa44a56eac48061cc3b9637aa2279d56b28e0a8f0eec8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:40:30.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:30 smithi044 podman[104922]: 2024-09-13 17:40:30.097104673 +0000 UTC m=+0.445808174 container start 44981327a156d520355fa44a56eac48061cc3b9637aa2279d56b28e0a8f0eec8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-13T17:40:30.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:30 smithi044 podman[104922]: 2024-09-13 17:40:30.159203171 +0000 UTC m=+0.507906688 container attach 44981327a156d520355fa44a56eac48061cc3b9637aa2279d56b28e0a8f0eec8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T17:40:30.357 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 13 17:40:30 smithi044 podman[104922]: 2024-09-13 17:40:30.293650777 +0000 UTC m=+0.642354284 container died 44981327a156d520355fa44a56eac48061cc3b9637aa2279d56b28e0a8f0eec8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-13T17:40:30.931 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.9.service' 2024-09-13T17:40:30.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:30.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:30.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:30.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:30.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:30.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:30.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:30 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:30.616+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:30.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:30.970 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-13T17:40:30.970 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-13T17:40:30.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.10 2024-09-13T17:40:31.272 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 systemd[1]: Stopping Ceph osd.10 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:31.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:30 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:30.796+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:31.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:30 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:30.796+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:31.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:30 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:30.796+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:31.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:30 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:30.796+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:31.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:30 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:30.796+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:31.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:30 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:30.796+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:31.627 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.270+0000 7f169bdba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.270+0000 7f169bdba640 -1 osd.10 109 *** Got signal Terminated *** 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.270+0000 7f169bdba640 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:31.628 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:31.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:31.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:31 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:31.625+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:32.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:31 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:31.814+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:32.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:31 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:31.814+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:32.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:31 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:31.814+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:32.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:31 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:31.814+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:32.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:31 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:31.814+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:32.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:31 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:31.814+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:32.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:32.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:32.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:32.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:32.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:32.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:32.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:32 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:32.587+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:33.177 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:32.851+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:33.177 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:32.851+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:33.177 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:32.851+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:33.177 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:32.851+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:33.177 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:32.851+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:33.177 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:32 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:32.851+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:33.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:33.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:33.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:33.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:33.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:33.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:33.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:33 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:33.538+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:34.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:33 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:33.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:34.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:33 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:33.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:34.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:33 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:33.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:34.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:33 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:33.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:34.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:33 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:33.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:34.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:33 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:33.868+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:34.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:34.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:34.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:34.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:34.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:34.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:34.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:34 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:34.562+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:35.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:34 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:34.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:35.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:34 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:34.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:35.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:34 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:34.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:35.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:34 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:34.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:35.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:34 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:34.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:35.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:34 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:34.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:35.947 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:14.865000+0000 front 2024-09-13T17:39:14.864951+0000 (oldest deadline 2024-09-13T17:39:40.164977+0000) 2024-09-13T17:40:35.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:20.165296+0000 front 2024-09-13T17:39:20.165276+0000 (oldest deadline 2024-09-13T17:39:46.065497+0000) 2024-09-13T17:40:35.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:28.266762+0000 front 2024-09-13T17:39:28.266774+0000 (oldest deadline 2024-09-13T17:39:54.166918+0000) 2024-09-13T17:40:35.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-09-13T17:39:39.368591+0000 front 2024-09-13T17:39:39.368389+0000 (oldest deadline 2024-09-13T17:40:01.668337+0000) 2024-09-13T17:40:35.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:45.769163+0000 front 2024-09-13T17:39:45.769211+0000 (oldest deadline 2024-09-13T17:40:09.269359+0000) 2024-09-13T17:40:35.948 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.670827+0000 front 2024-09-13T17:39:55.670666+0000 (oldest deadline 2024-09-13T17:40:17.370552+0000) 2024-09-13T17:40:35.949 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:35 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10[58826]: 2024-09-13T17:40:35.518+0000 7f16943c6640 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:03.473357+0000 front 2024-09-13T17:40:03.473317+0000 (oldest deadline 2024-09-13T17:40:26.873766+0000) 2024-09-13T17:40:36.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:35 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:35.871+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:36.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:35 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:35.871+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:36.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:35 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:35.871+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:36.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:35 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:35.871+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:36.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:35 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:35.871+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:36.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:35 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:35.871+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:36.863 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:36 smithi070 podman[70684]: 2024-09-13 17:40:36.590093514 +0000 UTC m=+5.417897093 container died eacdb723fa3c5b5b91468d555db93b3c3191db44630941c9bd2fa23694e88e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T17:40:37.197 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:36 smithi070 podman[70684]: 2024-09-13 17:40:36.944342339 +0000 UTC m=+5.772145926 container cleanup eacdb723fa3c5b5b91468d555db93b3c3191db44630941c9bd2fa23694e88e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-13T17:40:37.197 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:36 smithi070 bash[70684]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10 2024-09-13T17:40:37.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:36 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:36.893+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:37.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:36 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:36.893+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:37.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:36 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:36.893+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:37.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:36 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:36.893+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:37.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:36 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:36.893+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:37.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:36 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:36.893+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:37.550 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70701]: 2024-09-13 17:40:37.203832614 +0000 UTC m=+0.614804955 container remove eacdb723fa3c5b5b91468d555db93b3c3191db44630941c9bd2fa23694e88e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:40:37.808 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70789]: 2024-09-13 17:40:37.452247853 +0000 UTC m=+0.015555893 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:40:37.808 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70789]: 2024-09-13 17:40:37.571560481 +0000 UTC m=+0.134868549 container create 8bd550db8d7d1e3db4910552eb270d71a6b006eba7322fb337937cb3624afbfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T17:40:38.197 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70789]: 2024-09-13 17:40:37.807482451 +0000 UTC m=+0.370790487 container init 8bd550db8d7d1e3db4910552eb270d71a6b006eba7322fb337937cb3624afbfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-13T17:40:38.197 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70789]: 2024-09-13 17:40:37.810892614 +0000 UTC m=+0.374200645 container start 8bd550db8d7d1e3db4910552eb270d71a6b006eba7322fb337937cb3624afbfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-09-13T17:40:38.197 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70789]: 2024-09-13 17:40:37.890277482 +0000 UTC m=+0.453585509 container attach 8bd550db8d7d1e3db4910552eb270d71a6b006eba7322fb337937cb3624afbfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:40:38.197 INFO:journalctl@ceph.osd.10.smithi070.stdout:Sep 13 17:40:37 smithi070 podman[70789]: 2024-09-13 17:40:37.995225746 +0000 UTC m=+0.558533783 container died 8bd550db8d7d1e3db4910552eb270d71a6b006eba7322fb337937cb3624afbfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-13T17:40:38.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:37.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:38.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:37.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:38.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:37.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:38.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:37.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:38.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:37.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:38.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:37 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:37.879+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:38.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.10.service' 2024-09-13T17:40:38.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:38.631 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-13T17:40:38.631 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-13T17:40:38.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.11 2024-09-13T17:40:38.937 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 systemd[1]: Stopping Ceph osd.11 for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.899+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2024-09-13T17:40:12.186493+0000 front 2024-09-13T17:40:12.186570+0000 (oldest deadline 2024-09-13T17:40:38.086368+0000) 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.936+0000 7fd6768d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:38.938 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.936+0000 7fd6768d5640 -1 osd.11 109 *** Got signal Terminated *** 2024-09-13T17:40:38.939 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:38 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:38.936+0000 7fd6768d5640 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T17:40:40.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:40.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:40.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:40.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:40.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:40.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:40.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:39 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:39.877+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2024-09-13T17:40:12.186493+0000 front 2024-09-13T17:40:12.186570+0000 (oldest deadline 2024-09-13T17:40:38.086368+0000) 2024-09-13T17:40:41.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:41.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:41.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:41.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:41.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:41.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:41.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:40 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:40.901+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2024-09-13T17:40:12.186493+0000 front 2024-09-13T17:40:12.186570+0000 (oldest deadline 2024-09-13T17:40:38.086368+0000) 2024-09-13T17:40:42.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:42.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:42.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:42.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:42.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:42.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:42.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:41 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:41.931+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2024-09-13T17:40:12.186493+0000 front 2024-09-13T17:40:12.186570+0000 (oldest deadline 2024-09-13T17:40:38.086368+0000) 2024-09-13T17:40:43.234 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:43.234 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:43.234 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:43.235 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:43.235 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:43.235 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:43.235 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:42 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:42.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2024-09-13T17:40:12.186493+0000 front 2024-09-13T17:40:12.186570+0000 (oldest deadline 2024-09-13T17:40:38.086368+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-09-13T17:39:16.974059+0000 front 2024-09-13T17:39:16.974197+0000 (oldest deadline 2024-09-13T17:39:41.074264+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2024-09-13T17:39:25.075788+0000 front 2024-09-13T17:39:25.075817+0000 (oldest deadline 2024-09-13T17:39:46.775869+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6804 osd.2 since back 2024-09-13T17:39:30.777150+0000 front 2024-09-13T17:39:30.777208+0000 (oldest deadline 2024-09-13T17:39:54.877303+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2024-09-13T17:39:47.079725+0000 front 2024-09-13T17:39:47.079593+0000 (oldest deadline 2024-09-13T17:40:09.379570+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6812 osd.5 since back 2024-09-13T17:39:55.081579+0000 front 2024-09-13T17:39:55.081526+0000 (oldest deadline 2024-09-13T17:40:17.282152+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-09-13T17:40:02.384365+0000 front 2024-09-13T17:40:02.384277+0000 (oldest deadline 2024-09-13T17:40:26.384744+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2024-09-13T17:40:12.186493+0000 front 2024-09-13T17:40:12.186570+0000 (oldest deadline 2024-09-13T17:40:38.086368+0000) 2024-09-13T17:40:44.275 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:43 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11[60347]: 2024-09-13T17:40:43.885+0000 7fd672ee9640 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.100:6820 osd.8 since back 2024-09-13T17:40:19.787262+0000 front 2024-09-13T17:40:19.787220+0000 (oldest deadline 2024-09-13T17:40:43.187950+0000) 2024-09-13T17:40:44.612 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:44 smithi100 podman[73265]: 2024-09-13 17:40:44.346667201 +0000 UTC m=+5.529097331 container died b9233b7d026a8f18eb12f0c036979fec8b103e6945755117224c2babaefb630e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-13T17:40:44.962 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:44 smithi100 podman[73265]: 2024-09-13 17:40:44.702452486 +0000 UTC m=+5.884882616 container cleanup b9233b7d026a8f18eb12f0c036979fec8b103e6945755117224c2babaefb630e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-13T17:40:44.963 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:44 smithi100 bash[73265]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11 2024-09-13T17:40:45.274 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:44 smithi100 podman[73281]: 2024-09-13 17:40:44.961952489 +0000 UTC m=+0.604897098 container remove b9233b7d026a8f18eb12f0c036979fec8b103e6945755117224c2babaefb630e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-13T17:40:45.624 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:45 smithi100 podman[73370]: 2024-09-13 17:40:45.312714456 +0000 UTC m=+0.103276106 container create b8ddec03a940901a215c82a6a379b8291c5319540fc8fb6bb7cbe88ab06c9155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-13T17:40:45.624 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:45 smithi100 podman[73370]: 2024-09-13 17:40:45.2254666 +0000 UTC m=+0.016028249 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T17:40:45.624 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:45 smithi100 podman[73370]: 2024-09-13 17:40:45.523224011 +0000 UTC m=+0.313785660 container init b8ddec03a940901a215c82a6a379b8291c5319540fc8fb6bb7cbe88ab06c9155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-13T17:40:45.624 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:45 smithi100 podman[73370]: 2024-09-13 17:40:45.527102075 +0000 UTC m=+0.317663725 container start b8ddec03a940901a215c82a6a379b8291c5319540fc8fb6bb7cbe88ab06c9155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T17:40:45.624 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:45 smithi100 podman[73370]: 2024-09-13 17:40:45.623250633 +0000 UTC m=+0.413812292 container attach b8ddec03a940901a215c82a6a379b8291c5319540fc8fb6bb7cbe88ab06c9155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:40:45.996 INFO:journalctl@ceph.osd.11.smithi100.stdout:Sep 13 17:40:45 smithi100 podman[73370]: 2024-09-13 17:40:45.713949984 +0000 UTC m=+0.504511639 container died b8ddec03a940901a215c82a6a379b8291c5319540fc8fb6bb7cbe88ab06c9155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-13T17:40:46.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@osd.11.service' 2024-09-13T17:40:46.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:46.314 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-13T17:40:46.314 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-13T17:40:46.315 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.a 2024-09-13T17:40:46.607 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:40:46 smithi044 systemd[1]: Stopping Ceph mds.a for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:47.106 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:40:46 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a[72346]: 2024-09-13T17:40:46.615+0000 7fdae5ec0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:47.107 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:40:46 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a[72346]: 2024-09-13T17:40:46.615+0000 7fdae5ec0640 -1 mds.a *** got signal Terminated *** 2024-09-13T17:40:50.575 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:40:50 smithi044 podman[105041]: 2024-09-13 17:40:50.301768511 +0000 UTC m=+3.789446197 container died 5c56a401760415499910d78f9320cc1e8be3ec4aa33e1e64deacabd1691f2206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:40:50.857 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:40:50 smithi044 podman[105041]: 2024-09-13 17:40:50.673600948 +0000 UTC m=+4.161278633 container cleanup 5c56a401760415499910d78f9320cc1e8be3ec4aa33e1e64deacabd1691f2206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:50.857 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 13 17:40:50 smithi044 bash[105041]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-a 2024-09-13T17:40:51.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.a.service' 2024-09-13T17:40:51.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:51.108 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-13T17:40:51.108 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-13T17:40:51.108 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.d 2024-09-13T17:40:51.607 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:40:51 smithi044 systemd[1]: Stopping Ceph mds.d for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:51.607 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:40:51 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d[71402]: 2024-09-13T17:40:51.427+0000 7fdd5b1d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:51.607 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:40:51 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d[71402]: 2024-09-13T17:40:51.427+0000 7fdd5b1d4640 -1 mds.d *** got signal Terminated *** 2024-09-13T17:40:52.779 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:40:52 smithi044 podman[105163]: 2024-09-13 17:40:52.501310836 +0000 UTC m=+1.166788986 container died 94e9650c1045d50fea442074a2a529219516f92a3343427c84415e08263e599d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-13T17:40:53.107 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:40:52 smithi044 podman[105163]: 2024-09-13 17:40:52.885828312 +0000 UTC m=+1.551306461 container cleanup 94e9650c1045d50fea442074a2a529219516f92a3343427c84415e08263e599d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-13T17:40:53.107 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 13 17:40:52 smithi044 bash[105163]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-d 2024-09-13T17:40:53.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.d.service' 2024-09-13T17:40:53.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:53.288 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-13T17:40:53.288 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-13T17:40:53.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.g 2024-09-13T17:40:53.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:53 smithi044 systemd[1]: Stopping Ceph mds.g for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:53.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:53 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g[72004]: 2024-09-13T17:40:53.615+0000 7fe62b9c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:53.857 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:53 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g[72004]: 2024-09-13T17:40:53.615+0000 7fe62b9c7640 -1 mds.g *** got signal Terminated *** 2024-09-13T17:40:55.884 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:55 smithi044 podman[105284]: 2024-09-13 17:40:55.613145993 +0000 UTC m=+2.104665519 container died 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-13T17:40:56.320 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:55 smithi044 podman[105284]: 2024-09-13 17:40:55.956668804 +0000 UTC m=+2.448188329 container cleanup 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:40:56.320 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:55 smithi044 bash[105284]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g 2024-09-13T17:40:56.320 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 13 17:40:56 smithi044 podman[105295]: 2024-09-13 17:40:56.116159237 +0000 UTC m=+0.497440081 container remove 584279830ebf8291b6c4c74eb2bb6fbeb088195e9e34b4283923517b95e7695a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-13T17:40:56.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.g.service' 2024-09-13T17:40:56.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:56.372 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-13T17:40:56.372 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-13T17:40:56.372 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.j 2024-09-13T17:40:56.606 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:40:56 smithi044 systemd[1]: Stopping Ceph mds.j for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:57.107 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:40:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-j[71058]: 2024-09-13T17:40:56.693+0000 7fd27c2d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:57.107 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:40:56 smithi044 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-j[71058]: 2024-09-13T17:40:56.693+0000 7fd27c2d6640 -1 mds.j *** got signal Terminated *** 2024-09-13T17:40:58.107 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:40:57 smithi044 podman[105406]: 2024-09-13 17:40:57.833025302 +0000 UTC m=+1.233273092 container died caa882e3ae124d9140bbda102e8e2d973760889dfaf877fe0deccad24b18a836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T17:40:58.462 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:40:58 smithi044 podman[105406]: 2024-09-13 17:40:58.202151889 +0000 UTC m=+1.602399678 container cleanup caa882e3ae124d9140bbda102e8e2d973760889dfaf877fe0deccad24b18a836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:40:58.462 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 13 17:40:58 smithi044 bash[105406]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-j 2024-09-13T17:40:58.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.j.service' 2024-09-13T17:40:58.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:40:58.619 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-13T17:40:58.619 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-13T17:40:58.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.b 2024-09-13T17:40:58.947 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:40:58 smithi070 systemd[1]: Stopping Ceph mds.b for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:40:59.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:40:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b[64131]: 2024-09-13T17:40:59.042+0000 7fd46b9be640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:40:59.447 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:40:59 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b[64131]: 2024-09-13T17:40:59.042+0000 7fd46b9be640 -1 mds.b *** got signal Terminated *** 2024-09-13T17:41:00.353 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:41:00 smithi070 podman[70890]: 2024-09-13 17:41:00.085376147 +0000 UTC m=+1.278664821 container died e228f8664847cfb23602484052cbe9d36cd7f81ea50c3ccf1c55a56ac63c21d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-13T17:41:00.697 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:41:00 smithi070 podman[70890]: 2024-09-13 17:41:00.459366895 +0000 UTC m=+1.652655565 container cleanup e228f8664847cfb23602484052cbe9d36cd7f81ea50c3ccf1c55a56ac63c21d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T17:41:00.697 INFO:journalctl@ceph.mds.b.smithi070.stdout:Sep 13 17:41:00 smithi070 bash[70890]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-b 2024-09-13T17:41:00.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.b.service' 2024-09-13T17:41:00.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:00.891 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-13T17:41:00.891 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-13T17:41:00.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.e 2024-09-13T17:41:01.248 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:00 smithi070 systemd[1]: Stopping Ceph mds.e for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:01.697 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e[63518]: 2024-09-13T17:41:01.246+0000 7f23cdbbf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:01.697 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:01 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e[63518]: 2024-09-13T17:41:01.246+0000 7f23cdbbf640 -1 mds.e *** got signal Terminated *** 2024-09-13T17:41:03.406 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 podman[71026]: 2024-09-13 17:41:03.05851564 +0000 UTC m=+1.946574084 container died ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-13T17:41:03.697 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 podman[71026]: 2024-09-13 17:41:03.5129392 +0000 UTC m=+2.400997645 container cleanup ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T17:41:03.697 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 bash[71026]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e 2024-09-13T17:41:03.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.e.service' 2024-09-13T17:41:03.966 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 podman[71038]: 2024-09-13 17:41:03.714158225 +0000 UTC m=+0.651587319 container remove ab98e581e5b3cc350527d83dd0ddd9c643d45352967d1eea6babfcdc986456ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-13T17:41:03.967 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.e.service: Deactivated successfully. 2024-09-13T17:41:03.967 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 systemd[1]: Stopped Ceph mds.e for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:41:03.967 INFO:journalctl@ceph.mds.e.smithi070.stdout:Sep 13 17:41:03 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.e.service: Consumed 1.099s CPU time. 2024-09-13T17:41:03.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:03.979 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-13T17:41:03.979 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-13T17:41:03.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.h 2024-09-13T17:41:04.301 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:04 smithi070 systemd[1]: Stopping Ceph mds.h for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:04.697 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h[63827]: 2024-09-13T17:41:04.299+0000 7fcd3bcf4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:04.697 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:04 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h[63827]: 2024-09-13T17:41:04.299+0000 7fcd3bcf4640 -1 mds.h *** got signal Terminated *** 2024-09-13T17:41:08.426 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 podman[71148]: 2024-09-13 17:41:08.090659758 +0000 UTC m=+3.896717328 container died 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:41:08.693 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 podman[71148]: 2024-09-13 17:41:08.49923628 +0000 UTC m=+4.305293845 container cleanup 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2) 2024-09-13T17:41:08.693 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 bash[71148]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h 2024-09-13T17:41:08.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.h.service' 2024-09-13T17:41:08.947 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 podman[71160]: 2024-09-13 17:41:08.692108128 +0000 UTC m=+0.600473315 container remove 3c9933e75de7d4d923ec8544f7588ffee4fb06569f12503668ccccb35085d2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T17:41:08.947 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-13T17:41:08.947 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 systemd[1]: Stopped Ceph mds.h for ce020790-71f3-11ef-bceb-c7b262605968. 2024-09-13T17:41:08.947 INFO:journalctl@ceph.mds.h.smithi070.stdout:Sep 13 17:41:08 smithi070 systemd[1]: ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.h.service: Consumed 2.042s CPU time. 2024-09-13T17:41:08.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:08.963 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-13T17:41:08.963 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-13T17:41:08.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.k 2024-09-13T17:41:09.287 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:41:09 smithi070 systemd[1]: Stopping Ceph mds.k for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:09.697 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:41:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k[63202]: 2024-09-13T17:41:09.286+0000 7f338ec1c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:09.697 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:41:09 smithi070 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k[63202]: 2024-09-13T17:41:09.286+0000 7f338ec1c640 -1 mds.k *** got signal Terminated *** 2024-09-13T17:41:11.379 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:41:11 smithi070 podman[71267]: 2024-09-13 17:41:11.024780466 +0000 UTC m=+1.840073927 container died fb48a9aeda9f4c589e5ad9d6c6ac61f97e8f045dd896b2ff7fa36ac7edf8ea33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=) 2024-09-13T17:41:11.697 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:41:11 smithi070 podman[71267]: 2024-09-13 17:41:11.477735599 +0000 UTC m=+2.293029058 container cleanup fb48a9aeda9f4c589e5ad9d6c6ac61f97e8f045dd896b2ff7fa36ac7edf8ea33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-13T17:41:11.697 INFO:journalctl@ceph.mds.k.smithi070.stdout:Sep 13 17:41:11 smithi070 bash[71267]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-k 2024-09-13T17:41:11.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.k.service' 2024-09-13T17:41:11.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:11.952 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-13T17:41:11.952 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-13T17:41:11.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.c 2024-09-13T17:41:12.238 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:41:11 smithi100 systemd[1]: Stopping Ceph mds.c for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:12.524 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:41:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c[65592]: 2024-09-13T17:41:12.236+0000 7fb09de86640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:12.524 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:41:12 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c[65592]: 2024-09-13T17:41:12.236+0000 7fb09de86640 -1 mds.c *** got signal Terminated *** 2024-09-13T17:41:16.355 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:41:16 smithi100 podman[73473]: 2024-09-13 17:41:16.082014266 +0000 UTC m=+3.940513849 container died f9613c3f418c1953f1942fd498463b1421d215241ecfcc3aef2a9130049fbe57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:41:16.713 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:41:16 smithi100 podman[73473]: 2024-09-13 17:41:16.453048896 +0000 UTC m=+4.311548464 container cleanup f9613c3f418c1953f1942fd498463b1421d215241ecfcc3aef2a9130049fbe57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-13T17:41:16.713 INFO:journalctl@ceph.mds.c.smithi100.stdout:Sep 13 17:41:16 smithi100 bash[73473]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-c 2024-09-13T17:41:16.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.c.service' 2024-09-13T17:41:16.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:16.914 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-13T17:41:16.914 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-13T17:41:16.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.f 2024-09-13T17:41:17.242 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:16 smithi100 systemd[1]: Stopping Ceph mds.f for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:17.524 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f[65287]: 2024-09-13T17:41:17.240+0000 7fd0bbc0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:17.524 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:17 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f[65287]: 2024-09-13T17:41:17.240+0000 7fd0bbc0d640 -1 mds.f *** got signal Terminated *** 2024-09-13T17:41:20.366 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:20 smithi100 podman[73604]: 2024-09-13 17:41:20.081489129 +0000 UTC m=+2.949913455 container died 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-13T17:41:20.726 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:20 smithi100 podman[73604]: 2024-09-13 17:41:20.365210603 +0000 UTC m=+3.233634938 container cleanup 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-13T17:41:20.726 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:20 smithi100 bash[73604]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f 2024-09-13T17:41:20.726 INFO:journalctl@ceph.mds.f.smithi100.stdout:Sep 13 17:41:20 smithi100 podman[73616]: 2024-09-13 17:41:20.524662125 +0000 UTC m=+0.436589982 container remove 34b7c635ed720c0a2b71d6804a4b930f826b5a6f404c49e6a453df5672d7d2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T17:41:20.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.f.service' 2024-09-13T17:41:20.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:20.771 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-13T17:41:20.772 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-13T17:41:20.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.i 2024-09-13T17:41:21.024 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:41:20 smithi100 systemd[1]: Stopping Ceph mds.i for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:21.524 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:41:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: 2024-09-13T17:41:21.127+0000 7fc5b7c9f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:21.524 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:41:21 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i[65902]: 2024-09-13T17:41:21.127+0000 7fc5b7c9f640 -1 mds.i *** got signal Terminated *** 2024-09-13T17:41:24.274 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:41:24 smithi100 podman[73725]: 2024-09-13 17:41:24.088069983 +0000 UTC m=+3.092722736 container died 54717c131dc7e21ae2d9da18bbe59948cb62ce9710f10de4fa82b844367a91d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-13T17:41:24.730 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:41:24 smithi100 podman[73725]: 2024-09-13 17:41:24.513572826 +0000 UTC m=+3.518225569 container cleanup 54717c131dc7e21ae2d9da18bbe59948cb62ce9710f10de4fa82b844367a91d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-13T17:41:24.730 INFO:journalctl@ceph.mds.i.smithi100.stdout:Sep 13 17:41:24 smithi100 bash[73725]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-i 2024-09-13T17:41:24.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.i.service' 2024-09-13T17:41:24.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:24.974 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-13T17:41:24.974 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-13T17:41:24.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.l 2024-09-13T17:41:25.299 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:25 smithi100 systemd[1]: Stopping Ceph mds.l for ce020790-71f3-11ef-bceb-c7b262605968... 2024-09-13T17:41:25.774 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l[64984]: 2024-09-13T17:41:25.296+0000 7f9ac8872640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T17:41:25.774 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:25 smithi100 ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l[64984]: 2024-09-13T17:41:25.296+0000 7f9ac8872640 -1 mds.l *** got signal Terminated *** 2024-09-13T17:41:28.324 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:28 smithi100 podman[73848]: 2024-09-13 17:41:28.07147392 +0000 UTC m=+2.876891990 container died feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T17:41:28.758 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:28 smithi100 podman[73848]: 2024-09-13 17:41:28.406179599 +0000 UTC m=+3.211597705 container cleanup feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T17:41:28.758 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:28 smithi100 bash[73848]: ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l 2024-09-13T17:41:28.758 INFO:journalctl@ceph.mds.l.smithi100.stdout:Sep 13 17:41:28 smithi100 podman[73860]: 2024-09-13 17:41:28.565497075 +0000 UTC m=+0.490246810 container remove feff28de37ffc3d3fd776f01d69496a890c0ee3d73cae5df080a0da662805016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-ce020790-71f3-11ef-bceb-c7b262605968-mds-l, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-13T17:41:28.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ce020790-71f3-11ef-bceb-c7b262605968@mds.l.service' 2024-09-13T17:41:28.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T17:41:28.812 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-13T17:41:28.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ce020790-71f3-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T17:41:28.973 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:41:31.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ce020790-71f3-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T17:41:31.560 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:41:33.870 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ce020790-71f3-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T17:41:34.023 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:41:36.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:41:36.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:41:36.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T17:41:36.295 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-13T17:41:36.297 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi044/crash 2024-09-13T17:41:36.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash -- . 2024-09-13T17:41:36.343 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T17:41:36.343 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-09-13T17:41:36.346 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi070/crash 2024-09-13T17:41:36.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash -- . 2024-09-13T17:41:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T17:41:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2024-09-13T17:41:36.400 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi100/crash 2024-09-13T17:41:36.401 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash -- . 2024-09-13T17:41:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/ce020790-71f3-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T17:41:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-09-13T17:41:36.439 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-13T17:41:36.439 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ce020790-71f3-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-13T17:41:36.484 INFO:teuthology.orchestra.run.smithi044.stderr:grep: /var/log/ceph/ce020790-71f3-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-13T17:41:36.494 INFO:tasks.cephadm:Compressing logs... 2024-09-13T17:41:36.494 DEBUG:teuthology.orchestra.run.smithi044:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:41:36.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:41:36.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:41:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T17:41:36.523 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T17:41:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T17:41:47.064 INFO:tasks.cephadm:Archiving logs... 2024-09-13T17:41:47.065 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi044/log 2024-09-13T17:41:47.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:41:47.995 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi070/log 2024-09-13T17:41:47.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:41:49.129 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi100/log 2024-09-13T17:41:49.130 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:41:50.595 INFO:tasks.cephadm:Removing cluster... 2024-09-13T17:41:50.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ce020790-71f3-11ef-bceb-c7b262605968 --force 2024-09-13T17:41:50.757 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:41:51.042 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ce020790-71f3-11ef-bceb-c7b262605968 --force 2024-09-13T17:41:51.195 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:41:51.664 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ce020790-71f3-11ef-bceb-c7b262605968 --force 2024-09-13T17:41:51.817 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: ce020790-71f3-11ef-bceb-c7b262605968 2024-09-13T17:41:52.089 INFO:tasks.cephadm:Removing cephadm ... 2024-09-13T17:41:52.089 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T17:41:52.109 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T17:41:52.125 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T17:41:52.140 INFO:tasks.cephadm:Teardown complete 2024-09-13T17:41:52.141 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T17:41:52.152 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T17:41:52.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:41:52.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:41:52.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:41:52.293 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T17:41:52.304 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T17:41:52.304 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:41:52.309 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:41:52.311 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:41:52.323 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:41:52.323 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:41:52.325 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:41:52.327 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:41:52.327 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-13T17:41:52.327 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -357us[ -358us] +/- 36ms 2024-09-13T17:41:52.328 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 +1286us[+1286us] +/- 59ms 2024-09-13T17:41:52.328 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -979us[ -980us] +/- 83ms 2024-09-13T17:41:52.328 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:41:52.330 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:41:52.330 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-09-13T17:41:52.330 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -304us[ -296us] +/- 36ms 2024-09-13T17:41:52.330 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 +1310us[+1310us] +/- 59ms 2024-09-13T17:41:52.330 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 126 -917us[ -779us] +/- 82ms 2024-09-13T17:41:52.330 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:41:52.331 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:41:52.331 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-13T17:41:52.331 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 -344us[ -355us] +/- 36ms 2024-09-13T17:41:52.332 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 +1282us[+1271us] +/- 59ms 2024-09-13T17:41:52.332 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -978us[ -990us] +/- 83ms 2024-09-13T17:41:52.332 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:41:52.332 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T17:41:52.342 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T17:41:52.342 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T17:41:52.370 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:41:52.461 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:41:52.528 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:41:52.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T17:41:52.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:52.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T17:41:52.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:52.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T17:41:52.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:52.684 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T17:41:52.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T17:41:52.739 INFO:teuthology.task.internal:Duration was 1911.166312 seconds 2024-09-13T17:41:52.739 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T17:41:52.772 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T17:41:52.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:41:52.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:41:52.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:41:52.814 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:41:52.815 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:41:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:41:53.320 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T17:41:53.320 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T17:41:53.320 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:41:53.344 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T17:41:53.344 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:41:53.370 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-13T17:41:53.370 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:41:53.398 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T17:41:53.398 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:41:53.401 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:41:53.413 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:41:53.464 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T17:41:53.464 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:41:53.471 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:41:53.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:41:53.908 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T17:41:53.920 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T17:41:53.920 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:41:53.946 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:41:53.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:41:53.999 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T17:41:54.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:41:54.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:41:54.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:41:54.035 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-09-13T17:41:54.037 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-13T17:41:54.062 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-13T17:41:54.082 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:41:54.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:54.104 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:41:54.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:54.118 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:41:54.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:54.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T17:41:54.148 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T17:41:54.149 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi044 2024-09-13T17:41:54.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:41:54.192 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi070 2024-09-13T17:41:54.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:41:54.231 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903881/remote/smithi100 2024-09-13T17:41:54.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:41:54.268 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T17:41:54.269 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:41:54.271 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:41:54.273 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:41:54.318 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T17:41:54.330 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T17:41:54.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T17:41:54.355 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T17:41:54.355 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:41:54.358 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:41:54.360 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:41:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:41 /home/ubuntu/cephtest 2024-09-13T17:41:54.372 INFO:teuthology.orchestra.run.smithi070.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:41 /home/ubuntu/cephtest 2024-09-13T17:41:54.375 INFO:teuthology.orchestra.run.smithi100.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:41 /home/ubuntu/cephtest 2024-09-13T17:41:54.376 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T17:41:54.388 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T17:41:54.457 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 1911.1663115024567 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T17:41:54.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:41:54.545 INFO:teuthology.run:pass